Machine Learning and Big Data with kdb+/q

CHAPTER 8Parallelisation

By default, a q instance is single-threaded and aims to use only one core. Aside from interrupts to the main loop to run timer jobs or check external client requests, once a function is called, the entire calculation is sequential, and there is no competition for resources as everything runs in a single queue. We may override this default behaviour and enable concurrent processing on multiple CPU cores, by setting a number of slave threads using the command-line argument:

which launches the q process with N slaves. In order to check the number of slaves from within a process, we may run

which returns, in the case of :

From within the process, we may check the number of slaves using the above command, as well as redefine the number of slaves from within the process using \s N, as long as is less than the number of slaves set in the command-line. If we have launched a single-threaded process, we cannot add more slaves dynamically. For reference, the number of ...