Hands-On Software Engineering with Python

Local parallel processing

The primary focus for the local parallelization of processing will be on the multiprocessing module. There are a couple of other modules that might be usable for some parallelization efforts (and those will be discussed later), but multiprocessing provides the best combination of flexibility and power with the least potential for restrictions from the Python interpreter or other OS-level interference.

As might be expected from the module's name, multiprocessing provides a class (Process) that facilitates the creation of child processes. It also provides a number of other classes that can be used to make working with child processes easier, including Queue (a multiprocess-aware queue implementation that can be used ...