Python Parallel Programming Cookbook

How to use a process pool

The multiprocessing library provides the Pool class for simple parallel processing tasks. The Pool class has the following methods:

• apply(): It blocks until the result is ready.
• apply_async(): This is a variant of the apply() method, which returns a result object. It is an asynchronous operation that will not lock the main thread until all the child classes are executed.
• map(): This is the parallel equivalent of the map() built-in function. It blocks until the result is ready, this method chops the iterable data in a number of chunks that submits to the process pool as separate tasks.
• map_async(): This is a variant of the map() method, which returns a result object. If a callback is specified, then it should be callable, ...