Advanced Python Programming

The Executor interface

From version 3.2 onward, it is possible to execute Python code in parallel using the Executor interface provided in the concurrent.futures module. We already saw the Executor interface in action in the previous chapter, when we used ThreadPoolExecutor to perform multiple tasks concurrently. In this subsection, we'll demonstrate the usage of the ProcessPoolExecutor class.

ProcessPoolExecutor exposes a very lean interface, at least when compared to the more featureful multiprocessing.Pool. A ProcessPoolExecutor can be instantiated, similar to ThreadPoolExecutor, by passing a number of worker threads using the max_workers argument (by default, max_workers will be the number of CPU cores available). The main methods available ...