Modern Python Standard Library Cookbook

There's more...

multiprocessing.Pool works in nearly the same way as multiprocessing.pool.ThreadPool. In fact, they share a lot of their implementation as one is a subclass of the other.

But there are some major differences that are caused by the underlying technology used. One is based on threads and the other on subprocesses.

The major benefit of using processes is that the Python interpreter lock won't limit their parallelism, and they will be able to actually run in parallel with each other.

On the other side, there is a cost for that. Using processes is both more expensive in startup time (forking a process is usually slower than spawning a thread), and more expensive in terms of memory used, as each process will need to have its own ...