Thread and process pools

As mentioned before, pools are structures designed to hold N objects (threads, processes, and so on). When the usage reaches capacity, no work is assigned to a thread (or process) until one of those currently working becomes available again. Pools, therefore, are a great way to limit the number of threads (or processes) that can be alive at the same time, preventing the system from starving due to resource exhaustion, or the computation time from being affected by too much context switching.

In the following examples, I will be tapping into the concurrent.futures module to use the ThreadPoolExecutor and ProcessPoolExecutor executors. These two classes, use a pool of threads (and processes, respectively), to execute ...

Get Learn Python Programming - Second Edition now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.