Mastering Python - Second Edition

14

Multiprocessing – When a Single CPU Core Is Not Enough

In the previous chapter, we discussed asyncio, which can use the threading and multiprocessing modules but mainly uses single-thread/single-process parallelization. In this chapter, we will see how we can directly use multiple threads or processes to speed up our code and what caveats to keep in mind. This chapter can actually be seen as an extension to the list of performance tips.

The threading module makes it possible to run code in parallel in a single process. This makes threading very useful for I/O-related tasks such as reading/writing files or network communication, but a useless option for slow and heavy calculations, which is where the multiprocessing module shines.

With the ...