Expert Python Programming - Third Edition

Releasing GIL

We have already mentioned that extensions can be a way to bypass Python's GIL. It is a famous limitation of the CPython implementation that only one thread at a time can execute the Python code. While multiprocessing is the suggested approach to circumvent this problem (see Chapter 15, Concurrency), it may not be a good solution for some highly parallelizable algorithms, due to the resource overhead of running additional processes.

Because extensions are mostly used in cases where a bigger part of the work is performed in pure C without any calls to Python/C API, it is possible (even advisable) to release GIL in some application sections while still doing some data processing. Thanks to this, you can still benefit from having ...