O'Reilly logo

Python Unlocked by Arun Tigeraniya

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Using multiple processes

Key 2: Churning CPU-intensive tasks.

Multiple processes are helpful to fully utilize all CPU cores. It helps in CPU-intensive work as tasks are run in separate processes, and there is no GIL between actual working processes. The setup and communication cost between processes is higher than threads. In the following code section, proc_iotask, proc_cputask are processes that run them for various inputs:

import time from tasker import cputask, iotask from random import randint import multiprocessing,random,string def proc_iotask(i,outq): i[1] = iotask(i[0]) outq.put(i) def proc_cputask(i,outq): res = cputask(i[0]) outq.put((i[0],res)) stats = {} def process(rep, case=None): stats.clear() inputs = [[randint(1, 1000), None] ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required