Message passing between several workers

As mentioned earlier, our goal is to have a structure where there are several processes constantly executing workers from a queue, and if a process finishes executing one worker, then it will pick up another. To do this, we will be utilizing a subclass of Queue called JoinableQueue, which will provide the additional task_done() and join() methods, as described in the following list:

  • task_done(): This method tells the program that the calling JoinableQueue object is complete
  • join(): This method blocks until all items in the calling JoinableQueue object have been processed

Now the goal here, again, is to have a JoinableQueue object holding all the tasks that are to be executed—we will call this the ...

Get Mastering Concurrency in Python now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.