How to do it...

There are a number of questions we need to answer to decide if we need a library data collection instead of one of the built-in collections:

  1. Is the structure a buffer between the producer and the consumer? Does some part of the algorithm produce data items and another part consume the data items?


A common naive approach is for the producer to accumulate items in a list, and then the consumer processes the items from the list. This approach will tend to build a large intermediate data structure. A change in focus can interleave production and consumption, reducing the amount of memory used:

  • A queue is used for First-In-First-Out (FIFO) processing. Items are inserted at one end and consumed from the other end. We can use ...

Get Modern Python Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.