How to do it...

There are a number of questions we need to answer to decide if we need a library data collection instead of one of the built-in collections:

  1. Is the structure a buffer between the producer and the consumer? Does some part of the algorithm produce data items and another part consume the data items?

A common naive approach is for the producer to accumulate items in a list, and then the consumer processes the items from the list. This approach will tend to build a large intermediate data structure. A change in focus can interleave production and consumption, reducing the amount of memory used:

  • A queue is used for First-In-First-Out (FIFO) processing. Items are inserted at one end and consumed from the other end. We can use ...

Get Modern Python Cookbook now with O’Reilly online learning.

O’Reilly members experience live online training, plus books, videos, and digital content from 200+ publishers.