O'Reilly logo
live online training icon Live Online training

Lazy Looping in Python

Making and using generators and iterators

Topic: Programming
Trey Hunner

When processing large amounts of data in Python, we often reach for lists. Unfortunately, processing data using large lists results in ugly code that can be memory inefficient and slow. Python's solution to this problem is lazy looping using generators and iterators.

Expert Trey Hunner walks you through a number of lazy looping techniques that will help you write more efficient and more readable Python code. You’ll practice creating generators, playing with iterators, and using them both to drastically restructure your code in a more descriptive, data-centric way. You’ll also get hands-on experience with Python's various lazy looping constructs. Join in to gain a greater appreciation for how looping works under the hood in Python.

What you'll learn-and how you can apply it

By the end of this live online course, you’ll understand:

  • How to break up giant for loops into smaller, more readable chunks
  • When and where lazy looping techniques can be used to conserve system resources
  • How to compose generators and iterators to make readable, descriptive code

And you’ll be able to:

  • Embrace Python’s inherent laziness by working with Python’s built-in iterators directly
  • Create your own iterators for working with hundreds or millions of lines, rows, and records
  • Write readable and efficient looping code that works within the limitations of Python’s iterators

This training course is for you because...

  • You’re an experienced Python programmer.
  • You work with files or database queries with thousands of lines and rows.
  • You want to write looping logic that’s both readable and efficient.

Prerequisites

  • A computer with Python 3.7+ installed
  • A working knowledge of Python's for loops and lists
  • Familiarity with list comprehensions (useful but not required)

Recommended follow-up:

About your instructor

  • Trey Hunner helps software teams level up their Python skills through on-site Python training workshops and courses. Trey speaks at Python conferences, blogs about Python frequently, and runs Python Morsels, an online Python education service for individuals and small teams.

    Trey was formerly a director at the Python Software Foundation.

Schedule

The timeframes are only estimates and may vary according to how the class is progressing

Comprehensions (35 minutes)

  • Presentation: Comprehensions as a special-purpose tool for building up new lists; refactoring complex loops
  • Hands-on exercises: Refactor for loops into list comprehensions
  • Q&A

Break (5 minutes)

Generator expressions (30 minutes)

  • Presentation: Generator expressions as a tool for saving memory and time and avoiding making new data structures
  • Hands-on exercises: Refactor looping code to use generator expressions rather than building up new lists just to loop over them once
  • Q&A

Generator functions (40 minutes)

  • Presentation: Demo of code that loops over a file in multiline chunks; using a generator function to lazily loop over a file in multiline chunks in a memory-efficient way
  • Hands-on exercises: Create generator functions to save time and memory
  • Q&A

Break (5 minutes)

The iterator protocol (35 minutes)

  • Presentation: How generators (and Python’s for loops) work under the hood
  • Hands-on exercise: Work with the many iterator-powered looping utilities built into Python and create your own iterators
  • Q&A

Lazy looping helpers (30 minutes)

  • Presentation: Using the helpers in Python’s itertools module to use iterators and generators to loop efficiently
  • Exercise: Use the various lazy looping helpers in the itertools module to complement your efficient generator and iterator objects
  • Q&A