High Performance Python, 3rd Edition

Chapter 5. Iterators and Generators

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the author’s raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 5th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the editor at [e-mail].

Questions You’ll Be Able to Answer After This Chapter

• How do generators save memory?

• When is the best time to use a generator?

• How can I use itertools to create complex generator workflows?

• When is lazy evaluation beneficial, and when is it not?

When many people with experience in another language start learning Python, they are taken aback by the difference in for loop notation. That is to say, instead of writing

# Other languages
for (i=0; i<N; i++) {
    do_work(i);
}

they are introduced to a new function called range:

# Python
for i in range(N):
    do_work(i)

*TODO for Micha from Ian, a reader filed errata noting that range is not a generator but a range type https://docs.python.org/3/library/stdtypes.html#typesseq-range and suggests talking of laziness instead, they quoted https://stackoverflow.com/questions/13092267/if-range-is-a-generator-in-python-3-3-why-can-i-not-call-next-on-a-range ...