Python Cookbook, 3rd Edition

Chapter 4. Iterators and Generators

Iteration is one of Python’s strongest features. At a high level, you might simply view iteration as a way to process items in a sequence. However, there is so much more that is possible, such as creating your own iterator objects, applying useful iteration patterns in the itertools module, making generator functions, and so forth. This chapter aims to address common problems involving iteration.

4.1. Manually Consuming an Iterator

Problem

You need to process items in an iterable, but for whatever reason, you can’t or don’t want to use a for loop.

Solution

To manually consume an iterable, use the next() function and write your code to catch the StopIteration exception. For example, this example manually reads lines from a file:

with open('/etc/passwd') as f:
    try:
        while True:
            line = next(f)
            print(line, end='')
    except StopIteration:
        pass

Normally, StopIteration is used to signal the end of iteration. However, if you’re using next() manually (as shown), you can also instruct it to return a terminating value, such as None, instead. For example:

with open('/etc/passwd') as f:
     while True:
         line = next(f, None)
         if line is None:
             break
         print(line, end='')

Discussion

In most cases, the for statement is used to consume an iterable. However, every now and then, a problem calls for more precise control over the underlying iteration mechanism. Thus, it is useful to know what actually happens.

The following interactive example illustrates the basic mechanics of what happens ...