Python Programming in Context, 4th Edition

5.2 Using Files for Large Data Sets

All of the examples in the Introducing the Python Collections chapter used small data sets so we could concentrate on developing and testing the statistical methods needed to analyze the data. We assumed our data items were stored in lists, and we constructed our functions to process those lists accordingly. Now that we have those functions in place, we can turn our attention to using statistical tools to describe larger data sets.

We have already seen that Python provides some powerful collections in which to store and manipulate data. However, as the amount of data gets larger, having the user enter the data interactively requires a substantial amount of effort. Instead, large data sets are usually stored ...