Programming Python, 3rd Edition

Persistence Options in Python

In this chapter, our focus is on persistent data—the kind that outlives a program that creates it. That’s not true by default for objects a script constructs; things like lists, dictionaries, and even class instance objects live in your computer’s memory and are lost as soon as the script ends. To make data live longer, we need to do something special. In Python programming, there are today at least six traditional ways to save information in between program executions:

Flat files

Storing text and bytes

DBM keyed files

Keyed access to strings

Pickled objects

Serializing Python objects to files and streams

Shelve files

Storing pickled Python objects in DBM keyed files

ZODB object databases

Storing Python objects in persistent dictionaries

SQL relational databases

Table-based systems that support queries

In some sense, Python’s interfaces to network-based object transmission protocols such as SOAP, XML-RPC, and CORBA also offer persistence options, but they are beyond the scope of this chapter. Here, our interest is in techniques that allow a program to store its data directly and, usually, on the local machine. Although some database servers may operate on a physically remote machine on a network, this is largely transparent to most of the techniques we’ll study here.

We studied Python’s simple (or “flat”) file interfaces in earnest in Chapter 4, and we have been using them ever since. Python provides standard access to both the stdio filesystem (through the built-in ...