Designing Data-Intensive Applications, 2nd Edition

Chapter 5. Encoding and Evolution

Everything changes and nothing stands still.

Heraclitus of Ephesus, as quoted by Plato in Cratylus (360 BCE)

Applications inevitably change over time. Features are added or modified as new products are launched, user requirements become better understood, or business circumstances change. In Chapter 2 we introduced the idea of evolvability: we should aim to build systems that make it easy to adapt to change (see “Evolvability: Making Change Easy”).

In most cases, a change to an application’s features also requires a change to data that it stores. Perhaps a new field or record type needs to be captured, or existing data needs to be presented in a new way.

The data models we discussed in Chapter 3 have different ways of coping with such change. Relational databases generally assume that all data in the database conforms to one schema. Although that schema can be changed (through schema migrations; i.e., ALTER statements), exactly one schema is in force at any one point in time. By contrast, schema-on-read (“schemaless”) databases don’t enforce a schema, so the database can contain a mixture of older and newer data formats written at different times (see “Schema flexibility in the document model”).

When a data format or schema changes, a corresponding change to application code often needs to happen (e.g., you add a new field to a record, and the application code starts reading and writing that field). However, in a large application, code ...