97 Things Every Data Engineer Should Know

Chapter 1. A (Book) Case for Eventual Consistency

Denise Koessler Gosnell, PhD

Consider the life of owning a bookstore.¹ At some point, you will want to set up a system that maintains an inventory of all the books in your store.

In the store’s early days, you selected a system that was designed for about 1,000 books at one location. This system updates a book’s record during a customer’s checkout. When a customer approaches the counter to buy a book, your inventory system does the following:

1. Checks your ledgers for inventory details

2. Records the new purchase

3. Updates all the records

4. Returns a receipt to the customer

The receipt confirms both the customer’s purchase and that your inventory system is up-to-date. This type of inventory system requires all of your transactions to have strong consistency. In this sense, strong consistency refers to all accesses to the store’s inventory being processed sequentially and read from the same state in your store’s inventory system.

Well, good news, store owner! The book business is booming, and you are opening multiple stores to address your growing customer base. In this world, how do you maintain your company’s inventory across multiple stores?

Maybe you consider rearchitecting your current system. You decide to have one master register in your first store. Then, all of your other stores will have a new register that connects to the master. ...