Deciphering Data Architectures

Chapter 7. Approaches to Design

This chapter explores design approaches to handling and organizing data and how these methods help build powerful, adaptable, and reliable systems for data management. In simple terms, we’ll learn about the strategies that help us decide how data should be stored, processed, and accessed to best enhance the speed and dependability of our data systems.

To clarify, let’s distinguish between data design and data modeling, which is the subject of Chapter 8. Think of data design as like building a city. It’s about deciding where the buildings go, which roads connect different parts of the city, and how traffic flows. On the other hand, data modeling is more like designing individual buildings—it’s about arranging rooms, deciding how they connect, and defining the purpose of each room.

In this chapter, we’ll look at different types of data designs. We’ll compare and contrast methods like OLTP and OLAP, which are essentially different ways of processing and analyzing data. We’ll also explore concepts like SMP and MPP, which are strategies to process data more efficiently. Then, we’ll learn about Lambda and Kappa architectures, which are blueprints for handling large amounts of data in real time. Lastly, we’ll talk about an approach called polyglot persistence, which allows us to use different types of data storage technologies within the same application.

My aim here isn’t to push one approach as the best solution for every situation but to help you understand ...