System Design on AWS

Chapter 8. Architectural Designs and Patterns

In this chapter, we dive into the world of architectural design and patterns. These models and patterns form the backbone of modern software systems. We will explore how these designs support scalability, maintainability, and efficient data handling in diverse applications. This chapter will not only equip you with theoretical foundations but also inspire practical applications and innovations in your system design endeavors.

We begin our journey with change data capture (CDC), a pivotal technique in data architecture used for capturing changes made to data in a database. Then, advancing into the realm of asynchronous communication patterns, we discuss the pub/sub architecture, in which publishers emit events without knowledge of the subscribers’ identities. We’ll look at how message brokers and message queues manage data flow across systems. Our focus will be on Apache Kafka, a high-throughput distributed messaging system, and we’ll illustrate its architecture and use cases for handling large-scale data streams.

In distributed systems, managing service interactions is critical. In this chapter, we will explore key architecture patterns for building scalable, resilient, and maintainable systems in modern cloud environments. We’ll compare the choreography (decentralized) and orchestration (centralized) management approaches, examining scenarios where each is most effective. We’ll then move on to big data architectures, covering foundational ...