System Design on AWS

Chapter 5. Load Balancing Approaches and Techniques

In the quest for scalability and to ensure uninterrupted availability of services, organizations have adopted horizontal and vertical scaling techniques, as discussed in Chapter 1. These methods have addressed some of the scalability challenges, but they do not guarantee continuous service availability. The rapid growth of the internet and the need for modern high-traffic websites to handle hundreds of thousands of concurrent requests necessitate a more robust approach.

When a single server is responsible for handling all incoming requests, it becomes overloaded, which can lead to system failures. To overcome this limitation, we horizontally scale the system. Then, load balancing comes into play to efficiently distribute incoming network traffic across the group of backend servers, often referred to as a server pool. This is much like starting up a company on your own and later hiring more people (and distributing the tasks among them) to scale the business.

In this chapter, we will start with understanding different networking components for managing traffic and improving performance. We will then focus on the concept of load balancers (LBs), which play a crucial role in achieving efficient load distribution and ensuring high availability of services. We will delve into the mechanisms of load balancing as well as the various load balancing algorithms employed and their respective strengths and weaknesses. Understanding LBs and ...