Web Caching

Throughput and Load Sharing

A cache cluster with load sharing can improve both throughput and reliability. Throughput is increased because N caches can handle more traffic than just one. Reliability is increased because when one cache fails, the other cluster members absorb the increased load.

The really cheap way to implement load sharing is with the DNS. You simply list the IP addresses of all members under a single host name. Most DNS resolver libraries change the order of the addresses for each lookup. Thus, requests are sent to the caches in a round-robin fashion. This technique provides very rudimentary failover when one cache goes down. When a browser cannot connect to one of the IP addresses, it should try the next one in the list. Unfortunately, this can take a long time (up to two minutes), and browsers don’t usually remember which addresses don’t work.

A more robust approach, if you can afford it, is to use a layer four switch or dedicated load balancing product (see Table 5-1). These products have a number of load sharing algorithms (see Section 5.2.2) and generally do a better job of spreading the load among cluster members. Furthermore, users are less likely to experience delays or other problems if one member fails.

You may notice that I am intentionally avoiding the word balance. Some of the load sharing techniques may not distribute the load evenly among caches. The DNS-based approach is difficult to predict and control. One or two really busy clients could send ...