System Design on AWS

Chapter 4. Caching Policies and Strategies

In computing, a cache is a component or mechanism used to store frequently accessed data or instructions closer to the computing unit, reducing the latency of retrieving the information from slower storage or external resources. In traditional hardware, caches are typically implemented as high-speed, small-capacity memories located closer to the CPU than the main memory. The goal is to improve overall system performance by reducing the time required to access data or instructions.

The concept of caching revolves around the principle of locality, which suggests that programs tend to access a relatively small portion of their data or instructions repeatedly. By storing this frequently accessed information in a cache, subsequent access to the same data can be served faster, resulting in improved performance.

When data is accessed from a cache, there are two possible outcomes: cache hit and cache miss. A cache hit occurs when the requested data is found in the cache, allowing for fast retrieval without accessing the slower main memory or external resources. A cache miss happens when the requested data is not present in the cache, requiring the system to fetch the data from the main memory or external storage. Cache hit rates measure the effectiveness of the cache in serving requests without needing to access slower external storage, while cache miss rates indicate how often the cache fails to serve requested data.

This chapter will cover ...