Efficient C++ Performance Programming Techniques

Cache Effects

A cache provides more than just fast access for previously accessed data; it also provides a small prefetch region around that previously accessed data. Caches fetch and manage lines of data. A typical cache line consists of 32 bytes of data aligned on a 32-byte boundary. Typically caches lack the capability to manage anything other than complete lines. This means that if an instruction accesses a single byte of data at address 100, then the cache will load bytes 96–127 into a line. The initial access to address 100 may well result in a cache miss and cause the processor to stall for dozens of cycles while the desired data is fetched from memory (or half a dozen cycles if it is fetched from another level of the cache), but subsequent ...