Apache Spark 2.x Cookbook

Cache conscious computation

Algorithms and data structures are used to exploit memory hierarchy and enable more cache-aware computation.

CPU caches are small pools of memory that store the data the CPU is going to need next. CPUs have two types of caches: instruction cache and data cache. Data caches are arranged in a hierarchy of L1, L2, and L3:

• L1 cache is the fastest and most expensive cache in a computer. It stores the most critical data and is the first place the CPU looks for information.
• L2 cache is slightly slower than L1 but is still located on the same processor chip. It is the second place the CPU looks for information.
• L3 cache is still slower but is shared by all cores, such as dynamic random-access memory (DRAM).

These can ...