Parallel Computer Architecture

Q

Latency Tolerance

In Chapter 1, we saw that while the speed of microprocessors increases by more

than a factor of ten per decade, the access time of commodity memories (DRAMs) is

only halved. Thus, the latency of memory access in terms of processor clock cycles

grows by more than a factor of five in ten years! Multiprocessors greatly exacerbate

the problem. In bus-based systems, latency is increased by snooping. In distributed-

memory systems, the latency of the network, network interface, and endpoint pro-

cessing is added to that of accessing the local memory on the node. Caches help

reduce the frequency of high-latency accesses, but the