How to Make Things Faster

Chapter 47. Coherency Delay

Queueing degrades response times as your traffic intensity increases. A second type of delay is called coherency delay. Coherency delay is the duration required to make data consistent, or cache coherent, by virtue of point-to-point exchange of data between distributed resources.¹

Imagine a distributed system that runs on two separate computers (nodes). Each node has its own master cache. But when an application on node A requests a page from its cache, it has to execute extra instructions to check whether there’s a newer copy of the same page in the node B cache. If there is, then node A will need to fetch a copy of that page from node B. The extra instructions required to maintain cache coherency lengthen the service time of the page access.

Coherency delays can be particularly corrosive. Whereas queueing delays degrade only the Q component of R = Q + S, coherency delays degrade the S component, which creates two problems. Obviously making S bigger will make R = anything + S the same amount bigger. But, worse than that, increasing S increases traffic intensity, which increases Q as well. Coherency delays thus affect response times at high loads by more than just the service time increase.

On the bright side, if you have a lot of coherency delay, then eliminating even just a little bit of it may significantly improve the performance of your system. ...