How to Make Things Faster

Chapter 41. Queueing Delay

Why would “block read” performance be fantastic early in the month, and then horrible later in the month? It’s the same reason that riding Space Mountain® takes 10 minutes (yay!) when the park first opens, and then 90 minutes (ugh!) later that same morning. Even on healthy systems where everything is functioning properly, the more popular a resource gets, the longer you have to wait for it. It works that way for amusement parks, airport ticket counters, restaurants, highways, and—of course—computer components. It’s unavoidable.

Here’s how it happens. Let’s go back to Kevin’s story. Imagine that you are one of Kevin’s customers, trying to grab your online invoice in the latter half of a month when the system is starting to get busy. As that Create Invoice program runs, it reaches the point in the code where it makes a “block read” call. On a lightly loaded system, that call would take one service time to complete.¹ But the SSD array can’t start servicing your request right away, because it’s already busy working on another request. In fact, there are four “block read” calls ahead of you in the queue: one call that’s halfway done, and three others awaiting their turn.

So, your call gets in line. And waits.

Half a service time later, the call that was halfway done is finished. Three more service times later, the other three calls that were ahead of you are finished, and now it’s your turn. One more service time later, for a total of 4.5 service times, your ...