Release It!, 2nd Edition

Create Back Pressure

Every performance problem starts with a queue backing up somewhere. Maybe it’s a socket’s listen queue. Maybe it’s the OS’s run queue or the databases I/O queue.

If a queue is unbounded, it can consume all available memory. As the queue grows, the time it takes for a piece of work to get all the way through it grows too. (See Little’s law.^[16]) So as a queue’s length reaches toward infinity, response time also heads toward infinity. We really don’t want unbounded queues in our systems.

On the other hand, if the queue is bounded, we have to decide what to do when it’s full and a producer tries to stuff one more thing into it. Even if the object is wafer-thin, the queue has no space.

We really have only a few options: ...