How to Make Things Faster

Chapter 71. Nonlinear Behavior

Programs that queue a lot usually carry more opportunity for performance improvement than people tend to think. A few years ago, I wrote a program to demonstrate a common database application development mistake. To show how badly such a program could abuse a system, I simulated several users running this program all at the same time on a computer with very little CPU capacity. It spent most of its time queued for CPU:

Subroutine	Duration	Count	Mean
queued for CPU	6.819	20,265	0.000 336
other (CPU, disk, etc.)	4.782	50,295	0.000 095
Total	11.601	70,560	0.000 164

The point of my demonstration was that a simple modification to the source code would reduce the number of database calls (the requests that queue for CPU) from 20,000 to just 5,000. What do you think will happen to the “queued for CPU” duration if you were to make that modification?

A reasonable guess is that reducing the call count to 25% of its original value might reduce the duration to 25% of its original value, like this:

Subroutine	Duration	Count	Mean
queued for CPU (predicted)	1.705	5,066	0.000 336
	0.25 × 6.819	0.25 × 20,265

But in reality, reducing the call count to 25% of its original value reduced the duration to just 11% of its original value:

Subroutine	Duration	Count	Mean
queued for CPU (actual)	0.769	5,003	0.000 154

OK, that’s great news, but why did it happen that way?

The prediction was wrong because it didn’t account for the fact that ...