How to Make Things Faster

Chapter 93. Destructive Testing

Let’s say that you think your system will have to support 1,000 users, so you create a test—a test, of course, with data volumes and traffic intensities that are as realistic as you can make them. And let’s say you run that test, and the system performs just fine—all user experience durations are acceptable. OK, that’s good. Of course. The fact that you have this kind of a test at all is a very good thing.

But what if the test data you’ve generated accidentally makes the conditions of your test easier than the conditions your application will face later in reality? What if the concurrent workload you’ve generated accidentally fails to reveal all of the coherency delays and queueing delays your application will face later in reality? There are lots of ways that a test can underrepresent the harshness your application will face in the real world. This, of course, increases your risk.

You can mitigate some of that risk by testing to destruction. For example, if you think your system will need to support 1,000 users, then don’t just run your user count up to 1,000. Keep increasing the user count until your system breaks. Does your system break down with 1,017 users? Or does it not break until 7,132 users? These are two significantly different risk profiles, and you need to know which one is yours.

The same logic applies to your data. It doesn’t matter whether you think your big table will top out at 25 TB, crank up the table size until your application ...