Java Performance, 2nd Edition

Chapter 2. An Approach to Performance Testing

This chapter discusses four principles of getting results from performance testing: test real applications; understand throughput, batching, and response time; understand variability; and test early and often. These principles form the basis of the advice given in later chapters. The science of performance engineering is covered by these principles. Executing performance tests on applications is fine, but without scientific analysis behind those tests, they can too often lead to incorrect or incomplete analysis. This chapter covers how to make sure that testing produces valid analysis.

Many of the examples given in later chapters use a common application that emulates a system of stock prices; that application is also outlined in this chapter.

Test a Real Application

The first principle is that testing should occur on the actual product in the way the product will be used. Roughly speaking, three categories of code can be used for performance testing: microbenchmarks, macrobenchmarks, and mesobenchmarks. Each has its own advantages and disadvantages. The category that includes the actual application will provide the best results.

Microbenchmarks

A microbenchmark is a test designed to measure a small unit of performance in order to decide which of multiple alternate implementations is preferable: the overhead in creating a thread versus using a thread pool, the time to execute one arithmetic algorithm versus an alternate implementation, ...