Web Caching

How to Benchmark a Proxy Cache

Proxy cache benchmarking is tricky because many systems are involved, and many things can go wrong. To ensure we’re really measuring the proxy’s performance, we need to eliminate other devices and systems as potential bottlenecks and sources of uncertainty. For example, all computer systems used to drive the benchmark should be identically configured. Also, we need to test the networking equipment to guarantee that it’s up to the task.

Configure Systems

The first step is to select and configure a number of systems for use as clients and servers. The number of machines you need depends on the total throughput you intend to achieve. For Web Polygraph, you can plan on 400–500 requests per second for each client-server pair. Other benchmarks may have different characteristics. Later, I’ll describe how to run a test that proves your systems can adequately generate the load.

Benchmarking systems should be dedicated to their task. You don’t want other processes running that can interfere with the measurements. For example, a busy server running on the same machine can use up significant amounts of bandwidth. Even worse, a runaway program can use significant amounts of memory or CPU time, starving the benchmarking processes and adversely affecting your results.

Continuing along these lines, any unneeded services or processes should be disabled. For example, don’t run lpd, cron, sendmail, portmap, or even inetd. The standard FreeBSD installation, as an example, ...