Web Caching

Chapter 12.  Benchmarking Proxy Caches

Benchmarking is the process of measuring the performance of a product or service. Benchmarks are used for many different computer systems and components, such as CPUs, disk drives, Ethernet switches, and databases. A good benchmark must be stressful, reproducible, and meaningful. A test or workload that doesn’t stress the device under test is not very interesting. Reproducibility is important for tests that compare different products or slightly different configurations under the same conditions. A meaningful benchmark is one that accurately predicts the performance of the device or system under real-world conditions.

Benchmarks are used by engineers, marketers, and customers alike. Product engineers use benchmarks to evaluate design choices and overall system performance. A good benchmark identifies bottlenecks and tells them if changing a certain component improves or worsens the product’s performance. Benchmarking results are often used in marketing literature, especially when one company claims to have a superior product. Even when a particular product does poorly, marketing departments find some way to put a positive spin on the results.

Customers use benchmarking results to make buying decisions. Good benchmarks enable customers to compare different products with each other, find the one that best suits their needs, and understand the price versus performance tradeoffs. Without published, audited benchmark results, customers have no ...