Web Caching

Appendix A.  Analysis of Production Cache Trace Data

In this appendix, we’ll look at some interesting characteristics of web traffic, such as reply size distributions, HTTP headers, and expiration times. Such data is useful for a number of reasons. First, the information in this appendix backs up some of the statements I made earlier in the book. For example, when I said that small files are more more popular than large ones, I wasn’t just making that up. Second, this data can help you make decisions regarding your own caching proxies. The hit ratio analysis demonstrates how increasing your cache size may result in higher hit ratios.

For these analyses, I use data from two different sources. One is the NLANR/IRCache project, consisting of nine caches I maintain throughout the U.S.^[33] The other is a proxy cache located at a U.S. university, which I’ll call Anon-U. All data comes from production Squid caches with real users.

I use the IRCache data for most analyses because it is significantly larger and includes more information. The IRCache set includes client access logs, cache “store” logs, and HTTP header logs. The access logs are from March 5–25, 2000 and contain 216 million responses. The store logs are from March 8–25, 2000, and contain 71 million entries. The header logs are from April 2–29, 2000, and contain 268 million request and response entries.

The IRCache proxies are unique in certain ways that can skew the data. In other words, the data collected from these proxies ...