Web Caching

Disk Space

When buying or building a proxy cache, it is important to have enough disk space. If your disk size is too small, then your cache replaces valuable objects that otherwise would result in cache hits. It’s okay to have too much disk space, but after some point, adding more space does not significantly increase your hit ratio. Figuring out the right amount of disk space is complicated because you must consider a number of parameters and overheads. The following advice should help you get started.

As a rule of thumb, your cache should take at least three days to fill up. This magic number of three days comes from empirical observations and analysis of real Web traffic. It is essentially the average time that particular web objects remain popular and valuable. To find out how much disk space we can fill in three days, we need to perform a number of calculations, beginning with the rate that HTTP traffic enters your organization’s network.

Ideally, you already have a sense of how much HTTP traffic comes into your network. We’re looking for a number with units of bytes per second, averaged over a 24-hour period. Any network measurement tool should be able to give you a breakdown by port number. Here, we’re mostly interested in port 80 traffic, although there is likely to be a small amount of HTTP traffic on other ports as well. If you have no idea, you can use your network connection speed as an upper limit. For example, lets say that your company has two T1 connections to ...