Webmaster in a Nutshell, Second Edition

Preprocess Queries and Cache the Results

Do you ever wonder how the network news programs can have a detailed obituary story ready within hours of the death of a celebrity? What looks like superhuman performance is actually preprocessing. The TV networks keep preprepared obituaries of major celebrities, especially seriously ill ones. The networks don’t know when anyone is going to die, but since there are only a limited number of people whose death is considered newsworthy, the networks have the resources to write stories on all of them. The principle here is that the more you limit the input parameters, the fewer possible results there are. A smaller result set means you can do more effective caching of responses. While the point of a CGI is to output different HTML depending on user input and state information, if the number of possible input and state combinations is small, it makes sense to run the CGI for all possible input offline and cache each result in a plain HTML file. As another example, if a CGI can inform you of tomorrow’s weather forecast in 100 cities around the United States, you will certainly get better performance and scalability by regenerating 100 static HTML pages every night than by running a CGI in response to every query.

Even if there are a huge number of possible inputs, if there are a few common combinations of input it makes sense to dynamically cache those. Keep a server-side cache of frequently requested CGI output and have a stub CGI merely return ...