Web Performance Daybook Volume 2

Chapter 1. WebPagetest Internals

Patrick Meenan

I thought I’d take the opportunity this year to give a little bit of visibility into how WebPagetest gathers the performance data from browsers. Other tools on windows use similar techniques but the information here may not be representative of how other tools work.

First off, it helps to understand the networking stack on Windows from a browser’s perspective (Figure 1-1).

Figure 1-1. Windows networking stack from browser’s perspective

It doesn’t matter what the browser is, if it runs on Windows, the architecture pretty much has to look like the diagram above where all of the communications go through the Windows socket APIs (for that matter, just about any application that talks TCP/IP on Windows looks like the picture above).

Function Interception

The key to how WebPagetest works is its ability to intercept arbitrary function calls and inspect or alter the request or response before passing it on to the original implementation (or choosing not to pass it on at all). Luckily someone else did most of the heavy lifting and provided a nice open source library (http://newgre.net/ncodehook) that can take care of the details for you but it basically works like this:

• Find the target function in memory (trivial if it is exported from a dll).

• Copy the first several bytes from the function (making sure to keep x86 instructions intact).

• Overwrite the function ...