NGINX Cookbook, 2nd Edition

Chapter 4. Massively Scalable Content Caching

4.0 Introduction

Caching accelerates content serving by storing request responses to be served again in the future. Content caching reduces load to upstream servers, caching the full response rather than running computations and queries again for the same request. Caching increases performance and reduces load, meaning you can serve faster with fewer resources. The scaling and distribution of caching servers in strategic locations can have a dramatic effect on user experience. It’s optimal to host content close to the consumer for the best performance. You can also cache your content close to your users. This is the pattern of content delivery networks, or CDNs. With NGINX you’re able to cache your content wherever you can place an NGINX server, effectively enabling you to create your own CDN. With NGINX caching, you’re also able to passively cache and serve cached responses in the event of an upstream failure. Caching features are only available within the http context.

4.1 Caching Zones

Problem

You need to cache content and need to define where the cache is stored.

Solution

Use the proxy_cache_path directive to define shared memory-cache zones and a location for the content:

proxy_cache_path /var/nginx/cache 
                 keys_zone=CACHE:60m 
                 levels=1:2 
                 inactive=3h 
                 max_size=20g;
proxy_cache CACHE;

The cache definition example creates a directory for cached responses on the filesystem at /var/nginx/cache and creates a shared memory space named CACHE with 60 MB of memory. This example sets the directory structure levels, defines the release of cached responses after they have not been requested in 3 hours, and defines a maximum size of the cache of 20 GB. The proxy_cache directive informs a particular context to use the cache zone. The proxy_cache_path is valid in the http context and the proxy_cache directive is valid in the http, server, and location contexts.

Discussion

To configure caching in NGINX, it’s necessary to declare a path and zone to be used. A cache zone in NGINX is created with the directive proxy_cache_path. The proxy_cache_path designates a location to store the cached information and a shared memory space to store active keys and response metadata. Optional parameters to this directive provide more control over how the cache is maintained and accessed. The levels parameter defines how the file structure is created. The value is a colon-separated value that declares the length of subdirectory names, with a maximum of three levels. NGINX caches are based on the cache key, which is a hashed value. NGINX then stores the result in the file structure provided, using the cache key as a file path and breaking up directories based on the levels value. The inactive parameter allows for control over the length of time a cache item will be hosted after its last use. The size of the cache is also configurable with the use of the max_size parameter. Other parameters relate to the cache-loading process, which loads the cache keys into the shared memory zone from the files cached on disk.

4.2 Cache Locking

Problem

You don’t want NGINX to proxy requests that are currently being written to cache to an upstream server.

Solution

Use the proxy_cache_lock directive to ensure only one request is able to write to the cache at a time, where subsequent requests will wait for the response to be written:

proxy_cache_lock on;
proxy_cache_lock_age 10s;
proxy_cache_lock_timeout 3s;

Discussion

The proxy_cache_lock directive instructs NGINX to hold requests, destined for a cached element, that are currently being populated. The proxied request that is populating the cache is limited in the amount of time it has before another request attempts to populate the element, defined by the proxy_cache_lock_age directive, which defaults to 5 seconds. NGINX can also allow requests that have been waiting a specified amount of time to pass through to the proxied server, which will not attempt to populate the cache by use of the proxy_cache_lock_timeout directive, which also defaults to 5 seconds. You can think of proxy_cache_lock_age and proxy_cache_lock_timeout as “You’re taking too long, I’ll populate the cache for you,” and “You’re taking too long for me to wait, I’m going to get what I need and let you populate the cache in your own time,” respectively.

4.3 Caching Hash Keys

Problem

You need to control how your content is cached and retrieved.

Solution

Use the proxy_cache_key directive along with variables to define what constitutes a cache hit or miss:

proxy_cache_key "$host$request_uri $cookie_user";

This cache hash key will instruct NGINX to cache pages based on the host and URI being requested, as well as a cookie that defines the user. With this you can cache dynamic pages without serving content that was generated for a different user.

Discussion

The default proxy_cache_key, which will fit most use cases, is "$scheme$proxy_host$request_uri". The variables used include the scheme, HTTP or HTTPS, the proxy_host, where the request is being sent, and the request URI. Altogether, this reflects the URL that NGINX is proxying the request to. You may find that there are many other factors that define a unique request per application—such as request arguments, headers, session identifiers, and so on—to which you’ll want to create your own hash key.¹

Selecting a good hash key is very important and should be thought through with understanding of the application. Selecting a cache key for static content is typically pretty straightforward; using the hostname and URI will suffice. Selecting a cache key for fairly dynamic content like pages for a dashboard application requires more knowledge around how users interact with the application and the degree of variance between user experiences. Due to security concerns, you may not want to present cached data from one user to another without fully understanding the context. The proxy_cache_key directive configures the string to be hashed for the cache key. The proxy_cache_key can be set in the context of http, server, and location blocks, providing flexible control on how requests are cached.

4.4 Cache Bypass

Problem

You need the ability to bypass the caching.

Solution

Use the proxy_cache_bypass directive with a nonempty or nonzero value. One way to do this dynamically is with a variable set to anything other than an empty string or 0 within a location block that you do not want cached:

proxy_cache_bypass $http_cache_bypass;

The configuration tells NGINX to bypass the cache if the HTTP request header named cache_bypass is set to any value that is not 0. This example uses a header as the variable to determine if caching should be bypassed—the client would need to specifically set this header for their request.

Discussion

There are a number of scenarios that demand that the request is not cached. For this, NGINX exposes a proxy_cache_bypass directive so that when the value is nonempty or nonzero, the request will be sent to an upstream server rather than be pulled from the cache. Different needs and scenarios for bypassing cache will be dictated by your application’s use case. Techniques for bypassing cache can be as simple as using a request or response header, or as intricate as multiple map blocks working together.

You may want to bypass the cache for many reasons, such as troubleshooting or debugging. Reproducing issues can be hard if you’re consistently pulling cached pages or if your cache key is specific to a user identifier. Having the ability to bypass the cache is vital. Options include, but are not limited to, bypassing the cache when a particular cookie, header, or request argument is set. You can also turn off the cache completely for a given context, such as a location block, by setting proxy_cache off;.

4.5 Cache Performance

Problem

You need to increase performance by caching on the client side.

Solution

Use client-side cache-control headers:

location ~* \.(css|js)$ {
  expires 1y;
  add_header Cache-Control "public";
}

This location block specifies that the client can cache the content of CSS and JavaScript files. The expires directive instructs the client that their cached resource will no longer be valid after one year. The add_header directive adds the HTTP response header Cache-Control to the response, with a value of public, which allows any caching server along the way to cache the resource. If we specify private, only the client is allowed to cache the value.

Discussion

Cache performance has many factors, disk speed being high on the list. There are many things within the NGINX configuration that you can do to assist with cache performance. One option is to set headers of the response in such a way that the client actually caches the response and does not make the request to NGINX at all, but simply serves it from its own cache.

4.6 Cache Purging with NGINX Plus

Problem

You need to invalidate an object from the cache.

Solution

Use the purge feature of NGINX Plus, the proxy_cache_purge directive, and a nonempty or zero-value variable:

map $request_method $purge_method {
    PURGE 1;
    default 0;
}
server {
    # ...
    location / {
        # ...
        proxy_cache_purge $purge_method;
    }
}

In this example, the cache for a particular object will be purged if it’s requested with a method of PURGE. The following is a curl example of purging the cache of a file named main.js:

$ curl -XPURGE localhost/main.js

Discussion

A common way to handle static files is to put a hash of the file in the filename. This ensures that as you roll out new code and content, your CDN recognizes it as a new file because the URI has changed. However, this does not exactly work for dynamic content to which you’ve set cache keys that don’t fit this model. In every caching scenario, you must have a way to purge the cache. NGINX Plus has provided a simple method of purging cached responses. The proxy_cache_purge directive, when passed a nonzero or nonempty value, will purge the cached items matching the request. A simple way to set up purging is by mapping the request method for PURGE. However, you may want to use this in conjunction with the geo_ip module or simple authentication to ensure that not just anyone can purge your precious cache items. NGINX has also allowed for the use of *, which will purge cache items that match a common URI prefix. To use wildcards, you will need to configure your proxy_cache_path directive with the purger=on argument.

4.7 Cache Slicing

Problem

You need to increase caching efficiency by segmenting the file into fragments.

Solution

Use the NGINX slice directive and its embedded variables to divide the cache result into fragments:

proxy_cache_path /tmp/mycache keys_zone=mycache:10m;
server {
    # ...
    proxy_cache mycache;
    slice 1m;
    proxy_cache_key $host$uri$is_args$args$slice_range;
    proxy_set_header Range $slice_range;
    proxy_http_version 1.1;
    proxy_cache_valid  200 206 1h;
 
    location / {
        proxy_pass http://origin:80;
    }
}

Discussion

This configuration defines a cache zone and enables it for the server. The slice directive is then used to instruct NGINX to slice the response into 1 MB file segments. The cache files are stored according to the proxy_cache_key directive. Note the use of the embedded variable named slice_range. That same variable is used as a header when making the request to the origin, and that request HTTP version is upgraded to HTTP/1.1 because 1.0 does not support byte-range requests. The cache validity is set for response codes of 200 or 206 for 1 hour, and then the location and origins are defined.

The Cache Slice module was developed for delivery of HTML5 video, which uses byte-range requests to pseudostream content to the browser. By default, NGINX is able to serve byte-range requests from its cache. If a request for a byte range is made for uncached content, NGINX requests the entire file from the origin. When you use the Cache Slice module, NGINX requests only the necessary segments from the origin. Range requests that are larger than the slice size, including the entire file, trigger subrequests for each of the required segments, and then those segments are cached. When all of the segments are cached, the response is assembled and sent to the client, enabling NGINX to more efficiently cache and serve content requested in ranges.

Cache Slice module should be used only on large files that do not change. NGINX validates the ETag each time it receives a segment from the origin. If the ETag on the origin changes, NGINX aborts the cache population of the segment because the cache is no longer valid. If the content does change and the file is smaller, or your origin can handle load spikes during the cache fill process, it’s better to use the Cache Lock module described in the blog listed in the following Also See section. This module is not built by default, and needs to be enabled by the --with-http_slice_module configuration when building NGINX.

Also See

“Smart and Efficient Byte-Range Caching with NGINX & NGINX Plus”

¹ Any combination of text or variables exposed to NGINX can be used to form a cache key. A list of variables is available in NGINX.