NGINX Cookbook, 3rd Edition

Chapter 4. Massively Scalable Content Caching

4.0 Introduction

Caching accelerates content serving by storing responses to be served again in the future. By serving from its cache, NGINX reduces load on upstream servers by offloading them of expensive, repetitive work. Caching increases performance and reduces load, meaning you can serve faster with fewer resources. Caching also reduces the time and bandwidth it takes to serve resources.

The scaling and distribution of caching servers in strategic locations can have a dramatic effect on user experience. It’s optimal to host content close to the consumer for the best performance. You can also cache your content close to your users. This is the pattern of content delivery networks, or CDNs. With NGINX you’re able to cache your content wherever you can place an NGINX server, effectively enabling you to create your own CDN. With NGINX caching, you’re also able to passively cache and serve cached responses in the event of an upstream failure. Caching features are only available within the http context. This chapter will cover NGINX’s caching and content delivery capabilities.

4.1 Caching Zones

Problem

You need to cache content and need to define where the cache is stored.

Solution

Use the proxy_cache_path directive to define shared memory-cache zones and a location for the content:

proxy_cache_path /var/nginx/cache 
                 keys_zone=main_content:60m 
                 levels=1:2 
                 inactive=3h 
                 max_size=20g
                 min_free=500m;
proxy_cache CACHE;

The cache definition example creates a directory for cached responses on the filesystem at /var/nginx/cache and creates a shared memory space named main_content with 60 MB of memory. This example sets the directory structure levels, defines the eviction of cached responses after they have not been requested in three hours, and defines a maximum size of the cache of 20 GB. The min_free parameter instructs NGINX on how much disk space to keep free of the max_size before evicting cached resources. The proxy_cache directive informs a particular context to use the cache zone. The proxy_cache_path is valid in the http context, and the proxy_cache directive is valid in the http, server, and location contexts.

Discussion

To configure caching in NGINX, it’s necessary to declare a path and zone to be used. A cache zone in NGINX is created with the proxy_cache_path directive. The proxy_cache_path directive designates a location to store the cached information and a shared memory space to store active keys and response metadata. Optional parameters to this directive provide more control over how the cache is maintained and accessed.

The levels parameter defines how the directory structure is created. The value is a colon-separated list of numbers that defines the length of successive subdirectory names. Deeper structures help to avoid too many cached files appearing in a single directory. NGINX then stores the result in the file structure provided, using the cache key as a filepath and breaking up directories based on the levels value.

The inactive parameter allows for control over the length of time a cache item will be hosted after its last use. The size of the cache is also configurable with the use of the max_size parameter. Other parameters relate to the cache-loading process, which loads the cache keys into the shared memory zone from the files cached on disk, along with many other options. For more information about the proxy_cache_path directive, find a link to the documentation in the following “See Also” section.

See Also

proxy_cache_path Documentation

4.2 Caching Hash Keys

Problem

You need to control how your content is cached and retrieved.

Solution

Use the proxy_cache_key directive along with variables to define what constitutes a cache hit or miss:

proxy_cache_key "$host$request_uri $cookie_user";

This cache hash key will instruct NGINX to cache pages based on the host and URI being requested, as well as a cookie that defines the user. With this you can cache dynamic pages without serving content that was generated for a different user.

Discussion

The default proxy_cache_key, which will fit most use cases, is "$scheme$proxy_​host$request_uri". The variables used include the scheme (http, or https); the proxy_host, where the request is being sent; and the request URI. All together, this reflects the URL that NGINX is proxying the request to. You may find that there are many other factors that define a unique request per application—such as request arguments, headers, session identifiers, and so on—to which you’ll want to create your own hash key.¹

Selecting a good hash key is very important and should be thought through with understanding of the application. Selecting a cache key for static content is typically pretty straightforward; using the hostname and URI will suffice. Selecting a cache key for fairly dynamic content like pages for a dashboard application requires more knowledge around how users interact with the application and the degree of variance between user experiences. When caching dynamic content, using session identifiers such as cookies or JWT tokens is especially useful. Due to security concerns, you may not want to present cached data from one user to another without fully understanding the context. The proxy_cache_key directive configures the string to be hashed for the cache key. The proxy_cache_key can be set in the context of http, server, and location blocks, providing flexible control on how requests are cached.

4.3 Cache Locking

Problem

You want to control how NGINX handles concurrent requests for a resource for which the cache is being updated.

Solution

Use the proxy_cache_lock directive to ensure only one request is able to write to the cache at a time, where subsequent requests will wait for the response to be written to the cache and served from there:

proxy_cache_lock on;
proxy_cache_lock_age 10s;
proxy_cache_lock_timeout 3s;

Discussion

We don’t want to proxy requests for which earlier requests for the same content are still in flight and are currently being written to the cache. The proxy_cache_lock directive instructs NGINX to hold requests destined for a cached resource that is currently being populated. The proxied request that is populating the cache is limited in the amount of time it has before another request attempts to populate the resource, defined by the proxy_cache_lock_age directive, which defaults to five seconds. NGINX can also allow requests that have been waiting a specified amount of time to pass through to the proxied server, which will not attempt to populate the cache, by use of the proxy_cache_lock_timeout directive, which also defaults to five seconds. You can think of proxy_cache_lock_age and proxy_cache_lock_timeout as “You’re taking too long, I’ll populate the cache for you,” and “You’re taking too long for me to wait, I’m going to get what I need and let you populate the cache in your own time,” respectively.

4.4 Use Stale Cache

Problem

You want to send expired cache entries when the upstream sever is unavailable.

Solution

Use the proxy_cache_use_stale directive with a parameter value defining for which cases NGINX should use stale cache:

proxy_cache_use_stale error timeout invalid_header updating 
  http_500 http_502 http_503 http_504
  http_403 http_404 http_429;

Discussion

NGINX’s ability to serve stale cached resources while your application is not returning correctly can really save the day. This feature can give the illusion of a working web service to the end user while the backend may be unreachable. Serving from stale cache also reduces the stress on your backend servers when there are issues, which can in turn provide some breathing room for the engineering team to debug the issue.

The configuration tells NGINX to use stale cached resources when the upstream request times out, returns an invalid_header error, or returns 400- and 500-level response codes. The error and updating parameters are a little special. The error parameter permits use of stale cache when an upstream server cannot be selected. The updating parameter tells NGINX to use stale cached resources while the cache is updating with new content.

See Also

NGINX Use Stale Cache Directive Documentation

4.5 Cache Bypass

Problem

You need the ability to sometimes bypass caching.

Solution

Use the proxy_cache_bypass directive with a nonempty or nonzero value. One way to do this dynamically is with a variable set to anything other than an empty string or 0 within a location block that you do not want cached:

proxy_cache_bypass $http_cache_bypass;

The configuration tells NGINX to bypass the cache if the HTTP request header named cache_bypass is set to any value that is not 0. This example uses a header as the variable to determine if caching should be bypassed—the client would need to specifically set this header for their request.

Discussion

There are a number of scenarios that demand that the request is not cached. For this, NGINX exposes a proxy_cache_bypass directive so that when the value is nonempty or nonzero, the request will be sent to an upstream server rather than be pulled from the cache. Different needs and scenarios for bypassing the cache will be dictated by your application’s use case. Techniques for bypassing the cache can be as simple as using a request or response header, or as intricate as multiple map blocks working together.

You may want to bypass the cache for many reasons, such as troubleshooting or debugging. Reproducing issues can be hard if you’re consistently pulling cached pages or if your cache key is specific to a user identifier. Having the ability to bypass the cache is vital. Options include, but are not limited to, bypassing the cache when a particular cookie, header, or request argument is set. You can also turn off the cache completely for a given context, such as a location block, by setting proxy_cache off;.

4.6 Cache Purging with NGINX Plus

Problem

You need to invalidate an object from the cache.

Solution

Use the purge feature of NGINX Plus, the proxy_cache_purge directive, and a nonempty or zero-value variable:

map $request_method $purge_method {
  PURGE 1;
  default 0;
}
server {
  # ...
  location / {
    # ...
    proxy_cache_purge $purge_method;
  }
}

In this example, the cache for a particular object will be purged if it’s requested with the PURGE method. The following is a curl example of purging the cache of a file named main.js:

$ curl -X PURGE http://www.example.com/main.js

Discussion

A common way to handle static files is to put a hash of the file in the filename. This ensures that as you roll out new code and content, your CDN recognizes it as a new file because the URI has changed. However, this does not exactly work for dynamic content to which you’ve set cache keys that don’t fit this model. In every caching scenario, you must have a way to purge the cache.

NGINX Plus provides a simple method for purging cached responses. The proxy_cache_purge directive, when passed a zero or nonempty value, will purge the cached items matching the request. A simple way to set up purging is by mapping the request method for PURGE. However, you may want to use this in conjunction with the geoip module or simple authentication to ensure that not just anyone can purge your precious cache items. NGINX has also allowed for the use of *, which will purge cache items that match a common URI prefix. To use wildcards, you will need to configure your proxy_cache_path directive with the purger=on argument.

See Also

NGINX Cache Purge Example

4.7 Cache Slicing

Problem

You need to increase caching efficiency by segmenting the resource into fragments.

Solution

Use the NGINX slice directive and its embedded variables to divide the cache result into fragments:

proxy_cache_path /tmp/mycache keys_zone=mycache:10m;
server {
  # ...
  proxy_cache mycache;
  slice 1m;
  proxy_cache_key $host$uri$is_args$args$slice_range;
  proxy_set_header Range $slice_range;
  proxy_http_version 1.1;
  proxy_cache_valid  200 206 1h;
 
  location / {
    proxy_pass http://origin:80;
  }
}

Discussion

This configuration defines a cache zone and enables it for the server. The slice directive is then used to instruct NGINX to slice the response into 1 MB file segments. The cached resources are stored according to the proxy_cache_key directive. Note the use of the embedded variable named slice_range. That same variable is used as a header when making the request to the origin, and that request HTTP version is upgraded to HTTP/1.1 because 1.0 does not support byte-range requests. The cache validity is set for response codes of 200 or 206 for one hour, and then the location and origins are defined.

The cache slice module was developed for delivery of HTML5 video, which uses byte-range requests to pseudostream content to the browser. By default, NGINX is able to serve byte-range requests from its cache. If a request for a byte range is made for uncached content, NGINX requests the entire file from the origin. When you use the cache slice module, NGINX requests only the necessary segments from the origin. Range requests that are larger than the slice size, including the entire file, trigger subrequests for each of the required segments, and then those segments are cached. When all of the segments are cached, the response is assembled and sent to the client, enabling NGINX to more efficiently cache and serve content requested in ranges.

The cache slice module should be used only on large resources that do not change. NGINX validates the ETag each time it receives a segment from the origin. If the ETag on the origin changes, NGINX aborts the cache population of the segment because the cache is no longer valid. If the content does change and the file is smaller, or your origin can handle load spikes during the cache fill process, it’s better to use the cache_lock directive described in Recipe 4.3. Learn more about cache-slicing techniques in the blog listed in the “See Also” section.

See Also

“Smart and Efficient Byte-Range Caching with NGINX & NGINX Plus”

¹ Any combination of text or variables exposed to NGINX can be used to form a cache key. A list of variables is available in NGINX.