High Performance Drupal

Chapter 2. Frontend Performance

The principles of frontend performance apply regardless of the underlying application used to create a website. The browser receives an HTML document and, based on its contents, downloads CSS, JavaScript, fonts, and images; it then renders the page using all of these. The 14 rules defined by Steve Souders’s High Performance Websites (O’Reilly) remain a good reference point for examining the pages served by a site and identifying areas for improvement (see this page for a refresher). Google’s PageSpeed and Yahoo!’s YSlow will quickly grade a single page of your site and identify the highest-priority areas for improvement. For this chapter we’re going to assume you have a working grasp of the rules (cacheable headers, compression, minimizing HTTP requests, etc.) and, rather than discussing them, we’ll look at the challenges specific to developing Drupal websites when implementing those rules.

Limiting HTTP Requests

Drupal provides CSS and JavaScript aggregation via a configuration option. This allows potentially dozens of individual requests for CSS and JavaScript files to be reduced to just a few. While enabling this option in production should be one of the first steps you take to optimize frontend performance, there are several other steps you can take to minimize HTTP requests that require some more work on your part. Especially on mobile devices, or slow Internet connections in general, the number of HTTP requests can have the most serious negative impact, taking into account both back- and frontend performance. HTTP request latency applies for every file required to build a page, and can be hundreds of milliseconds multiplied by the number of files on the page. Assuming your code and infrastructure can scale to handle the traffic that comes to the site, this should be the very next thing you look at with regard to the overall user experience and performance of the site.

Audits

Many Drupal sites operate without any custom JavaScript and, if using a stock contributed theme or base theme, may only have a small amount of custom CSS. This, however, doesn’t mean that the site itself is running with a small amount of JavaScript or CSS, as both core and contributed modules provide their own files. As with many other Drupal performance issues, the most common cause of problems is particular combinations of configuration and site structure and how these interact with modules that are unable to know exactly how they’re used on every individual site.

When identifying bottlenecks, select two or three pages of different types to start with. Ideally these will be the most popular types of page on the site—for example, article pages or user profiles, as well as a landing page such as the front page.

What you audit depends on your priorities for optimization. When auditing, start by disabling JavaScript and CSS aggregation, then view the pages as either an anonymous user or an authenticated user. This allows you to see all the individual CSS and JavaScript files being added to the page. Do not look for performance issues while logged in as an administrator unless you’re specifically trying to find issues affecting administrators, since toolbars, contextual links, and the like add a lot of page weight that will often show up as frontend (and sometimes backend) performance issues, obscuring issues that affect users without access to those features.

Once you’re ready, look at the CSS and JavaScript requests in a tool such as Firebug or Chrome Developer Tools.

If there are no JavaScript-heavy features such as carousels or slideshows on these pages, the first thing to check is whether any JavaScript is loaded at all. If it’s not necessary, loading no JavaScript at all saves the most possible HTTP requests (no requests are better than any nonzero number of requests), as well as other overhead such as initialization of jQuery.

Drupal 6 and Drupal 8 will not add any core JavaScript to the page if no modules or themes add their own (this isn’t the case for Drupal 7 at the time of writing, but see https://drupal.org/node/1279226 for a core bug report). However, it’s often the case that contributed or custom themes will add small (or even large) JavaScript files to every page for every user via the scripts[] property in .info files, or via hook_init().

In addition to serving pages without any JavaScript at all, Drupal 8 also makes it more likely that pages can be served without jQuery. For basic event handling, DOM selection, etc., native JavaScript functions are often perfectly adequate with modern browsers, and core JavaScript is being refactored to take advantage of these where possible. Scripts that require jQuery should explicitly declare it as a dependency, and if you have only a handful of .js files on a page but jQuery is one of them, look at whether both the files themselves and the jQuery dependencies within them are really necessary.

If you’re not expecting to see any JavaScript on the page you’re looking at, take a note of each filename, then grep the code base for where it’s added. Start with the files loaded last, since the early files like jQuery and drupal.js may only be loaded due to dependencies. While all pages on Drupal sites will include CSS, a very similar approach can be taken when trying to reduce the amount of CSS loaded overall.

There are several common reasons why files might be added to a page despite not being actually needed:

• The file has been added via the scripts[] or styles[] .info property, despite not being needed on every request. Try to find which markup the file actually affects, then file a bug report for the module on Drupal.org to add it conditionally via #attached instead. For example, if a JavaScript file is only used when nodes are displayed in full view mode, it can be moved from .info to hook_node_view() as follows.

  Before:

example.info
name = Example
description = Example module
core = 7.x
scripts[] = js/example.js

After:

<?php
/**
 * Implements hook_node_view().
 */
function example_node_view($node, $view_mode, $langcode) {
  if ($view_mode == 'full') {
    $path = drupal_get_path('module', 'example’') . '/js/example.js';
    $node->content['foo']['#attached']['js'][$path] =
      array('every_page' => TRUE);
  }
}
?>

• The file is associated with a feature that is only available to users with a certain permission but has been added to the page outside the permission check. File a bug report for the module on Drupal.org to make including the file conditional on the access check.
• Often CSS and JavaScript files apply to more than one feature. Following core guidelines for CSS organization ensures that admin-only CSS is only served to admins, and that files are easier to override for themes. Similarly, with JavaScript, it’s worth evaluating if a file should be split up—aggregation puts it back together when needed anyway.
• Sometimes files are related to a specific feature but are still added site wide. This is usually an error, but in some cases, there might be CSS that applies to a search box, header menu, or similar feature that appears on every page, or a very high percentage of pages. Having this CSS in the site-wide aggregate saves it being duplicated in each per-page aggregate, reducing their size and increasing the effectiveness of the browser cache. In general this makes more sense for CSS than JavaScript—all pages need CSS, but some might render without any JavaScript at all.

After reviewing the site with aggregation disabled, reenable it and view the pages again. This time it won’t be possible to see which individual files are being included, but instead you can look at the resulting aggregates of the pages and their comparative sizes.

A common problem with both the Drupal 7.x/8.x and Drupal 6.x aggregation strategies is that they’re fragile when files are added incorrectly—for example, if they’re added in different orders by different modules, or if the every_page option is set for conditionally added files. This can have the result that even if very similar JavaScript or CSS files appear in two or more pages, different aggregate filenames get created, resulting in lower cache hit rates, more bytes to download, and a greater workload server side generating the aggregates.

To track down issues like this, first compare the list of aggregate filenames, locate any filenames that are unique to any of the pages being compared, and then look at the size and/or contents of those files to see if they’re actually different. On a live site that’s been running for some time, checking the number and date of aggregates in the css and js directories can also be an indicator of how many unique aggregates are being created. One or two small aggregates differing between pages is expected if the JavaScript added is genuinely different, but very minor changes between files or several files changing may indicate an underlying issue.

A further option to reduce HTTP requests with JavaScript, assuming only minimal JavaScript usage on a site (i.e., no jQuery dependency), is to add it inline rather than using an external file. Drupal’s JavaScript API supports this via the inline option.

Image Requests

Images embedded in content via the <img> tag are relatively hard to optimize in terms of the number of requests. You can optimize images for bandwidth using image derivatives, which ensure the images are scaled or cropped to the size they will be served at. Drupal 8 goes further by supporting responsive images via the Picture module (also available in Drupal 7 as a contributed module), so that the correct image derivative—and only that image derivative—is loaded based on breakpoints. For very image-heavy pages, you may want to explore more advanced techniques like deferred image loading via JavaScript.

For images loaded via CSS, there are more options. Go back to Firebug or Chrome DevTools to look for image requests; the paths will tell you whether they come from core, contributed, or custom modules, or themes.

Most Drupal 8 modules do not provide much default styling, with the exception of content forms, administrative features and user-facing menus which do have some icons.

There are several approaches for reducing image requests:

• Image sprites combine several images into a single file, then use CSS to display only the specific image needed. Creating and maintaining sprites can be quite time-consuming, but tools like SASS allow for automation of this process.
• Images can be base64 encoded within CSS files using data-uri. This means they are served as part of the CSS file itself rather than downloaded separately, saving an HTTP request for each image that’s inlined. Remember that the larger your CSS file is, the longer it takes before the browser can download and parse it and move on to other things (like downloading images served via img tags), so this is a trade-off that needs to be made carefully if at all. This is supported by a contributed module for Drupal called CSS Embedded Images that automatically inlines images when the CSS is preprocessed.
• Icon fonts allow for arbitrary images to be combined in a single font file. This has the same advantages as a sprite in terms of reducing HTTP requests, but since fonts use vector graphics, it also allows the icons to be scaled or presented in different colors without any modification to the original image. Fonts can be embedded into CSS using data-uri as well, saving a further HTTP request.
• Another approach is to use browser support for scalable vector graphics (SVG) directly. As with fonts, this allows for scaling, recoloring, etc., without modification of the original image. Since SVG files are XML, it’s also possible to style the SVG itself with CSS. SVGs can be used via a URI, embedded into CSS via data-uri, or embedded into HTML using either data-uri or the SVG format itself, which provides a great deal of flexibility in terms of how they’re served.

Both icon fonts and SVG have significant advantages over the older techniques of sprites and base64 encoding of binary images; however, some older browsers don’t support them, so you may need to include a polyfill library if your site requires them.

Minification

Drupal provides very rudimentary on-the-fly CSS whitespace and comment stripping as part of the core aggregation support. There is no core support for JavaScript minification—files are concatenated together when aggregation is enabled, but the contents are left unchanged.

This leaves three options for minifying/uglifying JavaScript files, as discussed in the following sections.

Minification On the Fly

Drupal 8 has added the Assetic library, which amongst other things provides support for minification of JavaScript and CSS via preprocessors. At the time of writing, the work to replace core’s own file aggregate generation with Assetic has not been completed; however, if this lands for Drupal 8, it will allow files served from Drupal to be preprocessed via any of the pluggable backends that Assetic supports. It should be simple to implement as a contributed project if support isn’t available in core. The main advantage of Assetic over previous on-the-fly preprocessors from contributed modules is that it supports native JavaScript backends such as uglify.js. While uglify.js (which requires Node.js) introduces an additional hosting requirement, the resulting minified code is much more efficient than that produced by PHP preprocessors, which are not well supported, use more server resources, and result in larger files.

Preminification for Modules/Themes

Drupal core ships with minified versions of jQuery and other external JavaScript libraries. Minification does not yet happen for JavaScript provided by Drupal core itself, nor for many contributed modules and themes that provide dedicated JavaScript files. Ensuring that external libraries are shipped as their minified versions (or both minified and unminified) allows sites to serve these by default without taking any additional steps, but it does introduce overhead for core or contrib developers whenever a file changes, and thus far there is not a system in place to support this. The Speedy module provides minified versions (via uglify.js) of core JavaScript files, which is a good one-stop solution for core, even if it will leave contributed projects unminified until they’re individually supported. Preminification also solves the problem of retaining license information in minified files, which is a requirement for open source JavaScript libraries.

Minifying During the Build Process

If you have automated code deployment, minification could be added as a step in building releases (this is also something that could be considered for Drupal.org project packages). This is really a site-specific version of using/contributing to the Speedy module and is only mentioned here for completeness.

Compression

Serving files with gzip compression and respecting Accept headers allows file size to be reduced drastically. Drupal handles this via PHP for cached pages via a setting, and via .htaccess rules for JavaScript/CSS aggregates (both gzipped and uncompressed files are saved during the aggregation process, then the .htaccess rule rewrites them). Compression for uncached HTML pages is not supported by core so needs to be handled at the server level via mod_deflate or equivalent, in which case the PHP gzip support for cached pages should be disabled via configuration as well. You may want to disable PHP gzipping of CSS and JavaScript files as well and handle this at the server level. This can be done via settings.php or variable_set() in Drupal 7, or via the configuration API in Drupal 8. You will also need to edit your .htaccess to comment out the rules for rewriting filenames, since the Apache module will be handling serving the correct file instead. Note that there’s no UI provided for this in the administration screens. To see the configuration options in Drupal 8, either review the aggregation code itself, or look at system.performance.yml:

cache:
  page:
    use_internal: '0'
    max_age: '0'
css:
  preprocess: '0'
  gzip: '1'
fast_404:
  enabled: '1'
  paths: '/\.(?:txt|png|gif|jpe?g|css|js|ico|swf|flv|cgi|bat|pl|dll|exe|asp)$/i'
  exclude_paths: '/\/(?:styles|imagecache)\//'
  html: '<!DOCTYPE html><html><head><title>404 Not Found</title></head>
         <body><h1>Not Found</h1><p>The requested URL "@path" was not found
                 on this server.</p></body></html>'
js:
  preprocess: '0'
  gzip: '1'
response:
  gzip: '0'
stale_file_threshold: '2592000'

Cacheable Headers

Drupal sets cacheable headers for all CSS, JavaScript, and images, as well as for cached HTML pages. The HTML max_age value of the Cache-Control header can be set via admin/config/development/performance or the configuration API; assets are set to have an Expires header of two weeks via .htaccess if mod_expires is enabled in Apache. For sites that aren’t undergoing frequent releases, you may want to tweak this upward. If you’re not using Apache, you’ll need to ensure that you handle cacheable headers for static assets in the web server you’re using.

CDNs

Content delivery networks have two primary goals. First, they allow files (and potentially whole pages via custom configuration) to be served from a location as close as their infrastructure allows to the visitor requesting a site. Therefore, a site hosted in the US but visited by a user in France may have all JavaScript, CSS, and images served from servers in France, dramatically reducing the latency of those requests. As a secondary benefit, they reduce the number of requests to your own infrastructure, freeing up bandwidth and server resources to serve only uncached requests that can’t be handled by the CDN.

See Chapter 19 for more information on CDNs.

Keep Third-Party Libraries Up to Date

Drupal 6 and 7 have frozen versions of jQuery. This means that the latest stable Drupal 6 release ships with jQuery 1.2.6 (released in 2008) and the latest stable version of Drupal 7 ships with jQuery 1.4.4 (released in 2010). jQuery’s release schedule is considerably faster than Drupal core’s for major releases, which means its developers often drop support for the version of jQuery shipped with the latest stable version Drupal core while the new Drupal release is still under development. To compensate for this, the contributed jQuery Update project exists: it includes more recent versions of jQuery, as well as replacing particular core JavaScript files dynamically if they’re incompatible with the newer versions. While sites usually install jQuery Update due to a frontend feature that specifies it as a dependency, the jQuery team is constantly adding optimizations to jQuery with each release. Simply installing jquery_update may result in both a smaller file size and access to performance optimizations within jQuery itself, such as faster selectors.

Drupal 8.x at the time of writing includes jQuery 2.0.0, and unlike Drupal 6 and 7, it’s intended to update third-party JavaScript libraries as they become available with point releases of Drupal 8, with an option for a site to pin/downgrade its jQuery version to the older one if necessary. This will be a first for Drupal core but may mean that jQuery Update is not necessary for Drupal 8 sites.

jQuery Update

jQuery Update also provides an option to serve the minified jQuery file via Google’s CDN rather than from the module folder. If you’re not already using a CDN, this allows quite a large file to be served via a CDN “for free.” There’s also the potential that site visitors will have visited other sites that serve the same jQuery version prior to visiting yours and already have it cached, although how likely this is depends on the traffic patterns of your site’s visitors and overall adoption of the Google CDN. If you have a family of sites all running Drupal with lots of traffic between them, the chances of this happening might be increased.

However, this does mean an extra DNS request, a dependency on Google’s infrastructure, and an extra HTTP request, since jQuery will no longer be included in aggregates, so be aware that there are trade-offs in both directions.

External Scripts

Regardless of the quality and performance of Drupal core, contributed modules, and your own custom module or themes, all of that optimization and thought can go to waste—or at least be cancelled out—as soon as you add analytics, social widgets, advertising, and similar external services to a site.

Services like these often drive either revenue or traffic (or both) to websites, and when building commercial or community websites, there’s often a lot of pressure (from either end users or business owners) to add as many as possible. This can result in many different JavaScript snippets from different services being included, which in turn may load other JavaScript files, CSS, and images.

All external services are different, but there are several rules of thumb that apply to most. We’ll look at some of them in the next section.

Single Points of Failure (SPOFs)

When JavaScript is loaded synchronously, browsers block all rendering until the file has finished downloading. If an external service is down or having performance trouble, this may cause a script included on your page to take longer than usual to load, or fail to load altogether.

Synchronous loading just means putting a normal JavaScript file in a normal script tag:

<script>http://example.com/some/file.js</script>

If example.com is unable to serve the request in a timely manner, browsers will wait until either it eventually serves the request, or the request times out before rendering the full page. This can result in large blank sections below where the script is included or even entirely blank pages, depending on the browser and the location of the script tag, not to mention potential delays of 30 seconds or more. Most of the optimizations in this chapter have focused on changes that are likely to save milliseconds, hundreds of milliseconds, or perhaps a couple of seconds at most; yet a single external script can render a site unusable—potentially as unusable as an outage on your own infrastructure, in terms of the end user experience.

Note

The SPOF-O-Matic browser plug-in by Patrick Meenan both flags likely single points of failure and can simulate complete failure for any external script it finds on your pages. This allows SPOFs to be found easily and provides an easy way to demo just how bad they are to anyone who might question the importance of handling external scripts carefully!

Many of the more popular services now provide asynchronous snippets that will not block page rendering; this is usually achieved by providing inline JavaScript, which then dynamically creates a script tag so that the JavaScript is loaded asynchronously.

Even scripts loaded asynchronously can block the browser onload event, on which real user monitoring, analytics, and in some cases site functionality might rely. A further optimization is executing the JavaScript within a dynamically created iframe so that it’s isolated from the parent window’s onload event. Note that techniques in this area change frequently; some services still support (and advertise in their documentation) snippets they provided several years ago and that might be found on sites in the wild, and some services have ignored these techniques and exclusively provide snippets that will cause a SPOF.

To avoid this, ensure you audit sites for SPOFs; SPOF-O-Matic is great for this. When adding scripts, avoid any temptation to embed markup or script tags directly into a page.tpl.php, head.tpl.php, or any other template or custom block, and use Drupal APIs such as #attached and drupal_add_html_head() instead. Better still, if a contributed module supports the service, consider enabling the widget or analytics via that module instead of custom code, as the contributed project has a better chance of keeping up with newer versions of the snippet than you do.

As well as SPOFs, it’s also worth checking for cacheable headers on any assets that scripts load themselves. Frontend audits of sites have often found CSS or secondary JavaScript files from external services loaded without minification or compression, and without cacheable HTTP headers—whoops!

For social widgets in particular, also consider their usage on the site itself. Most sites present lists of content on a single page, and it’s quite possible to have several social widgets enabled for each node teaser on such pages. For example, let’s take a page showing 20 node teasers. If widgets make requests back to services to load information such as Like/comment/+1 counts, that’s 20 times as many of those requests, as well as the JavaScript itself being executed 20 times for each request. A poorly optimized widget that appears once is bad enough, but when there are 20 of the same thing on a page, it could go from a sluggish response to crashing a browser.