Web server (httpd) and PHP configurations are often overlooked or trivialized. Many Drupal deployments make only small changes to the httpd or PHP configuration (raise your hand if your biggest change to php.ini has been to increase PHP’s memory limit), and otherwise blindly accept most default configuration values. This is sometimes with good reason: using a default httpd.conf file will work fine in some cases, and generally sites can perform pretty well with little to no tweaking of the httpd configuration. On the flip side, it is very possible for a default httpd configuration to grind your server to a halt should you experience a large traffic spike. Not only that, but there are configuration options that can help improve performance, sometimes drastically.
This chapter will focus mainly on changes that can be made to improve performance in PHP and Apache’s httpd daemon. We give Apache 2.2 more attention than other web servers because it is the most widely used web server; however, the chapter also contains a section on alternative web servers and PHP configurations (e.g., CGI versus running as an Apache module).
To understand what an opcode cache is and why it’s important, you first need to understand how PHP works server-side. When a PHP script is run on your web server, the PHP source file is read and then compiled into byte code before being executed. If you add an opcode cache, the intermediary executable code gets compiled on the first access of a script but then is stored in the cache and used for subsequent requests of the same script. This saves a lot of overhead, which is especially noticeable on a busy server.
Although there are a number of PHP opcode caches available, we’ll focus on APC, as it is the most widely used. However, with the release of PHP 5.5, the core PHP distribution now includes Zend Optimizer+, which after being open sourced has become the default opcode cache in PHP moving forward. PHP is now referring to Zend Optimizer+ as “OPcache.” APC currently ships as a PECL extension, meaning it can be installed via the PECL install tool, though most Linux distributions and add-on software repositories have prebuilt packages available. OPcache is built into PHP versions 5.5.0+ and is available as a PECL extension for versions 5.2, 5.3, and 5.4. An alternative to APC is most welcome, and we imagine most sites will soon migrate from APC to OPcache. In the meantime, however, the vast majority of sites are running on APC, so we will focus on APC throughout this chapter.
Once installed, the APC extension needs to be enabled either in php.ini or in its own apc.ini file. For servers hosting a single Drupal site, most of the default settings for APC will work fine; however, it is very important to set an appropriate memory size. For most sites, using a single shared memory segment with a size of 128 MB is a good starting point. Once set, restart Apache to pick up the change, then load your website and repeatedly load a few different pages. Then visit apc.php (a monitoring script included in most APC packages that can be moved to your webroot, but should not be left there after use) in your browser to see the current statistics. What you want to see is little to no fragmentation of the cache, and there should be some amount (at least 16–32 MB) of free space available. At this point, you should also see hit rates at or close to 100%. If your cache is totally full or showing high fragmentation, or if the full count is more than 0, you should increase the amount of memory allocated to APC. Continue increasing the memory allocation until the cache is left with free space; once the site has run for a few days without filling up the cache, you’re in good shape. It’s important to check on this periodically to be sure that the cache isn’t filling up or becoming fragmented.
Once your APC memory allocation has been set properly, there are many other configuration settings that allow some fine-tuning of how APC runs. As mentioned previously, these typically don’t need to change for most sites (especially if you are only running one Drupal web root on a server). The full list of APC runtime settings is available at http://www.php.net/manual/en/apc.configuration.php. A few of the most commonly customized are:
-
apc._num_files_hint -
This setting tells APC how many files you expect to be cached. It uses this information when allocating cache storage space. The default is
1000, which is a pretty good rough estimate for a single Drupal site. However, if you are running multiple sites or a site with a lot of modules, increasing this could help reduce fragmentation. -
apc.stat -
This setting controls whether or not APC will “stat” (check) a file to be sure it hasn’t been updated since the cache entry was created. The default is
1, meaning that APC will perform this check. If set to0, APC will not recheck files once they have been pulled into the cache. This means that if you change a file, you need to reload Apache in order for APC to read in that change. Obviously this is not ideal for development environments, but it can lead to performance improvements in infrastructures with high I/O latency. This setting needs to be approached carefully, as there are quite a few other stats in the process of serving a request and they must all be disabled for you to see much improvement.
Note
So far, our description of APC has been entirely as an opcode cache. However, APC also supports “user” caches, meaning that you could use APC to store Drupal cache data. We discussed that possibility in Chapter 16.
Again, it’s important to monitor APC status to ensure the cache does not fill or become too fragmented (usually fragmentation is caused by the cache being at or close to full). There are, for example, Munin scripts that scrape the APC stats from a local PHP script (a great candidate to live in an adminscripts directory!) and graph them over time. If you notice the usage approaching full on the APC cache, it’s likely that you’ll want to increase the cache size.
There are a few settings in php.ini that are important to consider when bringing up a site. The most important of these settings is memory_limit, which controls how much memory a single PHP process is allowed to consume. It is important to gauge this setting correctly because if a process consumes more than this amount of memory, it will error out at the point where it attempts to allocate more memory, and the entire page load will fail. For many Drupal sites, the maximum memory usage is somewhere around the 128 MB range. While it’s not uncommon for this to be set at 512 MB or 768 MB on larger websites, if you are using that much RAM in PHP, you may have a problem in your code that needs attention. The downside to setting this to a very large value is that a memory leak or “runaway” process can end up consuming quite a bit of system memory, potentially causing the server to run out of memory.
Watch the memory use of your PHP processes (or httpd processes if you’re running PHP as a module) to see how much they are using on average by running top and sorting by memory usage (type O to see sort options, then n to sort by resident memory). However, that just tells you what the current processes are doing and doesn’t tell you what the maximum usage may be. For a more precise view of PHP memory usage, consider enabling XHProf, described in Chapter 6, and use it to profile the different page types on your site. With this setting, it’s best to err on the large side because setting it too low will cause some user-visible errors with the site should you ever hit the limit.
The display_errors setting controls whether or not PHP errors will be shown on the output of the page where an error occurs. This is clearly something you want to disable on your production servers, but it can be very useful to enable on your development servers.
While you shouldn’t display errors to users on a production site, you likely want to keep track of errors if they do happen. For this reason there is a log_errors setting that controls whether or not errors are logged. If you are running PHP as an Apache module, setting this to On will log PHP errors to Apache’s error log.
Both display_errors and log_errors are selective about what types of errors they show. Both of them are affected by the error_reporting setting, which lets you define which types of errors to show/log and which to ignore. Settings here are applied bitwise, meaning you can combine different message types with “and,” “or,” and “not” in order to achieve the type of error reporting you desire. Generally on production servers, you would only care about actual errors and not about warnings, notices, or language deprecation messages. On development servers, it’s more useful to track everything so that you can see things like PHP notices when variables are used before being set, etc. php.ini files ship with a large section of comments explaining the various message types along with suggested settings for production versus development.
It’s also important to set the time zone in php.ini (or in your .htaccess file). As of PHP 5.3, a warning message will be sent when the date.timezone setting is unset—and importantly, it defaults to unset. Valid time zone values are listed here.
PHP is most commonly run as an Apache module called mod_php. This is the easiest way to run PHP, as it can be installed as a package that includes a very minimal Apache configuration to enable the module for PHP scripts. There are two main issues to understand when running mod_php:
- The PHP module will be loaded into each httpd process, increasing resource usage and making the processes heavier. This means that every access to httpd, even for a static (non-PHP) file, will have the overhead of the PHP module residing in the httpd process.
- PHP scripts will all be run as the same user, generally apache or www-data, that the httpd processes are run by. This is generally fine when you only host a single site on a server, but it’s not secure for multisite environments, where it is best to keep the sites separated as much as possible. One way to work around this issue is to run multiple instances of Apache, each with a different user. This is easily accomplished with the default configuration in Debian-based distributions, though it is not as easy with Red Hat-based-distributions.
It is possible to run PHP as an “external CGI,” which just boils down to having some way for httpd to call an external PHP process when it needs to execute PHP code. In order for this to work in production environments, you need to run a CGI manager such as mod_fastcgi to manage the PHP processes. This requires a bit of extra configuration over the built-in module, but you gain flexibility. For alternative web servers such as Nginx, you must run PHP as a CGI—more on this at the end of this chapter.
PHP now includes the FastCGI Process Manager, or FPM, also referred to as PHP-FPM. Prior to PHP-FPM, it was difficult to ensure PHP CGI processes shared an opcode cache, which was not great for performance. The addition of PHP-FPM allows its PHP processes to share a central APC cache, making it a stronger option for those who are looking for an alternative to running mod_php.
Many sites choose to skip the extra configuration needed to run PHP as a CGI binary, and instead stick with the default mod_php. mod_php can scale well and has the performance advantage of being in the same process space as Apache. However, it does add additional memory overhead in the httpd processes, and it’s not quite as tune-able as having PHP completely separated as a CGI. For example, in a PHP-FPM setup, you can limit the amount of memory and/or threads each FPM pool has available to it. If you need this extra configurability, are hosting multiple sites that need to run as different users, or cannot offload static file serving (such as by using Varnish or a CDN), FastCGI and PHP-FPM are options worth considering. For our purposes, we will continue to assume the use of mod_php since it is the most popular method to run PHP.
Apache is one of the most configurable and flexible HTTP servers in existence. It is in fact so configurable that you can even change its process model via your choice of multiprocessing module (MPM). The two major core MPMs are Prefork and Worker:
- Apache Prefork can be considered the “classic” multiprocessing module. It works by having a single parent httpd process managing a pool of child processes. Each request that comes into the server is “assigned” to a child process, which then takes over communication with the client and fulfillment of the request. If more processes are needed, the parent process will create them and will clean up old processes.
- Apache Worker is a newer multiprocessing module that relies on threads instead of processes. It works in much the same way as the Prefork module, except that instead of maintaining processes, it maintains threads. The advantage to this is that its lighter weight. The disadvantage is that you must ensure your code is entirely thread-safe.
In most cases, the choice between these two modules is made for you. If you are using mod_php, you should use the Prefork module. There are many non-thread-safe PHP modules, and some are required by many Drupal installations. On the other hand, if you are running PHP as a CGI, you have the option of using Apache Worker and making Apache a bit more lightweight. The remainder of this chapter will assume the use of the Apache Prefork module using mod_php.
The Prefork module has a number of settings that control its behavior. Generally, the defaults for the settings controlling the number of spare servers to start and leave idle work fine for most sites. However, it’s important to focus on the settings that control the number of processes and how often they are cycled, as these are the settings that have the potential to bring your server crashing down if not configured correctly. They are:
- StartServers
-
This setting defines how many processes will be started when the httpd service starts. For Prefork, this defaults to
5. For most sites this is fine, since httpd will spawn new processes as needed. In cases where you know your server will always be serving more than five clients at once, you might want to increase this so that there isn’t a wait involved when spinning up additional processes. - MinSpareServers and MaxSpareServers
-
These settings control how many idle (not actively serving requests) httpd processes will be kept around. They default to
5and10, respectively, meaning that if there are less than five idle httpd processes, new processes will be spawned (at a maximum rate of one per second) until there are five idle. Contrarily, if there are more than 10 idle httpd processes, the additional processes will get killed off by the parent httpd process. Again, for most sites these defaults work fine, although on some sites it may be worthwhile to increase these settings somewhat so that you can better deal with traffic spikes. Setting these values very high means that you will have a lot of processes on the system using up RAM even when traffic is rather low. - MaxClients
-
Defines the maximum number of httpd processes that can be running. It can “make or break” your site, in that if it is set too low, the server will queue incoming HTTP connections (eventually dropping them once a threshold is hit), and if it’s set too high, the server may swap itself to death during periods of high traffic. The default value for MaxClients for Prefork is
256. For static websites or very minimal PHP scripts, 256 processes may be fine, but for most Drupal sites, this is way too high. The reason that this value is too high is because when usingmod_phpmemory is allocated inside of each httpd process, and for larger Drupal sites, it’s not unusual to see httpd processes using somewhere from 64–150 MB (or more!) each. If your httpd processes are each averaging 150 MB of RAM, then at 256 processes, that’s nearly 38 GB of RAM. Some of us are lucky to have servers that large, but it’s not common, or inexpensive. Also, that doesn’t take into account memory usage by other processes on the system. The trick to settingMaxClientsappropriately is to first figure out how large your httpd processes are. You can get a general number for this by running top and looking at theRES(resident memory) column. Once you know how large your httpd processes are, you can use the following formula to see how many httpd processes you can support:
((System RAM) – (RAM used by other processes)) /
(httpd process size) = MaxClients“RAM used by other processes” would take into account any other major services running on the web server, plus some amount for basic OS-level processes. Generally we use 512 MB as a rough estimate for OS resource needs, and then add any additional service requirements to that. This 512 MB estimate is intended to cover resources used by the kernel as well as periodic processes that may be run on the server. Long-running services should be analyzed separately.
As an example, suppose your web server is running Varnish, httpd, and no other major services. The server has 8 GB of RAM. Varnish is allocated 2 GB of memory. Your average httpd process is around 85 MB. To provide some safety in our estimate, we’ll round that up to 100 MB. In this case, we would calculate MaxClients as follows:
((8 GB System RAM) – (512 MB OS resources + 2 GB Varnish)) /
(100 MB httpd process)
= 5632 MB / 100 MB
= 56.32You should always round down at the end for safety, so you would end up with 56 as your MaxClients setting.
- ServerLimit
-
This setting defines the maximum value that you can set
MaxClientsto during the life of the parent httpd process. In most cases, if you have calculatedMaxClientsas shown above, you can set this to the same number. If you want to give yourself some wiggle room in order to potentially increaseMaxClientswithout having to fully restart Apache, you could set this to be the maximum that you would increaseMaxClientsto. In either case,MaxClientsmust be less than or equal toServerLimit. - MaxRequestsPerChild
-
This setting controls how many requests a child httpd process will serve before it is killed. The Apache default is
10000, although many distributions ship with other values in httpd.conf. Both4000and0(disabled) are used by common distributions. When this is set to0, httpd child processes are never killed (except as part ofMaxSpareServers, as described above). As with most Apache settings, there is a trade-off involved when setting this. Setting this value on the high side can cause your httpd processes to consume more RAM over time due to memory leaks in PHP or other scripts, or due to a process that loads different code bases over its lifetime. On the other hand, setting this value too low will lead to a lot of overhead on the server caused by killing and respawning httpd processes. Generally, we would consider somewhere in the1000–4000range to be a safe value for this setting, based on the traffic your site experiences.2000is a good middle-of-the-road value that is high enough that you won’t see process thrashing in most cases, but low enough that httpd processes won’t stick around forever eating up a lot of memory.
KeepAlive allows a client to reuse the same connection with the server multiple times instead of having to open up a new connection for each request. For example, if you had a web page with a number of images and CSS files all served locally, the client would download the web page and then reuse the same connection to download the images and CSS. This saves time and resources on both ends of the connection. However, there is a downside to using KeepAlive, which is that the server needs to keep connections open for a certain amount of time (defined by KeepAliveTimeout). This can lead to a lot of open connections sitting around waiting and not actually serving any content. For this same reason, KeepAlive can make it easier for someone to launch a denial of service (DoS) attack against your site—once there are enough connections open waiting for clients, no other requests to the site can be served.
To be clear, KeepAlive is very beneficial for client-side performance. However, it can bring down sites if misconfigured, or if enabled at all during large traffic spikes. This is one of the benefits of having a caching reverse proxy, such as Varnish, as it is far more capable of handling KeepAlive connections to clients without adversely impacting Apache stability.
If you choose to use KeepAlive, be sure to also look at the KeepAliveTimeout setting, which defines how long a process will wait for an idle client to send another request. The default KeepAliveTimeout is five seconds, which in most cases is rather high. Ideally, set this as low as possible without having it force new connections on existing clients. One or two seconds can work generally, but it’s worth a bit of trial and error to figure out the correct setting for your site and the majority of your visitors. Also, we highly recommend providing KeepAlive to clients through a proxy and not directly with Apache.
There are a number of headers that are used to inform client browsers and intermediate caches whether or not to cache content, and if so, for how long. Configuring these headers properly can make a difference in user-visible performance on your site. The headers you will want to keep an eye on when setting up a production site are ETags, Expires, Cache-Control, Last-Modified, and Vary. These headers work together to determine just how aggressively a client can cache a page or static file. We will discuss each of these headers individually:
-
Last-Modified - This header is used exactly as you might expect, to check whether a page has been modified since it was last fetched. By default, Drupal adds this header to pages with a timestamp generated during the Drupal bootstrap process when serving a page.
-
ETag - The ETag, or entity tag, is meant to uniquely identify a single version of a page. When the ETag matches on two versions of a document, they are assumed to be the same. This tag can be as simple as an MD5 hash of the document or as complicated as hashes of various aspects of the request. What it should not be is the default Apache ETag, which includes server-specific information. If you use Apache’s default ETag, if you have four web nodes the ETags generated on each node will be different, and reverse proxy caching will be significantly worse.
-
Expires -
This header indicates when the page will expire from any cache it is in. In Pressflow and Drupal 7 or later, you can set this date by configuring the page cache maximum age. This is a legacy header, and if
Cache-Control(which we will discuss next) exists, this header is ignored by newer browsers and proxies. -
Cache-Control - This header controls caching options and is set by Pressflow and Drupal 7 later when they are configured to use an external page cache. The name is not as creative as one might have hoped. The header contains keywords and settings that impact caching, for example:
-
max-age=N - This informs downstream proxies of the maximum number of seconds this page can stay in its cache.
-
no-cache - This completely disables caching.
-
private - This allows for browser caching, but not intermediate proxy caching.
Apache allows you to set multiple log outputs for different types of information, though the majority of sites are configured only with a general access log and an error log. Most often, in the case of multiple sites on a single server, logs are defined per-virtual host so that it’s easy to track information for each individual site.
The format of the log files can be defined using the ErrorLogFormat and LogFormat directives. Generally, the default ErrorLogFormat is fine for most sites, and for the access log, most sites use one of the log formats that comes predefined in default Apache configuration files—either “common” or “combined.” The combined log format adds header information for Referer and User-Agent. If you decide to use your own custom log format, you can define the format and give it a name using the LogFormat directive, and then refer to that name in a CustomLog directive:
LogFormat "%h \"%r\"" hostandrequest CustomLog logs/request.log hostandrequest
This example defines a hostandrequest log format that includes only the remote host and the requested URL, and then uses that format to log to the file logs/request.log.
Note
Log destinations are defined relative to the ServerRoot unless the path begins with a slash. ServerRoot defaults to /etc/httpd on Red Hat-based distributions or /etc/apache2 on Debian-based distributions. Red Hat includes a symlink from /etc/httpd/logs to /var/log/httpd in order to ensure logs end up in the standard location on the system.
While the built-in Apache logging works great for many sites, it’s not quite as flexible as some sites require. For these situations, Apache supports using external logging tools. Cronolog is one example of an external logging application used in conjunction with Apache. It has some nice features, such as dynamic log file naming using date fields, making it possible to output to a log file with the current date in its filename. When the date changes, Cronolog will automatically start writing to a new file, meaning you get “built-in” log rotation as well as having log files nicely organized by date. Because new log files are written to every day, there’s no need to reload httpd to perform log rotation. You most likely will want to add your own scripts to compress these log files, and eventually to remove/archive them. The following is an example Apache configuration to use Cronolog:
ErrorLog "|/usr/sbin/cronolog /var/log/httpd/error/%Y%m%d-error.log" CustomLog "|/usr/sbin/cronolog /var/log/httpd/access/%Y%m%d-access.log" common
You can add the following script to cron.daily on the server to compress old log files:
#!/bin/bash
export PATH="/usr/bin:/bin"
if [ -d /var/log/httpd ]; then
find /var/log/httpd -name "*.log" -mtime +1 -exec gzip -9 "{}" ";"
fiand this one to eventually remove them:
#!/bin/bash
export PATH="/usr/bin:/bin"
if [ -d /var/log/httpd ]; then
find /var/log/httpd -name "*.log.gz" -mtime +60 -exec rm "{}" ";"
fiThere are a couple of places where Apache may output information about its version, and potentially version information for modules such as PHP. One of these is the “server signature” line that optionally shows up at the bottom of Apache-generated pages (e.g., 503 errors or directory listing pages); the other is the Server HTTP response header that gets sent by the server. Some people may consider this a security risk in that it makes it easy for potential attackers to know what software versions you are running and target you with specific attacks. Others would say that security through obscurity is not security at all, and if someone is dedicated enough, hiding information like this is only going to make it take a few minutes longer for them to figure out an attack. Whatever your view is, you may want to hide this information either for security reasons or just because you don’t like the way it looks to potential visitors.
The Server header and server signature lines are controlled by two different settings:
-
ServerSignature -
This is an on/off setting that controls whether or not to show the server signature line on Apache-generated pages. Setting this to
Offmeans that the server signature will not be shown on those pages. -
ServerTokens -
This is the setting that controls what information is shown in the server signature line and placed in the
ServerHTTP response header. The default setting forServerTokensisFull, which will show Apache version information as well as module and version information for running modules (e.g., PHP). While you can’t set this to be empty without modifying the source code, you can limit the information sent to be only the “Product” string, “Apache,” by settingServerTokenstoProd. Remember that changing this not only changes the server signature on documents, but also controls the text used in theServerHTTP header.
Often times, you’ll need to store various web-accessible scripts on a server used for monitoring or management. For example, it can be useful to have access to apc.php to review APC settings and statistics, but you typically don’t want to leave that in your publicly accessible web root. There are a couple of easy solutions for how to securely store those scripts: you can either use a locked-down directory or a separate virtualhost to store the scripts. In either case, securing the scripts directory using a list of authorized IPs and/or an htaccess username and password is critical.
Note
Some sites go so far as to only allow access from localhost, meaning you need to do something like proxy your traffic through SSH in order to load the administrative scripts. If you do lock down a directory to only localhost, ensure that you’re factoring any reverse proxies you may be running into your thinking. These will make many connections appear to be coming from localhost.
Storing these types of scripts in their own separate directory has a couple of great benefits. First, you don’t need to temporarily copy files into your production web root and remove them when you’re done. (How many times have you copied a phpinfo() script or similar into the web root and then forgotten to remove it?) Secondly, keeping them in their own virtualhost or directory means they are easily accessible by authorized users (or monitoring scripts) at a permanent URL that you can use across all of your web servers.
An example administrative scripts directory configuration looks like this:
Alias /adminscripts /var/www/adminscripts
<Directory "/var/www/adminscripts">
Options -Indexes
AllowOverride None
# Control who can get stuff from this directory.
Order Allow,Deny
Allow from 127.0.0.1
</Directory>Apache’s web server, httpd, is the most popular web server in use today, and has been for many years. Recently, however, some other open source web servers have gained some traction. Nginx is one alternative to Apache’s httpd that has become very popular.
There are a number of reasons why you might consider using Nginx for your site instead of Apache. Nginx is newer, and there is a lot of discussion around the Internet suggesting that Nginx is generally faster and lighter, especially for serving static files. While some tests may back this up, it’s by no means conclusively faster for all requests. One thing that may make Nginx faster than Apache for serving static files in our default setup (using mod_php) is that with mod_php Apache would have a full PHP process in memory for each httpd process. Nginx doesn’t have to worry about this, because it relies on an external PHP setup. This is one reason we recommend offloading static file serving if you are using Apache and mod_php.
One of the main differences in Nginx is its event-based model for dealing with new connections. Instead of dedicating a process or thread to each request, this process model only services a request when an event triggers (data read from disk, etc.), and then moves on to another request. This model has become very popular recently, due to how efficient it can be when executed well. Apache 2.4 now includes the event MPM, which works somewhat similarly to Nginx.
The most common Nginx setup for Drupal involves using PHP-FPM (see PHP Apache Module Versus CGI), which provides a pool of PHP processes that Nginx connects to over a Unix or TCP socket. This architecture provides a method to scale and tune Nginx separately from the PHP processes, keeping the Nginx processes very lightweight and able to quickly serve static requests.
Nginx and PHP-FPM can provide a performance enhancement and a reduction of resources as compared to the most common Apache configuration, so it may seem like there is no reason to ever use Apache. This may be true in certain cases, but there are indeed reasons why many people still prefer Apache. One major reason is that when you start caching static items in a reverse proxy and/or CDN—more on this in the next chapter—the web server is being hit much less often for those types of requests, and the performance difference for static files is only noticeable when first loading an item into the external cache. This leaves Apache and mod_php to be just an application server, which limits the advantages of Nginx as a static file server. Another major reason for people to use Apache is if they require htaccess files in their setup, as these are not supported by Nginx. At the end of the day, the question of which httpd server to use is usually decided by team familiarity and other requirements.
Get High Performance Drupal now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
