Chapter 4. Exim Operations Overview

The previous chapter used some fragments of a simple Exim configuration file to show how it goes about delivering a message. Later chapters go into more detail about the various options that can be used to set up configurations that can handle many different circumstances. However, if you have just installed Exim, or if you have inherited responsibility for an Exim system from somebody else, you most likely want to know a little bit about the basic operational aspects. This chapter is an introductory overview; the features that are described reappear later in more detailed discussions, and Chapter 21, covers Exim administration in more detail.

How Exim Identifies Messages

Each message that Exim handles is given a unique message ID when it is received. The ID is 16 characters long, and consists of three parts, separated by hyphens. For example:

11uNWX-0004fP-00

Each part is actually a number, encoded in base 62. The first is the time that the message started to be received, and the second is the ID of the process (the pid) that received the message. The third part is used to distinguish between messages that are received by the same process in the same second. It is almost always 00.

The uniqueness of Exim’s message IDs relies on the fact that Unix process IDs are used cyclically, so in practice there is no chance of the same process ID being reused within one second. For most installations, uniqueness is required only within a single host, and the scheme just described suffices. However, in some cluster configurations, it is useful to ensure that message IDs are unique within the cluster. For example, suppose two hosts are providing identical gateway or hubbing services for some domain, and one of the processors has a catastrophic failure. If its disk can be attached to the other processor, and the message IDs are unique across both systems, spooled message files can simply be moved into the survivor’s spool directory.

Uniqueness across several hosts can be ensured by assigning each host a number in the range 0–255, and specifying it in each Exim configuration. For example:

localhost_number = 4

When this option is set, the third part of the message ID is no longer a simple sequence number. Instead, it is computed as:

            sequence number * 256 + host number

For example, in the following message ID:

11vHQS-0006ZD-4C

the number 4C is 260 in decimal, which is 256 * 1 + 4, so this message ID was generated on host number 4 for the second message received by some process within one second. In the most common case, when the sequence number is zero, the final part of the message ID is just the host number.[42]

Watching Exim at Work

As a new administrator of an MTA, the first questions you should ask are:

  • How do I find out what messages are on the queue?

  • How do I find out what the MTA has been doing?

Exim can output a list of its queue in a number of ways, which are detailed in Section 20.7, in Chapter 20. The most basic is the -bp command-line option. This option is compatible with Sendmail, though the output is specific to Exim:[43]

$ exim -bp
25m  2.9K  0t5C6f-0000c8-00 <caesar@rome.example>
           brutus@rome.example

This shows that there’s just one message, from caesar@rome.example to brutus@rome.example, which is 2.9 KB in size, and has been on the queue for 25 minutes. Exim also outputs the same information if it is called under the name mailq, which is a fairly common convention.[44]

Exim logs every action it takes in its main log file. A log line is written whenever a message arrives and whenever a delivery succeeds or fails. The name of the log file depends on the configuration, with two common choices being /var/spool/exim/log/mainlog or /var/log/exim_mainlog.[45] If you have access to an X Window server, you can run the eximon utility, which displays a “tail” of the main log in a window (see Section 21.7, in Chapter 21). The entries that Exim writes to the log are described in detail in Section 21.1, in Chapter 21.

Exim uses two additional log files that are in the same directory as the main log. One is called rejectlog; it records details of messages that have been rejected for reasons of policy. The other is called paniclog; this is used when Exim encounters some disaster that it can’t handle. The paniclog should normally be empty; it is a good idea to set up some automatic monitoring to let you know if something has been written to it, because that usually indicates there has been an incident that warrants investigation.

The Runtime Configuration File

Exim’s runtime configuration is held in a single text file that you can modify with your favorite text editor. If you make a change, newly started Exim processes will immediately pick up the new file, but the daemon process will not. You have to tell the daemon to reread its configuration, and this is done in the traditional Unix way, by sending it a HUP signal. The process number of the daemon is stored in Exim’s spool directory, so that you can do this by running (as root or exim) the following command:

kill -HUP `cat /var/spool/exim/exim-daemon.pid`

On receiving a HUP signal, the daemon closes itself down, and then restarts in a new process, thereby picking up the new configuration.

Layout of the Configuration File

The runtime configuration file is divided into seven different sections, as shown in Figure 4-1. It consists of the following sections:

Main section

General option settings and input controls

Transport section

The configuration for the transports

Director section

The configuration for the directors

Router section

The configuration for the routers

Retry section

The retry rules for specifying how often Exim is to retry temporarily failing addresses (see Chapter 12)

Rewriting section

The global address rewriting rules (see Chapter 14)

Authenticator section

The configuration for the SMTP authenticators (see Section 15.1, in Chapter 15)

Runtime configuration file

Figure 4-1. Runtime configuration file

The arrows in the figure indicate that the drivers defined in the directors and routers sections refer back to the transports that are defined in the transports section. We saw an example of this in the previous chapter, where the lookuphost router referred to the remote_smtp transport:

lookuphost:
  driver = lookuphost
  transport = remote_smtp

In the actual file, the separators between the sections are lines containing just the word end. Sections that are not needed may be empty, and if they occur at the end of the file, they can be completely omitted. This means that a completely empty file is, in fact, a valid configuration file, but it would not be much use because no way to deliver messages is defined.

The retry and rewriting configuration sections each contain lines in a format that is unique to the section, and we discuss these in later chapters. The remaining sections contain option settings in the form name = value, one per line. Except when we are discussing a specific driver, unqualified references to options always refer to one of the options in the main configuration section.

A Minimal Usable Configuration File

The simplest complete configuration that is capable of delivering both local and remote mail is as follows:

# Main configuration: all defaults taken

end

# Transports: SMTP and local mailboxes

remote_smtp:
  driver = smtp

local_delivery:
  driver = appendfile
  file = /var/mail/$local_part

end

# Directors: local user mailbox only

localuser:
  driver = localuser
  transport = local_delivery

end

# Routers: standard DNS routing

lookuphost:
  driver = lookuphost
  transport = remote_smtp

Lines beginning with # characters are comments, which are ignored by Exim. This example is cut down from the default configuration, and is even simpler in its handling of local domains than the case we considered in the previous chapter; it does not support aliasing or forwarding. Because there are no retry rules in this configuration, messages that suffer temporary delivery failures will be returned to their senders without any retries, so this would not be a very good example to use for real.

Notice that, although there are no settings in the main part of the configuration (so that default values are used for all the options), the end line is still required to mark the end of the section.

Option Setting Syntax

We’ve already seen a number of examples of option settings. Each one is on a line by itself, and they can always be in the form name = value. For those that are on/off switches (Boolean options), other forms are also permitted. The name on its own turns the option on, whereas the name preceded by no_ or not_ turns it off. These settings are all equivalent:

sender_verify
sender_verify = true
sender_verify = yes

So are these:

no_sender_verify
not_sender_verify
sender_verify = false
sender_verify = no

You do not have to use quote marks for option values that are text strings, but if you do, any backslashes in the strings are interpreted specially.[46] For example, the sequence \n in a quoted string is converted into a linefeed character. This feature is not needed very often.

Some options specify a time interval; for example, the timeout period for an SMTP connection. A time interval is specified as a number followed by one of the letters w (week), d (day), h (hour), m (minute), or s (second). You can combine several of these to make up one value. For example, the following:

connect_timeout = 4m30s

specifies a time interval of 4 minutes and 30 seconds.

Macros in the Configuration File

For more complicated configuration files, it may be helpful to make use of the simple macro facility. If a line in the main part of the configuration (that is, before the first end line) begins with an uppercase letter, it is taken as a macro definition, of the form:

               name = rest of line

The name must consist of letters, digits, and underscores, and need not all be in uppercase, though that is recommended. The rest of the logical line is the replacement text, and has leading and trailing whitespace removed. Quotes are not removed.

Once a macro is defined, all subsequent lines in the file are scanned for the macro name; if there are several macros, the line is scanned for each in turn, in the order in which they are defined. The replacement text is not rescanned for the current macro, though it will be for subsequently defined macros. For this reason, a macro name may not contain the name of a previously defined macro as a substring. You could, for example, define the following:

ABCD_XYZ = something
ABCD = something

but putting those definitions in the opposite order would provoke a configuration error.

As an example of macro usage, suppose you have lots of local domains, but they fall into three different categories. You could set up the following:

LOCAL1 = domain1:domain2
LOCAL2 = domain3:domain4
LOCAL3 = dbm;/list/of/other/domains
local_domains = LOCAL1:LOCAL2:LOCAL3

and use the domains option on appropriate directors to handle each set of domains differently. This avoids having to list each domain in more than one place.[47] The values of macros can be overridden by the -D command-line option (see Section 20.6, in Chapter 20).

Hiding Configuration Data

The command-line option -bP requests Exim to output the value of one or more configuration options. This can be used by any caller of Exim, but some configurations may contain data that should not be generally accessible. For example, a configuration that references a MySQL database or an LDAP server may contain passwords for controlling such access. If any option setting is preceded by the word hide, only an admin user is permitted to see its value. For example, if the configuration contains:

hide mysql_servers = localhost/usertable/admin/secret

an unprivileged user gets this response:

$ exim -bP mysql_servers
mysql_servers = <value not displayable>

This feature was added to Exim at Release 3.20.

String Expansions

We have already met a simple string expansion in the following setting:

file = /var/mail/$local_part

for an appendfile transport. Expansions are a powerful feature in configuration files. We explain some more of their abilities in examples in subsequent chapters. If you want to know about everything they can do, skip ahead to Chapter 17, which has the full story. Meanwhile, remember that whenever you see a $ character in a configuration setting, it means that the string will change in some way whenever it is expanded for use.

Incorrect syntax in a string expansion is a serious error, and usually causes Exim to give up what it is trying to do; for example, an attempt to deliver a message is deferred if Exim cannot expand a relevant string. However, there are some expansion constructions that deliberately provoke a special kind of error, called a forced expansion failure; in a number of such cases, these failures just cause Exim to abandon the activity that uses the string, but otherwise to carry on. For example, a forced expansion failure during an attempt to rewrite an address just abandons the rewriting. Whenever a forced expansion failure has a special effect like this, we’ll mention it.

File and Database Lookups

The ability to use data from databases and files in a variety of formats is another powerful feature of Exim’s configuration. Earlier, we showed this director for handling traditional alias files:

aliasfile:
  driver = aliasfile
  file = /etc/aliases
  search_type = lsearch

This looks up data in /etc/aliases by means of a linear search, but it could equally use an indexed file format such as DBM:

aliasfile:
  driver = aliasfile
  file = /etc/aliases.db
  search_type = dbm

or, the aliasing data could be held in a database:

aliasfile:
  driver = aliasfile
  query = select addresses from aliases where name='$local_part'
  search_type = mysql

Each different lookup type is implemented in a different module. Which ones are included in the Exim binary is configured when Exim is built. As far as the main part of Exim is concerned, there is a fixed internal interface (API) to these lookups, and it is unaware of the details of the actual lookup mechanism. However, it does distinguish between the two different kinds of lookup:

Single-key

Use a single key string to extract data from a file. The key and the file have to be specified.

Query-style

Access a database using a query written in the query language of the database package.

As well as being configured in options for drivers such as aliasfile, lookups can be used in expansion strings to replace part of the string with data that comes from a file or database. They can also be used as a mechanism for managing lists of domains, hosts, or addresses. We encounter examples of these uses throughout the book. Full details of all the lookup types and how they operate are given in Chapter 16.

Domain, Host, and Address Lists

The list mechanism is the third facility that, together with string expansion and lookups, is the main building block of Exim configurations. Earlier, we showed the example:

local_domains = tiber.rivers.example:\
                *.cities.example:\
                dbm;/usr/exim/domains

in which the value of local_domains is a colon-separated list containing several types of patterns for matching a domain name. Similar list facilities are used for recognizing specific hosts and email addresses for particular purposes. The full description of lists is in Chapter 18, but we come across plenty of examples before then.

If a colon is actually needed in an item in a string list, it can be entered as two colons. Leading and trailing whitespace on each item in a string list is ignored. This makes it possible to include items that start with a colon, and in particular, certain forms of IPv6 address. For example:

local_interfaces = 127.0.0.1 : ::::1

defines the IPv4 address 127.0.0.1 followed by the IPv6 address ::1. Because the requirement to double colons is particularly unfortunate in the case of IPv6 addresses, a means of changing the separator was introduced with Exim Version 3.15.[48] If a list starts with a left-angle bracket followed by any punctuation character, that character becomes the list separator. The previous example could be rewritten as:

local_interfaces = <; 127.0.0.1 ; ::1

where the separator is changed to a semicolon.

The Default Qualification Domain

In a locally submitted message, if an unqualified address (that is, a local part without a domain) is found in the envelope or any of the header lines that contain addresses, it is qualified using the domain defined by qualify_domain (for senders) or qualify_recipient (for recipients) at the time the message is received. User agents normally use fully qualified addresses, but there are exceptions.

The default value for both these options is the name of the local host. If only qualify_domain is set, its value is used as a default for qualify_recipient. It is common in some installations to use these options to set a generic domain. For example, the Acme Widget Corporation might have two hosts handling its mail, mail1.awc.example.com and mail2.awc.example.com, but would probably require messages created on these hosts to use just awc.example.com as the default domain, rather than the individual hostnames. This can be done by setting the following:

qualify_domain = awc.example.com

in the Exim configurations on both hosts.

Handling Frozen Bounce Messages

When a message on Exim’s queue is marked as frozen, queue runner processes skip over it and do not attempt to deliver it. One reason why a message might be frozen is mentioned in Section 3.11.2, in Chapter 3; namely, there may be a problem with Exim’s configuration. However, by far the most common reason that a message becomes frozen is that it is a bounce message that cannot be delivered. Such messages are often the result of incoming junk mail that is addressed to an unknown local user, but which contains an invalid sender address that causes the resulting bounce message to fail.[49]

In order to avoid mail loops, Exim does not let a failing bounce message give rise to another bounce message. Instead, Exim freezes the message to bring it to the postmaster’s attention. On busy systems, frozen messages of this type may be quite common.

Some administrators do not have the human resources to inspect each frozen message in order to determine what the problem is, and their policy may be to discard such failures. Exim can be configured to do this by setting ignore_errmsg_errors, which has the effect of discarding failing addresses in bounce messages (the action is logged). A slightly less harsh option is to set ignore_errmsg_errors_after, which allows such failures to be kept for a given time before being discarded. For example, the following:

ignore_errmsg_errors_after = 12h

keeps such messages for 12 hours. After the first failure, the message is frozen as in the default case, but after it has been on the queue for the specified time, it is automatically unfrozen at the next queue run; if delivery fails again, the message is discarded.

Reducing Activity at High Load

In the main section of the configuration file, there are several options that allow you to limit or reduce Exim’s activities when a lot of mail arrives at once, or when the system load is too high. “System load” in this sense is the average number of processes in the operating system’s run queue over the last minute, a figure that can be obtained by running the uptime command to obtain output like this:

4:15pm  up 1 day(s), 22:23,  75 users,  load average: 0.09, 0.15, 0.22

The first of the “load average” figures is the one-minute average. On an unloaded system, it is a small number, usually well under 10. When it gets too high, everything slows down; reducing the load created by mail reception and delivery can alleviate the impact of this.

Delaying or Suspending Delivery When the Load Is High

By default, Exim starts a delivery process for each new message, and uses its queue for messages that cannot be delivered immediately. You can use various configuration options to modify Exim’s behavior when system load is sufficiently high.

If the system load is higher than the value of queue_only_load, automatic delivery of incoming messages does not occur; instead, they wait on Exim’s queue until the next queue runner process finds them. The effect of this is to serialize their delivery because a queue runner delivers just one message at a time. This reduces the number of simultaneously running Exim processes without significantly affecting mail delivery, as long as queue runners are started fairly frequently. For example, a setting of:

queue_only_load = 8

is a useful insurance against an overload caused by the simultaneous arrival of a large number of messages. If, on the other hand, deliver_load_max is set to:

deliver_load_max = 10

no deliveries at all are done if the load is higher that this setting, and if this is detected during a queue run, the remainder of the run is abandoned. A different threshold can be specified for queue runs by setting deliver_queue_load_max, for example:

deliver_queue_load_max = 14

If combined with the previous setting, this would allow deliveries only from queue runs when the load was between 10 and 14.

These three options are not fully independent. If queue_only_load (described earlier) is set, forcing all deliveries to take place in queue runs above a given load level, you can set either deliver_load_max or deliver_queue_load_max to a higher value in order to suspend all deliveries when the load is above that value. For example:

queue_only_load = 8
deliver_queue_load_max = 11

Setting both deliver_load_max and deliver_queue_load_max is useful only when queue_only_load is not set.

Deliveries that are forced with the -M or -qf command-line options override these load checks.

Suspending Incoming Mail When the Load Is High

There is no option for stopping incoming messages from local processes when the load is high, but mail from other hosts can be stopped or restricted to certain hosts. If smtp_load_reserve is set, and the system load exceeds its value, incoming SMTP calls over TCP/IP are accepted only from those hosts that match an entry in smtp_reserve_hosts. If this is unset, all calls from remote hosts are rejected with a temporary error code. For example, with the following:

smtp_load_reserve = 5
smtp_reserve_hosts = 192.168.24.0/24

only hosts in the 192.168.24.0/24 network can send mail to the local host when its load is greater than 5. The host list in smtp_reserve_hosts is also used by the smtp_accept_reserve option, which is described later.

If you are running user agents that submit messages by making TCP/IP calls to the local interface, you should probably add 127.0.0.1 (or ::1 in an IPv6 system) to smtp_reserve_hosts, to allow these submissions to proceed even at high load.

Controlling the Number of Incoming SMTP Connections

It’s a good idea to set a limit on the number of simultaneous incoming SMTP calls, because each one uses the resources required for a separate process. Exim has the smtp_accept_max option for this purpose. The default setting is 20, which is reasonable for small to medium-sized systems, but if you are running a large system, increasing this to 100 or 200 is reasonable.

You can reserve some of these incoming SMTP slots for specific hosts. If you set smtp_accept_reserve to a value less than smtp_accept_max, that number of slots is reserved for the hosts listed in smtp_reserve_hosts. This feature is typically used to reserve slots for hosts on the local LAN so that they can never be all taken up by external connections. For example, if you set:

smtp_accept_max = 200
smtp_accept_reserve = 40
smtp_reserve_hosts = 192.168.24.0/24

then once 160 connections are active, new connections are accepted only from hosts in the 192.168.24.0/24 network.

You can also set smtp_accept_queue; if the number of simultaneous incoming SMTP calls exceeds its value, automatic delivery of incoming SMTP messages is suspended; they are placed on the queue and left there for the next queue runner. The default for this option is unset, so that all messages are delivered immediately.

If new SMTP connections arrive while the daemon is busy setting up a process to handle a previous connection, they are held in a queue by the operating system, waiting for the daemon to request the next connection. The size of this queue is set by the smtp_connect_backlog option, which has a default value of 5. On large systems, this should be increased, say to 50 or more.

Checking for Free Disk Space

You can arrange for Exim to refuse incoming messages temporarily if the amount of free space in the disk partition that holds its spool directory falls below a given threshold. For example:

check_spool_space = 50M

specifies that no mail can be received unless there is at least 50 MB of free space in which to store it.[50] The check is not a complete guarantee because of the possibility of several messages arriving simultaneously.

Limiting Message Sizes

It is a good idea to set a limit on the size of message your host will process. There is no default in Exim, but you can set, for example:

message_size_limit = 20 M

to apply a limit of 20 MB per message.

Parallel Remote Delivery

If a message has a number of recipients on different remote hosts, Exim does these deliveries one at a time, unless you set remote_max_parallel to a value greater than one. On systems that are handling mostly personal mail, where messages typically have at most two or three recipients, this is not an important issue. However, on systems that are handling mailing lists, where a single address may end up being delivered to hundreds or even thousands of addresses, parallel delivery can make a noticeable improvement to performance. Setting, for example:

remote_max_parallel = 10

allows Exim to create up to 10 simultaneous processes to do remote deliveries for a message that has multiple recipients. Note that this option applies to the parallel delivery of individual messages; it is not an overall limit on Exim delivery processes.

Controlling the Number of Delivery Processes

In a conventional configuration, where Exim attempts to deliver each message as soon as it receives it, there is no control over the number of delivery processes that may be running simultaneously. On a host where processing mail is just one activity among many, this is not usually a problem. However, on a heavily loaded host that is entirely devoted to delivering mail, it may be desirable to have such control. It can be achieved by suppressing immediate delivery (which means that all deliveries take place in queue runs) and limiting the number of queue runner processes, for example, by placing these settings in the cofiguration file:

queue_only
queue_run_max = 15

Setting queue_only disables immediate delivery, and queue_run_max specifies the maximum number of simultaneously active queue runners. The maximum number of simultaneous delivery processes is then:

queue_run_max x remote_max_parallel

With this kind of configuration, you should arrange to start queue runner processes frequently (up to the maximum number) so as to minimize any delivery delay. This can be done by starting a daemon with an option such as -q1m, which starts a new queue runner every minute.[51]

Large Message Queues

Back in Chapter 3, we explained that Exim is designed for an environment in which most messages can be delivered almost instantaneously. Consequently, the queue of messages awaiting delivery is expected to be short. In some situations, nevertheless, large queues of messages occur, resulting in a large number of files in a single directory (usually called /var/spool/exim/input). This can affect performance significantly. To reduce this degradation, you can set:

split_spool_directory

When this is done, the input directory is split into 62 subdirectories, with names consisting of a single letter or digit, and incoming messages are distributed between them according to the sixth character of the message ID, which changes every second. This requires Exim to do a bit more work when it is scanning through the queue, but the directory access performance is much improved when there are many messages on the queue.

Large Installations

One of the advantages of Exim’s decentralized design is that it scales fairly well, and can handle substantial numbers of mailboxes and messages on a single host. However, when the numbers start to get really large, a conventional configuration may not be able to cope. In this section, a number of general observations are made that are relevant to large installations.

Linear Password Files

Above a thousand or so users, the use of a linear password file is extremely inefficient, and can slow down local mail delivery substantially. Some operating systems (for example, FreeBSD) automatically make use of an indexed password file, or can be configured to do so, which is one easy way round this problem if you happen to be using such a system. The alternative is to make use of NIS or some other database for the password information, provided that it operates quickly.

Even if you don’t have any login accounts on your mail server, you still need some kind of list of local users, and it is important to make the searching of this list as efficient as possible.

It is not only mail delivery that provokes password file lookups. If you are running a POP daemon, a password check happens every time a POP client connects; in environments where users remain connected and leave their POP MUAs running, these checks happen every few minutes for each user, whenever the POP client checks for the arrival of new mail.[52] IMAP is much less expensive than POP in this regard, because it establishes a session that remains active, so there is a password check only at the start.

Mailbox Directories

You will get very bad performance if you have too many mailboxes in a single directory. What constitutes too many depends on your operating system; the default Linux filing system starts to degrade at about one thousand files in a single directory, whereas for Solaris the number is around ten thousand. This applies whether you are using individual files as multimessage mailboxes, or delivering messages as separate files in a directory.

The solution to this is to use multiple directory levels. For example, instead of storing jimbo’s mailbox in /var/mail/jimbo, you could use /var/mail/j/jimbo. Splitting on the initial character(s) of the local part is easy to implement, but it is not as good as using some kind of hashing function. Exim’s string expansion facilities can be used to implement either a substring-based or hash-based split. Of course, you will have to ensure that all the programs that read the mailboxes use the same algorithm.

For a very large number of mailboxes, a two-level split is recommended, using Exim’s numeric hash function, as in this example:

/var/mail/${nhash_8_512:$local_part}/$local_part

The hashing expansion generates two numbers separated by a slash, in this case using the local part as the data and ensuring that the numbers are in the ranges 0–7 and 0–511. This example places jimbo’s mailbox in /var/mail/6/71/jimbo. The initial split could be between different disks or file servers, and the second one could be between directories on the same disk.

Simultaneous Message Deliveries

If two messages for the same mailbox arrive simultaneously, they cannot both be delivered at once if the mailbox is just a single file. One delivery process has to wait for the other, thus tying up resources. The default way that Exim does this (in the appendfile transport) is by sleeping for a bit, and then starting the process of locking the mailbox from scratch. This is the safest approach, and the only way to operate when lock files are in use.

Attempts to lock a mailbox continue for a limited time. If a process cannot gain access to a mailbox within that time, it defers delivery with the error message:

failed to lock mailbox

and Exim will try the delivery again later. If you see a lot of these messages in the main log file, it is an indication that there is a problem with contention for the mailbox.

If you are in an environment in which only fcntl() locks are used, and no separate lock files, you can configure the appendfile transport to use blocking calls, instead of sleeping and retrying. This gives better performance because a waiting process is released as soon as the lock is available instead of waiting out its sleep time. In this environment, this single change can make a big performance difference.

Note

If the mailbox files are NFS-mounted, and more than one host can access them, you must not disable the use of lock files. If you do, you are likely to end up with mangled mailboxes.

The whole problem of locking can be bypassed if you use mailboxes in which each message is stored in a separate file.[53] One example of this type of message storage, called maildir format, is now quite popular, and has support in a number of MUAs and other programs that handle mailboxes. Because each message is entirely independent, no locking is required, several messages can be delivered simultaneously, and old messages can even be deleted while new ones are arriving. See Section 9.5.5, in Chapter 9, for a description of how to configure Exim to use maildir format.

Minimizing Name Server Delays

A busy general mail server makes a large number of calls to the DNS. For this reason, you should arrange for it to run its own name server, or make sure that there is a name server running on a nearby host with a high-speed connection, typically on the mail server’s LAN. Ensure that the name server has plenty of memory so that it can build up a large cache of DNS data.

Storing Messages for Dial-up Hosts

You should not plan to store large numbers of messages for intermittently connected clients in Exim’s spool. As explained in Section 12.12, in Chapter 12, it is much better to have them delivered into local files, for onward transmission by some other means.

Hardware Configuration

If you keep increasing the workload of an Exim installation, disk I/O capacity is what runs out first. Each message that is handled requires the creation and deletion of at least four files. Large installations should therefore use disks with as high a performance as possible. Also, it does not make sense to keep on increasing the performance of the processor if the disks cannot keep up.

Better overall performance can be obtained by splitting up the work between a number of different hosts, each with its own set of disks. For example, separate hosts can be used for incoming and outgoing mail. A general form of scalable configuration that is used by some very large installations is shown in Figure 4-2.

Large system configuration

Figure 4-2. Large system configuration

This configuration has separate servers for incoming and outgoing messages, and can be expanded “sideways” by the addition of more servers (indicated by the dashed lines) as necessary. Incoming mail is delivered to one or more file servers, which hold local mailboxes in a split directory structure, as described earlier, and also messages that are waiting for dial-up hosts. The mailboxes are accesssed from POP and IMAP servers, and the dial-up hosts use yet another server to access their stored mail.

The outgoing servers send messages that they cannot deliver in a short time to a long-term outgoing server, so as not to impact their performance with very long message queues. This can be implemented using fallback_hosts on appropriate drivers on the main servers, or using the $message_age variable to move messages after some fixed time.



[42] In this type of configuration, the maximum sequence number is 14. If more than 14 messages are received by one process within one second, a delay of one second is imposed before reading the next message, in order to allow the clock to tick.

[43] In examples of commands that are run from the shell, the input is shown in boldface type.

[44] Many operating systems are set up with the mailq command as a symbolic link to sendmail; if this in turn has been linked to exim, the mailq command will “just work.”

[45] It is possible to configure Exim to use syslog instead, but this has several disadvantages.

[46] Exim recognizes only double-quote characters for this purpose.

[47] However, there may be a difficulty if you are using negated items in the list. This is explained in Section 18.1, in Chapter 18.

[48] This applies to all lists, with the exception of log_file_path andtls_verify_ciphers.

[49] It is possible to do some checking on the sender and recipients before a message is accepted, as described in Section 13.7.1, in Chapter 13. This can dramatically cut down the number of frozen messages, but there may still be undeliverable messages that get through.

[50] Digits in a numerical option setting can always be followed by K or M, which cause multiplication by 1024 and 1024x1024, respectively.

[51] See Section 11.7, in Chapter 11, for more details of the daemon process.

[52] Users have been known to configure their MUAs to check as often as every 20 or 30 seconds; such usage will eat up your machine and should be strongly discouraged.

[53] There is still some locking, of course, between processes that are updating the mailbox directory, but it is handled internally in the file system and is no longer Exim’s responsibility.

Get Exim: The Mail Transfer Agent now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.