qmail

Chapter 4. Getting Comfortable with Qmail

This chapter guides you through the basics of running qmail and delivering mail to users on your qmail host. It’s quite possible to run qmail in parallel with your old mail system, which is usually a good idea during a transition, so you can do everything in this chapter while leaving your old mail system in place.

Mailboxes, Local Delivery, and Logging

Before you start up qmail, you must make a few configuration decisions. None of these are irrevocable, but if you know what you want, it’s easier to set them that way at first than to change them later.

Mailbox Format

Qmail supports two mailbox formats: the traditional mbox and Dan’s newer Maildir. I won’t belabor the difference here (see Chapter 10 for more details) other than to note that mbox stores all its messages in a single file and is supported by all existing Unix mail software, while Maildir stores each message in a separate file in a directory, and is supported by a reasonable set of software (including procmail, the mutt MUA, and several POP and IMAP servers) but not as many as mboxes. If you’re converting from an existing mail system that uses mboxes, it’s easier to keep using mboxes, but if you’re starting from scratch, go with Maildirs.

Local Delivery

If you use mbox files, qmail normally puts the incoming mailboxes in users’ home directories. That is, for user fred, the mailbox would be ~fred/Mailbox. Older mail programs often put all of the mailboxes into /var/mail. For both security and disk management reasons, it’s better to put the mail in the user’s home directory with his or her other files, but if you have existing mailboxes in /var/mail, it’s not hard to persuade qmail to continue delivering mail there.

If you’re converting from an older MTA, you can either set up qmail to deliver into the same mailboxes as the old MTA or, if you’re feeling cautious, set qmail to deliver into Maildirs or home directory mboxes while the old MTA still delivers to /var/mail. (The disadvantage is that once you’re happy with qmail, you have to convert and merge the old mailboxes. See Section 4.7 later in this chapter.)

Logging

The traditional way to make log files is with the system syslog facility. It turns out that syslog is a serious resource hog and on a busy system can lose messages. On a small system this doesn’t matter, but on a busy mail host, it sucks up significant resources that otherwise could be devoted to something more useful. Dan Bernstein wrote a logging program called multilog, part of the daemontools package, which is far faster and more reliable than syslog, but not particularly compatible with it. If you’re sure that syslog won’t be a bottleneck, go ahead and use it, but if you might eventually want to use multilog, you’re better off starting with it because switching a running system is a pain in the neck.

An Excursion into Daemon Management

A daemon is a program that runs in the background (that is, without interacting with a user) and is useful. On Unix systems, there are two kinds of daemons: the ones that run continuously and the ones that run on demand. Familiar Unix examples of continuous ones include named, the DNS server, and httpd, the Apache web server, while on-demand ones include servers for network services, such as telnet and ftp, and cleanup scripts run once a day or once a week. The on-demand ones are all started from continuous servers such as cron for time-based services, and inetd or tcpserver (the qmail replacement for inetd) for network services.

The daemontools package provides a consistent way to run continuous daemons, optionally (but almost invariably) also arranging to collect log information that the daemons produce. The two key programs are supervise, which controls a single daemon, and svscan, a “superdaemon,” which controls multiple copies of supervise and connects each daemon with its logger.

For each daemon to be controlled, supervise uses a directory containing information about the daemon. The only file that you must create in that directory is run, the program to run. Although it can be a link to the daemon, it’s usually a short shell script that sets up the environment and then exec’s the daemon. The supervise program creates a subdirectory also called supervise, where it stores info about what it’s doing. Once supervise is running, you can use the svc program to stop and start the daemon, and send signals to it. (This consistent way to signal daemons is one of supervise’s greatest strengths.)

To run supervise, follow these two steps: create /service, which you do with a regular mkdir command as the super-user, and start svscan, which I cover in the next section. Once svscan is running, it looks at /service and starts a supervise process in each of its subdirectories. Every five seconds it looks again and creates new processes for any new subdirectories. If a subdirectory has a sub-subdirectory called log, svscan arranges to log the output of the program. In this case, it starts a pair of processes connected by a pipe, equivalent to:

supervise subdir | supervise subdir/log

The log subdirectory contains a run file that invariably runs multilog to write the output into a rotating set of log files.

Starting a Daemon

One of the least standardized aspects of Unix and Unix-like systems is the way that you start daemons at system boot time. Even if you use supervise as I recommend, you still must start the svscan daemon to get everything else going. Here are some hints to start svscan. If you ignore my advice and run daemons directly, start each of them the way I recommend you start svscan.

Versions 0.75 and later of daemontools include a startup script for svscan called svscanboot, and the daemontools installation process tries, usually successfully, to edit a call to that program into your system startup scripts. It sets up the environment and runs svscan, piping its output into a new program called readproctitle that copies anything it reads on top of its program arguments, which means that any error messages from svscan will show up in ps listings in the arguments to readproctitle. This kludge makes it possible to see what’s wrong if svscan has trouble starting up or starting supervise for any of the directories under /service:

Single rc file

Solaris and some versions of BSD put all of the startup commands in a file called /etc/rc, usually with local modifications in /etc/rc.local. If the daemontools installation hasn’t already done so, add this line to either of those files:

/command/svscanboot

If it’s convenient to reboot your system, do so. If not, just run svscanboot from a root shell prompt, detaching it from the terminal:

# daemon /command/svscanboot      # if you have the "daemon" command
# csh -cf '/command/svscanboot &' # if not

Either way, check with ps to be sure that svscan is running.

SysV /etc/inittab

System V and its derivatives and clones, including most versions of Linux, start daemons from a file called /etc/inittab. If the daemontools installation hasn’t already done so, add this line to the end of it:

SV:123456:respawn:/command/svscanboot

Then, to tell the system to rescan inittab, type:

kill -HUP 1

Again, check with ps to be sure that svscan is running.

Setting Up the Qmail Configuration Files

The final hurdle before starting up qmail is to create a minimal set of configuration files. The qmail distribution includes a script called config that makes a set of configuration files that’s usually nearly right. I suggest you run the config script, then look at the files to see what it did and fix the files up as needed. All of the configuration files are in /var/qmail/control. The ones you need to create include:

me

The name of this host, e.g., mail.example.com. This provides the default to use for many other configuration files.

defaulthost

The hostname to add to unqualified addresses in submitted mail. If your email addresses are of the form [e-mail], this would contain example.com, so that mail to, say, fred is rewritten to [e-mail]. (Note that this rewriting happens only to locally submitted mail sent via qmail-inject, not to mail that arrives via SMTP.)

defaultdomain

The domain to add to unqualified domains in submitted mail addresses, usually your base domain, such as example.com. This would rewrite fred@duluth to [e-mail]. (This rewriting also happens only in locally submitted mail.)

locals

Domain names to be delivered locally, one per line. Mail to any domain listed in locals is delivered by treating the mailbox part as a local address. This usually contains the name of the host and the name of the domain used for user mailboxes, such as example.com and mail.example.com. Do not list virtual domains (domains hosted on this machine but with their own separate sets of mailboxes) in locals. I discuss them later.

rcpthosts

Domains for which this host should accept mail via SMTP. This generally contains all of the domains in locals, as well as any virtual domains and any domains for which this host is a backup mail server. If rcpthosts does not exist, qmail accepts and delivers mail for any domain, a severe misconfiguration known as an “open relay,” which will be hijacked by spammers. Be sure your rcpthosts file exists before starting qmail. If you haven’t defined any virtual domains, just copy locals to rcpthosts.

There are over 20 more control files, but the rest can be left for later.

Starting and Stopping Qmail

Starting qmail is easy in principle. You run qmail-start and it starts the four communicating daemons that qmail needs. Two details complicate the situation: the default delivery instructions, and connecting the daemons to whatever you want to use for logging.

Because the daemontools package of which supervise is a part wasn’t written until after qmail 1.03 was released, all of the provided startup files use splogger to send the log information to syslog. I find daemontools greatly preferable, so I primarily discuss how to set up qmail using supervise.

Choosing a Startup File

Qmail 1.03 comes with a selection of startup files you can use, either directly or as a starting point for a customized startup file of your own. You can find the startup files in /var/qmail/boot. None of them are usable directly with daemontools, but they’re useful as templates. The differences among them only affect what happens when mail is delivered to a user who has no .qmail file, because the only difference is the string to use as a default .qmail. They include:

binm1

Default delivery using /usr/libexec/mail.local, the 4.4BSD mail delivery agent, which puts mail in /var/spool/mail

binm1+df

Same as binm1 , also providing dot-forward emulation

binm2

Default delivery using /bin/mail with SVR4 flags, which also puts mail in /var/spool/mail

binm2+df

Same as binm2, also providing dot-forward emulation

binm3

Default delivery using /bin/mail with flags for older versions of Unix; puts mail in /var/spool/mail

binm3+df

Same as binm3, also providing dot-forward emulation

home

Default delivery using qmail’s internal qmail-local, which puts mail in the user’s Mailbox

home+df

Same as home, also providing dot-forward emulation

proc

Default delivery using procmail, which puts mail wherever procmail puts it, usually /var/spool/mail unless you patch procmail as I describe later

proc+df

Same as proc, also providing dot-forward emulation

Which flavor of startup depends mostly on your existing mail configuration. If you use procmail, keep using it. If you have a lot of users with .forward files, use a dot-forward version. (If you only have a few .forward files, it’s easier to hand-translate them into .qmail files.) I don’t recommend using any of the old mail delivery programs unless you really, really want to keep delivering mail in /var/mail. For testing and usually for production, I suggest either plain home-directory mailbox delivery or procmail.

Assuming that you’ve installed and started daemontools as suggested earlier in this chapter, you now must create a pair of supervise directories for qmail. I use /var/qmail/supervise/qmail-send and /var/qmail/supervise/qmail-send/log to be consistent with the widely used qmail setup instructions at http://www.lifewithqmail.org. Create them like this (as the super-user, which is why the following command lines start with a # prompt):

# mkdir /var/qmail/supervise/qmail-send
# mkdir /var/qmail/supervise/qmail-send/log
# chown root /var/qmail/supervise/qmail-send /var/qmail/supervise/qmail-send/log
     
# mkdir /var/qmail/supervise/qmail-send/log/main
# chown qmaill /var/qmail/supervise/qmail-send/log/main

The log directory contains a subdirectory main that contains the actual logs. It belongs to qmaill, the qmail log pseudo-user.

Then create run files in both the main qmail and log directories, as in Example 4-1.

Example 4-1. qmail run

1. #!/bin/sh
2.
3.  limit open 1000
4.  limit maxproc 100
5. 
6.  exec env - PATH="/var/qmail/bin:$PATH" \
7.   qmail-start ./Mailbox

The two limit commands on lines 3 and 4 ensure that qmail can run many deliveries in parallel. Set maxproc to be larger than the number of parallel remote deliveries permitted. (By default the number of deliveries is 20, but you’ll probably want to increase it unless you have a very slow or overloaded network connection, or handle a very small amount of mail.) Also set open, the per process open-file limit, to be at least twice the greater of the number of simultaneous local or remote deliveries permitted, because qmail-lspawn and qmail-rspawn use two pipes per delivery subprocess. Then the exec env command on line 6 clears out the environment, sets PATH to a known value, and runs qmail-start. The argument to qmail-start is copied from the example in /var/qmail/boot/home to default deliveries to Mailbox in a user’s home directory. (You can copy the startup command from one of the other example files, such as boot/proc.)

Also create log/run to start up the logging process, as in Example 4-2.

Example 4-2. qmail log/run

 1. #!/bin/sh
 2.   exec setuidgid qmaill \
 3.   multilog t s4000000 ./main

The setuidgid command switches to the qmail log pseudo-user, then runs multilog to store qmail’s output into rotating log files. The arguments say to prefix each line with a time stamp, and to create log files of up to 4 MB in the subdirectory main.

Supervise starts the run scripts directly, so they need to be executable:

# chmod +x /var/qmail/supervise/qmail-send/run
# chmod +x /var/qmail/supervise/qmail-send/log/run

Be sure the initial #!/bin/sh line is present in each of the scripts so they are self-running.

Fire `er Up

Once you’ve created the run files, it’s time to start qmail:

# ln -s /var/qmail/supervise/qmail-send /service

Assuming you have svscan running, within a few seconds of making the line, qmail will start. Look at the log file /var/qmail/supervise/qmail-send/log/main/current to be sure. It should contain a line similar to this:

status: local 0/10 remote 0/20

Now try telling qmail to send some local mail:

$ /var/qmail/bin/qmail-inject
To: me
     
my first qmail message
^D

(Use your own username instead of me, of course.) The log file should now contain lines logging the local delivery:

new msg 175283
info msg 175283: bytes 230 from <[e-mail]> qp 5524 uid 100
starting delivery 1: msg 175283 to local [e-mail]
status: local 1/10 remote 0/20
delivery 2: success: did_0+0+1/
status: local 0/10 remote 0/20
end msg 175283

Your file Mailbox should contain the message. If not, the log should contain evidence of the problem, which is usually files or directories not created with the correct owner or permissions.

Now try a message to a nonexistent address:

$ /var/qmail/bin/qmail-inject 
To: baduser
     
oops
^D

In this case, qmail attempts to deliver the message, then finds it can’t and sends back a failure notice, which should end up in your mailbox. The log should look like this:

new msg 175283
info msg 175283: bytes 212 from <[e-mail]> qp 5690 uid 100
starting delivery 1: msg 175283 to local [e-mail]
status: local 1/10 remote 0/20
delivery 1: failure: Sorry,_no_mailbox_here_by_that_name._(#5.1.1)/
status: local 0/10 remote 0/20
bounce msg 175283 qp 5695
end msg 175283
new msg 175284
info msg 175284: bytes 746 from <> qp 5695 uid 124
starting delivery 2: msg 175284 to local [e-mail]
status: local 1/10 remote 0/20
delivery 2: success: did_0+0+1/
status: local 0/10 remote 0/20
end msg 175284

Finally, try a test message to a mailbox on a remote system. If you don’t have a remote mailbox handy, use the author’s autoresponder at [e-mail]. (It will send a response message telling you how clever you were to write to it, with a blurb for my books.)

$ /var/qmail/bin/qmail-inject 
To: [e-mail]
     
boing
^D

The logs show the remote delivery, including the IP address of the remote system and the remote system’s response:

new msg 175283
info msg 175283: bytes 223 from <[e-mail]> qp 6808 uid 100
starting delivery 3: msg 175283 to remote [e-mail]
status: local 0/10 remote 1/20
delivery 3: success: 208.31.42.43_accepted_message./Remote_host_said:_250_ok_
993021663_qp_16918/
status: local 0/10 remote 0/20
end msg 175283

If all three of these tests work, you have correctly installed qmail. Congratulations!

Stopping Qmail

When you’re running qmail for real, you’ll almost never want to stop it, but when debugging, just tell supervise to stop qmail and mark it as down:

# svc -td /service/qmail-send

If there are deliveries in progress, qmail will wait for them to finish or time out. Then it exits. Use svc -u to bring qmail back up.

Incoming Mail

Next, install the SMTP daemon to receive incoming mail. If you have another mail system already running, set up qmail’s SMTP daemon for testing on a different port than the standard port 25.

Configuration Files

The SMTP daemon only needs one configuration file to run: /var/qmail/rcpthosts. For simple applications, rcpthosts can contain the same list of domains as locals. It is very important that you set up rcpthosts before starting your SMTP daemon. If you don’t, your mail system will be an “open relay,” which will transmit mail from anywhere to anywhere and be abused by spammers and blacklisted.

A little later we’ll also be setting up a control file to tell the daemon what IP addresses are assigned to local users allowed to relay mail.

Setting Up the Daemons

Setting up SMTP involves three layers of daemons. Supervise runs tcpserver, which waits for incoming network connections. Each time a remote system connects, tcpserver starts a copy of qmail-smtpd, which collects the incoming message and passes it to qmail-queue for delivery. To run it under supervise, create a pair of directories, and call them /var/qmail/supervise/qmail-smtpd and /var/qmail/supervise/qmail-smtpd/log:

# mkdir /var/qmail/supervise/qmail-smtpd
# mkdir /var/qmail/supervise/qmail-smtpd/log
# chown root /var/qmail/supervise/qmail-smtpd  /var/qmail/supervise/qmail-smtpd/log
     
# mkdir /var/qmail/supervise/qmail-smtpd/log/main
# chown qmaill /var/qmail/supervise/qmail-smtpd/log/main

The run script eventually becomes rather complex as you add code to handle local versus remote users, spam filters, and the like, but this is adequate to start (see Example 4-3).

Example 4-3. Running the SMTP daemon

 1. #!/bin/sh
 2. limit datasize 3m
 3. exec tcpserver \
 4.    -u000 -g000 -v -p -R \
 5.      0 26 \
 6.       /var/qmail/bin/qmail-smtpd 2>&1

The limit command on line 2 defends against a denial-of-service attack in which the attacker feeds the SMTP daemon a gargantuan message that fills up all of memory and crashes the machine. Then the tcpserver command on line 3 accepts SMTP connections and runs qmail-smtpd for each one. The -u and -g flags on line 4 set the user and group numbers; substitute the values on your system for qmaild. The -v flag does verbose logging (recommended, it’s not that verbose) and -p does “paranoid” validation of deduced hostnames of remote systems. The -R flag means to not try to collect ident information from the remote host. (Ident information is rarely useful and a failed ident request can stall the daemon startup for 25 seconds.) On line 5, host number 0 means to accept connections on any IP address assigned to this machine, and 26 means to use port 26 rather than standard SMTP port 25, which allows you to run the daemon for testing without interfering with an existing MTA on port 25. (If there’s no other MTA running, you might as well use port 25.) Finally, line 6 has the command for tcpserver to run once a connection is open. At the end, 2>&1 combines any output to standard error with the regular output so both appear in the log files.

The log/run file is the same as the one for qmail logging:

 1. #!/bin/sh
 2.   exec setuidgid qmaill \
 3.      multilog t s4000000 ./main

Once you have all the files created, symlink the smtpservice directory so svscan starts it up:

# chown +x /var/qmail/supervise/qmail-smtpd/run
# chown +x /var/qmail/supervise/qmail-smtpd/log/run
     
# ln -s /var/qmail/supervise/qmail-smtpd /service

If you look at log/current, you should see this:

tcpserver: status: 0/40

Now try sending yourself some mail, using Telnet to talk to the SMTP server:

$ telnet localhost 26
Trying 127.0.0.1...
Connected to localhost.example.com.
Escape character is '^]'.
220 example.com ESMTP
helo localhost
250 tom.example.com
mail from:<[e-mail]>
250 ok
rcpt to:<[e-mail]>
250 ok
data
354 go ahead
Subject: a message
     
hi
.
250 ok 993620568 qp 5602
quit
221 example.com
Connection closed by foreign host.

The log file for the SMTP daemon in /service/qmail-smtpd/log/main/current should show the connection to the daemon:

tcpserver: status: 1/40
tcpserver: pid 5582 from 127.0.0.1
tcpserver: ok 5582 localhost:127.0.0.1:26 localhost:127.0.0.1::54044
tcpserver: end 5582 status 0
tcpserver: status: 0/40

Check the qmail log in /service/qmail-send/log/main/current to be sure the message has been delivered:

new msg 175297
info msg 175297: bytes 198 from <[e-mail]> qp 5845 uid 120
starting delivery 1: msg 175297 to local [e-mail]
status: local 1/10 remote 0/20
delivery 1: success: did_0+0+1/
status: local 0/10 remote 0/20
end msg 175297

(The numbers vary somewhat; qmail uses the inode number of the spool file as the msg number.)

If this works, you now have a working mail system. If not, the qmail and tcpserver logs should give you hints about what’s wrong. The most likely problems are missing directories or configuration files, or incorrect file modes. Also be sure you just didn’t make a typing error while telnetting to the SMTP port.

If you want to stop the SMTP daemon, use svc -td just as you did to stop qmail. It’s perfectly OK for the SMTP daemon to be running while qmail isn’t. In this case, incoming mail is queued but won’t be delivered until qmail is started.

Once you believe that qmail works, kill any other mail daemon listening on port 25, change port 26 to 25 in the run file, and restart the daemon with svc -t to start receiving mail on the standard port. The rest of the examples in this chapter use port 25 rather than port 26, on the assumption that qmail is now your production mail system, but for testing, they all work equally well on port 26.

Make Some Mail Aliases

Every mail system on the Internet should define a few standard addresses, such as postmaster, webmaster, and mailer-daemon. (The last is the return address in the From: line of bounce messages.) To define an address, just create a .qmail file for the address in the home directory of the alias user:

# echo fred > /var/qmail/alias/.qmail-postmaster
# echo fred > /var/qmail/alias/.qmail-mailer-daemon

(If your login name isn’t fred, adjust these examples appropriately.)

Now try using qmail to send mail to postmaster and check that it lands in your mailbox. On a busy system, postmaster gets a lot of mail and you’ll probably want to use procmail (discussed later) to sort it to some place other than your personal mailbox.

Relaying for Local Users

Your qmail system most likely is a mail hub for a bunch of PCs or workstations. You want to accept mail destined for any address from your users so they can use your mail hub as a “smart host,” but for security reasons, you want to accept only mail destined for your own network from elsewhere. Setting up relay control involves two steps: defining the list of locally handled domains for which you’ll accept mail from outside and defining the addresses of hosts that are allowed to relay. A third step is to treat mail from local PCs as “injected” mail that must have its headers validated and completed. (As opposed to mail that’s relayed from other systems that should already have valid headers, but I save that for later in this chapter.)

You should have already put the list of locally handled domains into /var/qmail/rcpthosts. (If not, do so now.)

Arranging for your users to relay is a little more complicated, because tcpserver and qmail-smtpd provide a general scheme that permits mail to be treated differently depending on what IP address it is received from. You create a file of IP address ranges and environment variables to set and compile it into a CDB database that tcpserver reads. When it receives a connection from an IP address in the database, it passes the corresponding environment variable to qmail-smtpd. For relay control, the relevant variable is RELAYCLIENT. If it’s set, qmail-smtpd permits mail to any address, not just the ones in relayhosts, and appends the contents of RELAYCLIENT to each envelope recipient address.

Different people have different preferences for the location of the TCP rules file. I prefer to keep them with the rest of the qmail files in a directory called /var/qmail/rules, so create a file called /var/qmail/rules/smtprules.txt with contents like this (the # lines are comments):

# allow relay from this host
127.:allow,RELAYCLIENT=""
     
# allow relay from other hosts on this network
172.16.42.:allow,RELAYCLIENT=""
172.16.15.1-127.:allow,RELAYCLIENT=""
     
# otherwise, allow connections but no relay
:allow

The first line says to accept connections from any address starting with 127, that is, the loopback pseudo-network used for connections from the qmail host to itself, and to create an empty RELAYCLIENT variable. This permits any SMTP connection from the host that qmail is running on to relay. The second and third lines permit relay from any address in 172.16.42.x, and in the range 172.16.15.1 through 172.16.15.127. Replace these lines with ones listing the IP range(s) of your own network. You can have as many lines as you want; more lines don’t make the lookup any slower once the file is compiled into a CDB. The last line is the default, and permits connections from anywhere else, but without setting any variables.

Now you must compile the rules into a CDB file, using tcprules. Although it’s not hard to run tcprules by hand, it’s a pain to do it every time you update your smtprules file (which you will, to block IP addresses that send a lot of spam). It’s easy to automate the process using a Makefile to rebuild the CDB, as in Example 4-4.

Example 4-4. Makefile to rebuild the rules file for the SMTP listener

default: smtprules.cdb
     
smtprules.cdb: smtprules.txt
    cat $> | /usr/local/bin/tcprules $@ $@.$$$$

(The odd looking $@.$$$$ is the temporary name of the new CDB, the real name with the PID of the make process added to ensure uniqueness.) Finally, tell tcpserver to look at the rules file. Edit /var/qmail/supervise/qmail-smtpd/run and add an --x flag to the tcpserver line, as in Example 4-5.

Example 4-5. Running the SMTP listener

 1. #!/bin/sh
 2. limit datasize 2m
 3. exec  \
 4.    tcpserver -u000 -g000 -v -p -R    \
 5.    -x/var/qmail/rules/smtprules.cdb 0 25  \
 6.    /var/qmail/bin/qmail-smtpd 2>&1

You’re all set. Finally, use svc -t /service/supervise/qmail-smtpd to restart tcpserver with the new arguments.

To test this, send mail from a computer on the local network to an address somewhere else (such as a Hotmail account), and check the logs to verify that it’s accepted and mailed back out.

Procmail and Qmail

If you do any sorting or filtering of incoming mail, you should install the popular procmail mail filtering package. Although procmail’s filter definition language is terse to the point of obscurity, it’s very powerful and easy to use once you get the hang of it. In the past, procmail’s default mailbox location was in /var/mail, and it didn’t support Maildirs. Recent versions of procmail work well with qmail. Version 3.14 added support for Maildirs, and it’s now easy to compile procmail to put the default mailbox in qmail’s preferred place.

The source for procmail is available at http://www.procmail.org. Download it to a local work directory and unpack it. To make its default delivery be to Mailbox, edit the file src/authenticate.c. Around line 47 find the definition of MAILSPOOLHOME, remove the comment characters at the start of the line, and change the file name to Mailbox:

#define MAILSPOOLHOME "/Mailbox"

Or to make the default delivery to a user’s Maildir, type:

#define MAILSPOOLHOME "/Maildir/"

(Note the slash after the directory name, which tells procmail that it’s a Maildir rather than an mbox.)

Then make and install procmail as described in its INSTALL file. The procmail installation recommends that you install procmail as set-uid to root. When working with qmail, it does not need set-uid to work correctly, and I recommend that you don’t do this. When used as the mail delivery agent for sendmail, procmail needs set-uid to run as the id of the delivered-to user. Qmail switches to the correct user ID before running procmail, as it does for any delivery agent, so procmail doesn’t need to do so. Installing as set-uid won’t cause any immediate problems, but it will pose a possible security problem should there turn out to be lurking bugs in procmail.

To use procmail as your default delivery agent, use this in your qmail run file:

exec env - PATH="/var/qmail/bin:$PATH" \
        qmail-start '|preline procmail'

(The preline command is a qmail component that inserts a From line that procmail needs at the front of the message.) Alternatively, to make procmail the delivery agent for an individual user, put the procmail command into the user’s .qmail file:

|preline procmail

Sendmail systems often pass the address extension as an argument to procmail so it can be used as $1 in scripts. That’s easy enough to do in .qmail-default:

|preline procmail -a "$EXT"

Procmail makes most environment variables available in its rule files anyway, so if you’re not converting from sendmail, just use $EXT in your scripts.

It’s frequently advantageous to use different procmail filter definitions for different qmail subaddresses. For example, if you are user fred and use the address fred-lists for your mailing list mail, .qmail-lists could contain this:

|preline procmail procmaillists

to use procmaillists to sort list mail.

Creating Addresses and Mailboxes

With the setup so far, every user in /etc/passwd automatically has a mailbox with the same name as the login name.^[1] If you’re using mbox mailboxes, each mailbox is created the first time a message is delivered to it. If you’re using Maildirs, you must create the Maildirs yourself using maildirmake. If all of your home directories are stored in /usr/home or /home, it’s easy enough to give everyone a Maildir. Run a script like this as root to create them:

cd /home
for u in *
do
    maildirmake $u/Maildir
    chown -R $u/Maildir
done

The chown is important so that each user owns his own Maildir.

If you have more than two or three mailboxes to create, use the convert-and-create script from http://www.qmail.org/. It creates Maildirs for every user with a mailbox, and copies the mail from /var/mail mboxes into the new Maildirs.

Once you’ve created Maildirs for all of your existing users, creating them for new users is considerably easier. Just add a line or two to your system’s adduser script to create the Maildir as it creates the rest of the new user’s files. On Linux systems, use maildirmake to create /etc/skel/Maildir, a prototype that gives every subsequent new user a Maildir.

Reading Your Mail

If you use mbox mailboxes, the only additional change you may have to make is to tell your mail program (and your shell if it’s one that reports new mail) that the mailbox is in ~/Mailbox rather than in /var/mail. Most mail programs check the shell variable $MAIL. For testing, change the MAIL variable at your shell prompt:

% setenv MAIL ~/Mailbox    (in csh)
$ export MAIL=~/Mailbox    (ksh and bash)
$ export MAIL=$HOME/Mailbox (in sh)

Once you’re committed to qmail and your mail is in /var/mail, you want to copy everyone’s mailbox to their home directory, using the convert-and-create script mentioned previously. Then, find the place in /etc/profile or /etc/cshrc that sets MAIL and change it to refer to the new mailbox location.

If you use Maildirs, your options are simpler. The only mail program with built-in Maildir support is mutt. On qmail.org there are some patches for pine to handle Maildirs, and a version of movemail for GNU Emacs users. If you use something else, you can use the scripts distributed with qmail such as elq or pinq that copy mail from a Maildir into an mbox and then run elm or pine. Honestly, if a user normally uses a mail program that expects mbox mailboxes, it’s easier to tell qmail to use mboxes than to tell the program to use Maildirs.

An alternative that makes Maildirs available to most mail clients is to use an IMAP server such as Courier that handles Maildirs (see Chapter 13). The IMAP server can retrieve mail from the Maildir and from any number of Maildir-format subfolders. You can set up pine or Mozilla to use IMAP to deal with the Maildir folders, and use its built-in mbox support to handle mboxes directly as files. This has the added advantage that you can check your mail using any IMAP client from other computers if you’re away from your usual computer.

Configuring Qmail’s Control Files

Qmail is controlled by a large set of control files stored in /var/qmail/control. Unlike some other MTAs that group everything into one huge file that they have to parse to figure out what’s what, qmail puts each different kind of information into a separate file, so that each file needs little or no parsing. All files are lines of plain text (although a few files are compiled into CDB databases before use). Some, noted below, allow comment lines with a # at the beginning of the line. In files where each line contains multiple fields, the fields are separated by colons.

Most of the control files are optional, and qmail uses a reasonable default in most cases if a file isn’t present. The only files that are absolutely essential are me, which contains the hostname of the local host, and rcpthosts, which lists the names of the domains for which this host accepts mail.

Here’s a list of all the control files in alphabetical order, noting which component uses each one. Many of the optional patches introduce new control files, which are discussed during the description of the patch.

Checking Your Configuration with qmail-showctl

Because qmail has a long and somewhat daunting set of configuration files, the package includes a program to tell you what your current configuration is. Run /var/qmail/bin/qmail-showctl and pipe its output through a pager like more to see a detailed narrative of the contents of all of the configuration files. For missing files it tells you what defaulted values are in use.

If you receive the message I have no idea what this file does, it means that the file is not one that qmail-showctl recognizes. You can put any extra files you want in the control directory and qmail doesn’t care. Occasionally, it’s useful to leave notes to yourself or to save an old version of a file you’re changing. There is an exception: qmail-showctl looks at all the files and makes that complaint about the ones not in its list. If you apply patches that use new configuration files, most of the patches don’t bother to update qmail-showctl, so it’ll typically complain about those files too, which is equally harmless.

badmailfrom (qmail-smtpd)

Envelope addresses not allowed to send mail. If the envelope From address on an incoming message matches an entry in badmailfrom, the SMTP daemon will reject every recipient address. Entries may be either email addresses, or @dom.ain to reject every address in a domain. This is a primitive form of spam filtering. These days, it’s mostly useful to block quickly a mailbomb or flood of rejection messages.

bouncefrom (qmail-send)

Default: MAILER-DAEMON. The mailbox of the return address to put in bounce messages. I’ve never found any reason to change it.

bouncehost (qmail-send)

Default: me. The domain of the return address to put in bounce messages. I’ve never found any need to change it, although it’s possible that if your mail host is mail.example.com, you might want to have the bounces come from example.com.

concurrencylocal (qmail-send)

Default: 10. The maximum number of simultaneous local deliveries. Unless you have very slow delivery programs, the default is adequate for all but very large systems. Keep in mind that if you have slow delivery programs, it is quite possible to have all 10 or however many running as the same user, so be sure that the per-user process limit is high enough to permit them all to run.

concurrencyremote (qmail-send)

Default: 20. The maximum number of simultaneous remote deliveries. The default is adequate for small systems, but too low for large systems or systems that host mailing lists. You should adjust it so that qmail uses as much of your outgoing bandwidth as you want it to. In the distributed version of qmail, you can increase this up to 120, which is enough for a moderately busy system with mailing lists sharing a T1 with other services. See Chapter 16 for advice on increasing it past 120 on large systems.

defaultdomain (qmail-inject)

Default: the literal string defaultdomain. The domain to add to unqualified host names (names with no dot) on outgoing mail. That is, if someone injects a message with a sender or recipient address of fred@bad and this file contains example.com, the address is rewritten as [e-mail]. You invariably want to set this to the local domain. Note that only mail injected via qmail-inject has its header addresses rewritten. Addresses in mail that arrives via SMTP or is injected directly via qmail-queue aren’t modified.

defaulthost (qmail-inject)

Default: me. Similar to defaultdomain; the domain to add to addresses in outgoing mail that have no domain at all. If defaulthost doesn’t contain a dot, defaultdomain is added, too. Set this to the name of the local domain.

databytes (qmail-smtpd)

Default: 0, meaning no limit. The maximum message size to accept via SMTP. I usually set it to about 1/10 the size of the typical amount of free space on the partition where the qmail queue resides, to keep a single bloated incoming message from causing qmail to run out of disk space. The DATABYTES environment variable overrides the control file, so if there are certain systems from which you want to accept huge messages, you can put entries into the SMTP rules file to permit that. For example:

# allow 50 megabyte powerpoints from the boss
209.58.173.10:allow,DATABYTES="50000000"
     
# allow 20 meg outgoing mail from nearby hosts
172.16.15.1-127.:allow,RELAYCLIENT="",DATABYTES="20000000"

doublebouncehost (qmail-send)

Default: me. The domain to which to send double-bounce messages. There’s rarely any reason to change it.

doublebounceto (qmail-send)

Default: postmaster. The mailbox to which to send double-bounce messages, that is, they go to doublebounceto@doublebouncehost. You can also send these messages to a special mailbox that you examine rarely, or because these days there are vast numbers of double bounces caused by spam with fake return addresses, you can set it to nobody or some other address that just throws them away.

envnoathost (qmail-send)

Default: me. The domain to add to envelope recipient addresses with no domain. This value is used by qmail-send, while defaultdomain is used by qmail-inject, so in practice this value is used to fix up mail received by SMTP. The default value is fine, unless you receive a lot of spam with bare addresses, in which case you can set it to something like invalid to make incoming mail with no domain bounce.

helohost (qmail-remote)

Default: me. The domain to use in the HELO command of outgoing SMTP sessions. The default is fine.

idhost (qmail-inject)

Default: me. The domain to use when creating Message-ID: lines in outgoing mail. The default is fine. If you want to do something special with message ID’s, you can provide them yourself on mail you send, in which case qmail won’t alter them.

localiphost (qmail-smtpd)

Default: me. When qmail-smtpd sees incoming mail to an address using a dotted quad rather than a domain name, like fred@[10.11.12.99], and the IP address is one on this host, it substitutes in localiphost. The default is usually fine unless you want to change it to the local mail domain.

locals (qmail-send)

Default: me. Domains to treat as local. Any addresses in domains listed in this file are considered to be local and are routed using the local delivery rules. All local domains are equivalent; if foo.org and bar.com both appear in locals, the addresses [e-mail] and [e-mail] are handled identically.^[2] This file always includes the name of the local host (the same as what’s in me) and generally includes the local domain as well and any other domains that may have been used for the same set of addresses. For example, the locals file on my mail server tom.iecc.com also includes iecc.com (the current local domain), iecc.cambridge.ma.us (its old name), and iecc.us (a trendy vanity equivalent.)

Note that local domains are not the same as virtual domains, nor are they the same as the SMTP recipient domains listed in rcpthosts.

me (qmail-send)

Default: none; this file is required. The name of the current host. This should be the same as what the hostname command returns.

morercpthosts (qmail-smtpd and qmail-newmrh)

Default: none. More domains for which this host accepts SMTP mail. The contents of this file are compiled into morercpthosts.cdb by qmail-newmrh. The SMTP daemon consults the cdb file after it checks rcpthosts. If a host accepts mail for more than about 50 domains, Dan suggests that you put the 50 busiest into rcpthosts and the rest into morercpthosts.

percenthack (qmail-send)

Default: none. The “percent hack” is a primitive form of source routing introduced by sendmail in the early 1980s. If you send mail to [e-mail], the mail would be sent to out.side, where the address would be rewritten to [e-mail] and sent along to in.side. In the past 20 years, most of the connectivity problems that require source routing have been solved, and for the ones that remain there are better tools such as smtproutes (described later), so the percent hack is obsolete. If for some reason you absolutely need it (you have an ancient mission-critical program for which all the source code has been lost that sends mail using the percent hack, perhaps) any addresses in domains listed in percenthack are scanned for percent signs and rewritten. In the previous example, out.side would have to be listed there.

If a domain listed in percenthack is also listed in rcpthosts, your system is an open relay, because spammers can send mail anywhere through your system by putting the actual target address in percent form inside an address in the listed domain. Yes, spammers actually do so. The solution is simple: don’t do it.

plusdomain (qmail-inject)

Default: me. If the domain part of an address in an injected message ends with a plus sign, the contents of plusdomain are appended to the end. In environments with many subdomains of a single main domain, say east.bigcorp.com, west.bigcorp.com, and south.bigcorp.com, this lets people abbreviate addresses to fred@south+. No longer widely used.

qmqpservers (qmail-qmqpc)

Default: none. A list of servers to which messages can be queued using QMQP. See Chapter 17.

queuelifetime (qmail-send)

Default: 604800 seconds (a week). How long to keep trying to deliver a message. More precisely, if qmail tries to send a message and the attempt fails with a temporary error, the error is treated as permanent if the message is older than queuelifetime, in which case the message bounces.

The default time of a week is reasonable, but you might want to decrease it to three or four days if you’d rather know sooner that a message isn’t getting through, at the risk that the destination host might have come back to life if you’d waited longer.

rcpthosts (qmail-smtpd)

Default: every domain. The list of domains for which this host accepts SMTP mail. It is extremely important that this file exist. If it doesn’t, qmail will accept mail destined for anywhere and will be an “open relay,” and a magnet for spammers.

If you receive mail for more than 50 domains, see morercpthosts.

smtpgreeting (qmail-smtpd)

Default: me. When another hosts connects via SMTP to send you mail, the greeting string to send. The default is fine.

smtproutes (qmail-remote)

Default: none. Explicit routes to use to deliver outgoing mail, overriding MX data. Each line is of one of these forms:

                        domain:relay
                        domain:relay:port

domain is the domain in the destination email address, relay is the name of the host to which to deliver the mail, and the optional port is the port number if not the standard port 25.

The domain can use wildcards; if it starts with a dot, it matches any target domain that ends with that domain. If the domain is empty, it matches all addresses, providing “smarthost” routing to send all mail to a single smarthost for delivery. If relay is empty, qmail uses the standard MX lookup, letting you override a broader wildcard or smarthost route.

Most systems can get by without smtproutes, but there are three situations where it can come in handy. The first is a smarthost, mentioned previously, if your computer is on a dialup, DSL, or cable modem, and the smarthost is your ISP’s outgoing mail server. The second is to temporarily patch around broken MX records or mail relays. The third is to route mail for private domains within your network.

timeoutconnect (qmail-remote)

Default: 60 seconds. How long to wait for a remote server to accept the initial connection to send mail. Unless you need to exchange mail with extremely slow and overloaded remote servers, don’t change it.

timeoutremote (qmail-remote)

Default: 1200 seconds. Once a remote server is connected, how long to wait for each response before giving up. The default of 20 minutes is extremely conservative, and can lead to all of your remote sending slots being tied up while waiting for somnolent remote hosts to time out. Unless you communicate with extraordinarily slow and overloaded remote servers, you can drop it to a minute.

timeoutsmtpd (qmail-smtpd)

Default: 1200 seconds. How long qmail-smtpd waits for each response from a remote client before timing out and giving up. As with timeoutremote, you can decrease this to a minute unless you have some really slow remote clients.

virtualdomains (qmail-send)

Default: none. The list of virtual users and domains for which this system receives mail. If you don’t handle any virtual domains, you don’t need this file.

The virtual domain scheme works by taking the mailbox in the virtual domain, prepending a string and a hyphen to create a local address, and redelivering the mail to the local address. The virtual domain file lists the prepend string to use for each virtual user and domain. (See Chapter 12.) Each line is of one of these forms:

[e-mail]:string (1)
dom.ain:string (2)
.domain:string (3)
domain: (4)
:string (5)

Form (1) controls mail to a specific address. Forms (2) and (3) control mail to any address in a domain or in subdomains of a domain, respectively. Form (4), with an empty prepend, is used to create an exception to a domain that would otherwise be handled by a line of form (3) or (5) and means to handle the domain normally, not as a virtual domain. Form (5) is a catchall and controls all domains not listed in locals or elsewhere in virtualdomains.

If a domain erroneously appears both in locals and virtualdomains, the listing in locals takes precedence. Don’t do that.

Using ~alias

Although qmail automatically handles deliveries to most users with entries in the Unix password file (or qmail’s adjusted version of it; see Chapter 15), any useful mail setup also needs to deliver mail to addresses unrelated to entries in the password file. Qmail handles this in a simple, elegant way with the alias pseudo-user. As part of the installation process, create a user called alias and set its home directory to /var/qmail/alias. When qmail is running, if mail arrives for a local mailbox that isn’t in the normal list of users, qmail prepends alias- to the address and retries the delivery. This makes any address not otherwise handled in effect a subaddress of alias, so you can handle addresses by putting .qmail files into ~alias. For example, if you have a user robert and want mail addressed to bob to be forwarded to him, create ~alias/.qmail-bob and in it put &robert. Since qmail handles deliveries using the .qmail files in ~alias the same way that it handles any other deliveries, you have all of the same options delivering to nonuser addresses that you do to user addresses.

Because qmail doesn’t deliver to root and other users that have a 0 user ID or that don’t own their home directories, you should arrange to send root’s mail to the system manager by creating ~alias/.qmail-root. Also create .qmail-postmaster, .qmail-abuse, .qmail-webmaster, and any other role addresses that you want to support.

The final default delivery is, not surprisingly, found in ~alias/.qmail-default. If that file doesn’t exist, unknown addresses bounce, often just what you want. The most common thing to put in that file is a line to run the fastforward program (see the next section) to take delivery instructions from a file of addresses, roughly as sendmail does. You can also implement other default delivery rules. For example, if you want to make mail to subaddresses of ~alias users default to the base address, so mail to fred-foop is delivered to fred if it’s not otherwise handled, put a line like this in your default delivery file. (It appears wrapped here, but it has to be on one long line in the file.)

| case "$DEFAULT" in *-*) forward "${DEFAULT%%-*}" ;; *) bouncesaying 
"Sorry, no mailbox here by that name. (#5.1.1)" ;; esac

This says that if an address contains a hyphen, strip off the hyphen and everything after it and redeliver it. Otherwise bounce the message. The bouncesaying command lets you provide your own failure message, but a simple exit 100 would do the trick as well, telling qmail to bounce.

fastforward and /etc/aliases

Sendmail and other MTAs use configuration files such as /etc/aliases that contain lists of mailboxes and forwarding instructions. While qmail doesn’t have a built-in feature to do that, the add-on fastforward package (available at http://cr.yp.to/fastforward.html provides both a mostly compatible way to handle existing /etc/alias files) and a more general scheme to handle files with forwarding instructions and mailing lists.

Installing fastforward

You can download and install the fastforward package the same way you install Dan’s other programs, as described in Chapter 3. This section describes fastforward Version 0.51.

Using fastforward

The central program in the fastforward package is fastforward itself. It’s designed to be run from a .qmail file. When run, it gets the recipient address from $RECIPIENT or optionally $DEFAULT@$HOST, looks up the address in a delivery database, and if it finds the address, follows the delivery instructions for the address.

fastforward takes its instructions from a CDB-format file. There are two ways to create the file: using newaliases to create /etc/aliases.cdb from /etc/aliases, which is in sendmail format, or using setforward to create a CDB from an arbitrary file, which is in a different, more flexible format. All of fastforward’s CDB files have the same format, regardless of which program created them.

The CDB file can refer to mailing list files of addresses; the difference is that the CDB file contains addresses and delivery instructions, while a mailing list file just contains a list of addresses and other mailing list files, for use within a delivery instruction. Mailing list files can be created by newinclude, which reads input containing a list of addresses in a format similar to the one sendmail uses for :include: files, or by setmaillist, which reads input in a more flexible format. Mailing list files created by either program have the same format, so you can use the input format that is more convenient. Compiled mailing list files have the extension .bin. In this section, I describe /etc/alias compatibility and leave the rest for the sections on virtual domains (Chapter 12) and mailing lists (Chapter 14).

The most common way to use fastforward is to call it from ~alias/.qmail-default so it can take a crack at any addresses not handled otherwise:

| fastforward /etc/aliases.cdb

Or you can also combine it with other default rules. For example, to use fastforward and then redeliver mail to subaddresses to the base address of the subaddress:

| fastforward -p /etc/aliases.cdb
| case "$DEFAULT" in *-*) forward "${DEFAULT%%-*}" ;; *) bouncesaying "Sorry, no 
mailbox here by that
name. (#5.1.1)" ;; esac

The -p flag says to “pass through,” that is, exit 99 if an address is found or exit 0 if not, so qmail goes on to the next line in the .qmail file if fastforward didn’t deliver it. (In the absence of -p, fastforward exits 0 if it forwards the message and 100 otherwise to bounce the mail.)

Alias File Format

The format of /etc/alias is a sequence of forwarding instructions. The most common instruction forwards an address to one or more other addresses:

bob: robert
ted: edward, edwin, eduardo
[e-mail]: frederick
[e-mail]: nobody
@good.example.com: mary

Mail to ted is forwarded to edward, edwin, and eduardo. This form is useful for role accounts that are handled by several people or tiny mailing lists that change rarely. If there are multiple names in localhosts for this host, distinguish addresses by putting the domain of the address, and forward all addresses in a domain by using @domain. (This feature is more often used to handle addresses in virtual domains; see Chapter 12.) As a concession to sendmail compatibility, addresses can have comments and can be quoted as they are in To: and From: lines. Any line that starts with # is a comment, and any line can be continued by starting continuation lines with whitespace:

bell: |ringthebell
klaxon: "|ringthebell --reallyloud"

Any address that starts with a vertical bar is treated as a command for program delivery. If the command contains whitespace or at-signs, it has to be quoted. fastforward runs the program as whatever user it’s running as, which is alias if it’s called from ~alias/.qmail-default. (To run a program as another user, it has to be called from a .qmail file belonging to that user. See Chapter 15.) The program is run as:

preline sh -c command

so that the message starts with a sendmail-style From line.

cephalopods: :include:/usr/fred/cephalopods
owner-cephalopods: fred

Any address that starts with :include: refers to the contents of a mailing list file. The mailing list file must have been compiled by newinclude or setmaillist, so in the previous example, fastforward looks for /usr/fred/cephalopods.bin, and the delivery is deferred if the file isn’t available. If there is an entry for both listname and owner-listname, any forwarded mail to listname has its envelope sender changed to owner-listname so bounces will go back to the owner of the list.

Note that mailing list files are read by fastforward when they’re needed, not by newaliases. This means that, in the previous example, the addresses on the list belong to user fred, who can update the list file and rerun newinclude as needed. Mailing list files can refer to other mailing list files, but for security reasons (and unlike sendmail), they cannot contain program deliveries. This is not much of a problem in practice. In the previous example, if Fred wanted to, say, fax list messages to someone using a fax program, he could add an address fred-squidfax to the mailing list, then create ~fred/.qmail-squidfax with whatever program deliveries he wants, running as fred, not as alias.

fastforward lives up to its name when doing list deliveries, and it can dispatch messages to huge lists very quickly. Nonetheless, if you have a large list with hundreds or thousands of recipients, it’s better to use a mailing list manager like ezmlm (Chapter 14) to provide automated bounce handling, and a partly or fully automated subscribe and unsubscribe service for list members.

---

^[1] That’s not quite true; for security reasons qmail won’t deliver mail to the root user.

^[2] A very perverse user could test $HOST in a delivery rule in a .qmail file to tell two local domains apart, but I don’t think I’ve ever seen anyone do so.