Programming Internet Email

The Basic Internet Email System

The Internet mail system appears to be rather complex. In reality, it is a system of components, each of which does a simple job. The devil, of course, is in the details.

In one sense, the Internet mail system is a distributed client/server system with server peering. In other words, clients send and receive mail by talking to servers, and the servers talk to one another. A client may send an outgoing message directly to the server (MTA) that will deliver it to the recipient’s mailbox, or to another MTA that will forward it on. This system is intended to be highly scalable by arranging servers in a hierarchy. Each server need know only its immediate peers and the server above it in the hierarchy.

In practice, it’s not that easy. The Internet being a network of cooperating networks, the Internet mail system must be able to correspond with other types of mail systems. Much effort is expended in translating messages coming from and going to these foreign mail systems. The mail system elements that perform these translations are called gateways. For historical reasons, Internet MTAs often perform some gateway-like functions for older mail systems used on the Internet, such as UUCP. sendmail, the popular freeware MTA that handles most of the mail on the Internet core, was originally written to integrate multiple mail systems being used on newly connected networks, providing message address translation functions for simple text messages.

Since the Internet is constantly evolving, the mail system must change as well. The system has evolved to deliver mail to users dialing onto the network part-time and, more recently, has evolved means to deliver mail to those using network computers without local storage and those accessing their mail via a World Wide Web browser.

The following sections break down the separate parts of the Internet mail system and describe each in turn.

What we’ll call the “basic” mail system is the peered distributed client/server system built around the Simple Mail Transfer Protocol (SMTP) and its extensions. This has become the most commonly used Internet mail protocol, although others have existed before and since SMTP became popular.

Figure 1-1 shows the basic system. This is SMTP as it existed when most users retrieved their mail directly from a mail server, either from a terminal connected to a mainframe, which received mail, or from a Unix workstation, which was configured to receive mail itself. Most modern users do not access their mailbox directly, as shown in the figure, since their desktop machines rarely receive mail directly and in real time. Nevertheless, the basic system still provides the backbone of Internet mail.

As we shall see, modern methods simply extend the basic model to provide greater accessibility options.

Figure 1-1. Simplified Internet mail system

A mail user agent that wishes to send a message uses the SMTP protocol (or its more recently enhanced version, Extended Simple Mail Transfer Protocol, or ESMTP) to speak with a mail transfer agent. As shown, an MUA may send the message directly to the MTA that handles mail for the recipient, or to a local MTA that will forward the message onward.

The latter case has become much more popular since the vast majority of Internet users currently access the network only part-time, via the telephone system. It is important to understand why this is so. The Internet mail system is asynchronous in nature, meaning that an immediate delivery of a message is not guaranteed, although it often occurs. If a sending MUA cannot immediately reach the recipient’s MTA due to downtime, network congestion or other reasons, an error will occur. The MUA may not be connected to the network on a permanent basis, so rescheduling the message to be sent later may be ineffective. However, if the message can be sent to an MTA that is permanently connected and always reachable (such as a mail server on the user’s network or a mail server at an Internet service provider), the message may be queued for later delivery, and multiple attempts can be made to deliver it before failure is admitted. When a message cannot be sent, the sending MTA can deliver a failure notification to the user’s email address (which often ends up in the user’s mailbox on the same machine, if the sending MTA is also the user’s receiving mail server).

The machine that receives a message is an MTA, called here the receiving MTA. Its receipt is not a notification that the message has been read by the user, but simply that it has been delivered to the user’s mailbox. There is no native facility within SMTP to notify the sender of message receipt, only of delivery failure.

MTAs do not actually deliver messages themselves. They prepare messages to take the next step on their journey, such as onto another type of network. The receiving MTA calls upon an external program on the same physical machine ^[2] to deliver the message. This program is known as a mail delivery agent (MDA).

Mail delivery agents are used to write messages into a user’s mailbox, to forward messages to other MTAs, and to forward messages to other mail systems. Typically, separate programs are used to perform each of these functions. An MTA will normally have an MDA to do local delivery (directly writing into a user’s mailbox) and another to perform TCP-based socket delivery to another MTA. Many MTAs also have MDAs to handle UUCP or other local mail systems, such as those to deliver mail to DECnet or Macintosh networks, or even to a facsimile system.

Local delivery appends a message onto a file that constitutes the mailbox. Note that this mailbox is on the MDA’s (and MTA’s) filesystem and may not constitute final delivery to the user. In fact, unless the user is using an MUA that is local to the server, delivery to this mailbox is likely to be just another step along the path.

If a user cannot access his mailbox directly, the user’s MUA may use a mail retrieval agent (MRA) to access the mailbox remotely. MRAs are described in detail in the section “Retrieval of Mail by MRAs,” later in this chapter.

Forwarding of a message to another MTA takes place if the message is destined for a user that does not have an account on the local system. In this case, the recipient’s address is parsed to determine the destination domain, and the Internet Domain Name System (DNS) is used to find the destination MTA. The MDA handling the appropriate delivery mechanism will then be called by the MTA to deliver the message to the destination MTA.

Note that commercial products often blur the logical distinction between mail system elements. Netscape Messaging Server, for example, combines a mail transfer agent, several mail delivery agents, and the server components of mail retrieval agents.

---

^[2] Normally—it would be more accurate to say that it calls an external program with access to the same filesystem.