1.6 Basic Roles of sendmail

The sendmail program plays a variety of roles, all critical to the proper flow of electronic mail. It listens to the network for incoming mail, transports mail messages to other machines, and hands local mail to a local program for local delivery. It can append mail to files and pipe mail through other programs. It can queue mail for later delivery and understand the aliasing of one recipient name to another.

1.6.1 Role in the Filesystem

The sendmail program's role (position) in the local filesystem hierarchy can be viewed as an inverted tree (see Figure 1-3).

Figure 1-3. The sendmail.cf file leads to everything else

When sendmail is run, it first reads the /etc/mail/sendmail.cf configuration file. Among the many items contained in that file are the locations of all the other files and directories that sendmail needs.

Files and directories listed in sendmail.cf are usually specified as full pathnames for security (such as /var/spool/mqueue rather than mqueue). As the first step in our tour of those files, run the following command to gather a list of them:^[6]

^[6] If you are not currently running sendmail V8.7 or above, you will have to grep(1) for "/[^0-9].*/" instead. If you're not running sendmail at all, you won't be able to do this, so for now just read along instead.

% grep =/ /etc/mail/sendmail.cf

The output produced by the grep(1) command might look something like the following:^[7]

^[7] Lines that begin with F or K might also appear. If so, ignore them for now.

O AliasFile=/etc/mail/aliases
#O ErrorHeader=/etc/mail/error-header
O HelpFile=/etc/mail/helpfile
O QueueDirectory=/var/spool/mqueues/q.*
O StatusFile=/etc/mail/statistics
#O UserDatabaseSpec=/etc/mail/userdb
#O ServiceSwitchFile=/etc/mail/service.switch
#O HostsFile=/etc/hosts
#O SafeFileEnvironment=/arch
#O DeadLetterDrop=/var/tmp/dead.letter
O ControlSocketName=/var/spool/mqueues/.control
#O PidFile=/var/run/sendmail.pid
#O DefaultAuthInfo=/etc/mail/default-auth-info
Mlocal,       P=/usr/lib/mail.local, F=lsDFMAw5:/|@qPSXfmnz9, S=EnvFromSMTP/HdrFromL,
Mprog,        P=/bin/sh, F=lsDFMoqeu9, S=EnvFromL/HdrFromL, R=EnvToL/HdrToL, D=$z:/,

Notice that some lines begin with an O character, some with an M, and others with a #. The O marks a line as a configuration option. The word following the O is the name of the option. The options in the preceding output show the location of the files that sendmail uses. AliasFile, for example, defines the location of the aliases(5) database. The lines that begin with M define delivery agents. The lines that begin with a # are comments.

First we will examine the files in the O option lines. Then we will discuss local delivery and the files in the M delivery agent lines.

1.6.2 The Role in the Aliases File

Aliasing is the process of converting one recipient name into another. One use is to convert a generic name (such as root) into a real username. Another is to convert one name into a list of many names (for mailing lists).

Take a few moments to examine your aliases file. Its location is determined by the AliasFile option in your sendmail.cf file. For example:

O AliasFile=/etc/mail/aliases

Compare what you find in your aliases file to the brief example of an aliases file listed here:

# Mandatory aliases.
postmaster:     bob
MAILER-DAEMON:  postmaster
abuse:          postmaster

# The five forms of aliases
John_Adams:     adamj
xpres:          ford,carter,reagan,bush,clinton
oldlist:        :include:/usr/local/oldguys
nobody:         /dev/null
ftphelp:        |/usr/local/bin/sendhelp

Your aliases file is probably far more complex, but even so, note that the example shows all the possible forms of aliases.

Lines that begin with # are comments. Empty lines are ignored. As the first comment indicates, three aliases are mandatory in every aliases file. They are the simplest form of alias: a name and what to change that name into. The name on the left of the : is changed into the name on the right. Names are not case-sensitive. For example, POSTMASTER, Postmaster, and postmaster are all the same.^[8]

^[8] According to RFC2822, all usernames are case-sensitive except postmaster. And RFC2142 defines additional names, such as abuse, that are not case-sensitive. But sendmail, when processing its aliases file, normally views all other names as case-insensitive too, if F=u (F=u) is set on the local delivery agent.

For every envelope that lists a local user as a recipient, sendmail looks up that recipient's name in the aliases file. (A local user is any address that would normally be delivered on the local machine. That is, postmaster is local, whereas postmaster@remote might not be.) When sendmail is processing the envelope, and when it matches the recipient to one of the names on the left of the aliases file, it replaces that recipient name with the text to the right of the : character. For example, the envelope recipient postmaster becomes the new envelope recipient bob.

After a name is substituted, the new name is then looked up, and the process is repeated until no more matches are found. The name MAILER-DAEMON is first changed to postmaster. Then postmaster is looked up again and changed to bob. Because there is no entry for bob in the aliases file, the mail message is delivered into bob's mailbox.

Every aliases file must have an alias for postmaster that will expand to the name of a real user.^[9] Mail about mail problems is always sent to postmaster both by mail-related programs and by users who are having trouble sending mail.

^[9] The name postmaster is required by RFC2822, so resist the temptation to redefine it as postperson or sysop.

When mail is bounced (returned because it could not be delivered), it is always sent from MAILER-DAEMON. That alias is needed because users might reply to bounced mail. Without it, replies to bounced mail would themselves bounce.

The five types of lines in an aliases file are as follows:

John_Adams:    adamj
xpres:         ford,carter,reagan,bush,clinton
oldlist:       :include:/usr/local/oldguys
nobody:        /dev/null
ftphelp:       |/usr/local/bin/sendhelp

You have already seen the first line (it was the form used to convert postmaster to bob). In the previous example, mail sent to John_Adams is delivered to the user whose login name is adamj.

The xpres: line shows how one name can be expanded into a list of many names. Each new name becomes a new name for further alias processing. If a name can't be further expanded, a copy of the mail message is delivered to it.

The oldlist: line shows how a mailing list can be read from a file. The expression :include: tells sendmail to read a specific file and to use the names in that file as the list of recipients.

The nobody: line shows how a name can be aliased to a file. The mail message is appended to the file. The /dev/null file listed here is a special one. That file is an empty hole into which the mail message simply vanishes.

The ftphelp: line shows how a name can be replaced by the name of a program. The | character causes sendmail to pipe the mail message through the program whose full pathname follows (in this case, we specified the full pathname as /usr/local/bin/sendhelp).

The aliases file can become very complex. It can be used to solve many special mail problems. The aliases file is covered in greater detail in Chapter 12.

1.6.3 Role in Queue Management

A mail message can be temporarily undeliverable for a wide variety of reasons, such as when a remote machine is down or has a temporary disk problem. To ensure that such a message is eventually delivered, sendmail stores it in a queue directory until the message can be delivered successfully.

The QueueDirectory option in your configuration file tells sendmail where to find its queue directory:

O QueueDirectory=/var/spool/mqueue

The location of that directory must be a full pathname. Its exact location varies from vendor to vendor, but you can always find it by looking for the QueueDirectory option in your configuration file.

Beginning with V8.10, sendmail allows multiple queue directories to be used. Such a declaration can look like this:

O QueueDirectory=/var/spool/queues/q.*

Here, sendmail will use the subdirectories in /var/spool/queues that begin with the name q. for storage of messages. Such directories might be called, for example, q.00 and q.01.

If you have permission, take a look at a sendmail queue directory. It might be empty if no mail is waiting to be sent. If it is not empty, it will contain files such as these:

dfg17NVhbh002596 dfg1BHotav010793 qfg17NVhbh002596 qfg1BHotav010793

When a mail message is queued, it is split into two parts, each part being saved in a separate file. The header information is saved in a file whose name begins with the characters qf. The body of the mail message is saved in a file whose name begins with the characters df.

The previous example shows two queued mail messages. One is identified by the unique string g17NVhbh002596 and the other by g1BHotav010793.

The internals of the queue files and the processing of those files are covered in Chapter 11.

1.6.4 Role in Local Delivery

Another role of the sendmail program is to deliver mail messages to local users. A local user is one who has a mailbox on the local filesystem. Delivering local mail is done by appending a message to the user's mailbox, by feeding the mail message to a program, or by appending the message to a file other than the user's mailbox.

In general, sendmail does not put mail messages directly into files. You saw the exception in the aliases file, in which you could specifically tell sendmail to append mail to a file. This is the exception, not the rule. Usually, sendmail calls other programs to perform delivery. Those other programs are called delivery agents.^[10]

^[10] Although for historical reasons, the sendmail developers still continue to use the term "mailers."

In your sendmail.cf file you found two lines that defined local delivery agents, the ones that sendmail uses to deliver mail to the local filesystem:

Mlocal,  P=/usr/lib/mail.local, F=lsDFMAw5:/|@qPSXfmnz9, S=EnvFromSMTP/HdrFromL,
Mprog,   P=/bin/sh, F=lsDFMoqeu9, S=EnvFromL/HdrFromL, R=EnvToL/HdrToL, D=$z:/,

The /usr/lib/mail.local program is used to append mail to the user's mailbox. The /bin/sh program is used to run other programs that handle delivery.

1.6.5 Delivery to a Mailbox

The configuration file line that begins with Mlocal defines how mail is appended to a user's mailbox file. That program is usually /usr/lib/mail.local (or with older systems, /bin/mail) but can easily be a program such as deliver(1) or procmail(1).

Under Unix a user's mailbox is a single file that contains a series of mail messages. The usual Unix convention (but not the only possibility) is that each message in a mailbox begins with a line that starts with the five characters "From " (the fifth is a blank space) and ends with a blank line.

The sendmail program neither knows nor cares what a user's mailbox looks like. All it cares about is the name of the program that it must run to add mail messages to that mailbox. In the example, that program is /usr/lib/mail.local. The M configuration lines that define delivery agents are covered in detail in Chapter 20.

1.6.6 Delivery Through a Program

Mail addresses that begin with a | character are the names of programs to run. You saw one such address in the example aliases file:

ftphelp:       |/usr/local/bin/sendhelp

Here, mail sent to the address ftphelp is transformed via an alias into the new address |/usr/local/bin/sendhelp. The | character at the start of this new address tells sendmail that this is a program to run rather than a file to append to. The intention here is that the program will receive the mail and do something useful with it.

The sendmail program doesn't run mail delivery programs directly. Instead, it runs a shell and tells that shell to run the program. The name of the shell is listed in the configuration file in a line^[11] that begins with Mprog:

^[11] Actually, delivery agent definitions often span multiple lines.

Mprog,   P=/bin/sh, F=lsDFMoqeu9, S=EnvFromL/HdrFromL, R=EnvToL/HdrToL, D=$z:/,

In this example the shell is the /bin/sh(1). Other programs can appear in this line, such as /bin/ksh(1), the Korn Shell, or smrsh(1), the sendmail restricted shell that is supplied with the source distribution.

1.6.7 Role in Network Transport

Another role of sendmail is that of transporting mail to other machines. A message is transported when sendmail determines that the recipient is not local. The following lines from a typical configuration file define delivery agents for transporting mail to other machines:

Msmtp,   P=[IPC], F=mDFMuX, S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP/HdrFromSMTP,
Muucp,   P=/usr/bin/uux, F=DFMhuUd, S=FromU, R=EnvToU/HdrToU, M=100000,

The actual lines in your file might differ. The name smtp in the preceding example might appear in your file as ether or ddn or something else. The name uucp might appear as suucp or uucp-dom. There might be more such lines than we've shown here. The important point for now is that some delivery agents deal with local delivery, while others deal with delivery over a network.

1.6.8 TCP/IP

The sendmail program has the internal ability to transport mail over only one kind of network, one that uses TCP/IP; the following line instructs sendmail to do this:

Msmtp,   P=[IPC], F=mDFMuX, S=EnvFromSMTP/HdrFromSMTP, R=EnvToSMTP/HdrFromSMTP,

The [IPC] might appear as [TCP], but note that, beginning with V8.10 sendmail, the expression [TCP] is deprecated, and it has been dropped entirely in V8.12.

When sendmail transports mail on a TCP/IP network, it first sends the envelope-sender's address to the other site. If the other site accepts the sender's address as legal, the local sendmail then sends the list of envelope-recipient addresses. The other site accepts or rejects each recipient address one by one. If any recipient addresses are accepted, the local sendmail sends the message (header and body together). This kind of transaction for sending email is called SMTP and is defined in RFC2821.

1.6.9 UUCP

UUCP is an old-style means of moving email between machines that are only connected with dial-up modems. The line in the configuration file that tells sendmail how to transport over UUCP might look, in part, like this:

Muucp,   P=/usr/bin/uux, F=DFMhuUd, S=12, R=22/42, M=10000000,

This line tells sendmail to send UUCP network mail by running the /usr/bin/uux (UNIX to UNIX eXecute) program.

1.6.10 Other Protocols

sendmail can use many other kinds of network protocols to transport email. Some of them might have shown up when you ran grep earlier. Other common possibilities might look, in part, like one of these:

Mfax,    P=/usr/local/bin/faxmail, F=DFMhu, S=14, R=24, M=100000,
Mmail11, P=/usr/etc/mail11, F=nsFx, S=Mail11From, R=Mail11To,
Mmac,    P=/usr/bin/macmail, F=CDFMmpsu, S=MailMacFrom, R=MailMacTo, A=macmail -t $u

The Mfax line defines one of the many possible ways to send a fax using sendmail. A fax machine transports images of documents over telephone lines. In the preceding configuration line, the /usr/local/bin/faxmail program is run, and a mail message is fed to it for conversion to and transmission as a fax image.

The Mmail11 line defines a way of using the mail11(1) program to transport email over a DECnet network, used mostly by the Open VMS operating system (formerly by Digital Equipment Corporation).

The Mmac line defines a way to transport mail to Macintosh machines that are connected on an AppleTalk network.

In all these examples, note that sendmail sends email over other networks by running programs that are tailored specifically for that use. Remember that the only network sendmail can use directly is a TCP/IP-based network.^[12]

^[12] Actually, we're fudging for simplicity. V8 sendmail can also send messages over an ISO network.

1.6.11 Role as a Daemon

Just as sendmail can transport mail messages over a TCP/IP-based network, it can also receive mail that is sent to it over the network. To do this, it must be run in daemon mode. A daemon is a program that runs in the background independent of terminal control.

As a daemon, sendmail is started once, usually when your machine is booted. Whenever an email message is sent to your machine, the sending machine talks to the sendmail daemon that is listening on your machine:

% grep sendmail /etc/rc*            BSD-based systems
% grep sendmail /etc/init.d/*       SysV-based systems
% grep sendmail /etc/*rc            HP-UX systems (prior to HP-UX 10.0)

One typical example of what you will find is:

/etc/rc.local:if [ -f /usr/lib/sendmail -a -f /etc/mail/sendmail.cf ]; then
/etc/rc.local:   /usr/lib/sendmail -bd -q1h; echo -n ' sendmail'

The second line in this example shows that sendmail is run at boot time with a command line of:

/usr/lib/sendmail -bd -q1h

The -bd command-line switch tells sendmail to run in daemon mode. The -q1h command-line switch tells sendmail to wake up once per hour and process the queue. Command-line switches are covered in Chapter 15.