11.7 How the Queue Is Processed

Over time, messages can gather in the queue awaiting delivery. They remain there until sendmail performs a queue run to process the queue. The sendmail program can be told either to process the queue periodically (when run as a daemon) or to process the queue once, then exit. Each time sendmail processes the queue, it also performs a series of operations that are intended to improve the efficiency with which it delivers messages.

First the queue directory is opened for reading. If that directory cannot be opened, sendmail syslog(3)s the following message at LOG_CRIT and exits:

cannot opendir(/var/queue): reason here

This error is usually the result of a user running sendmail in an unsafe manner, with a -C command-line argument, for example. It can also result from sendmail attempting to open an NFS-mounted queue directory, where root is mapped to nobody.

Next, the qf files are read to gather their priorities and times (the P and T lines). If a qf file cannot be opened, it is quietly ignored unless a -d41.2 debugging command-line switch is specified, in which case the following error message is printed:

orderq: cannot open qfdB928RR04181 (reason)

Prior to V8.7 sendmail, there was a hard limit on the number of messages that could be processed at any time. If more than QUEUESIZE (defined in conf.h, typically 1000) messages were in queue, only the first QUEUESIZE (1000) of them would be processed! Ordinarily, this was not a problem. But it could quickly become one if your queue were clogged with a huge number of undeliverable messages (where the first 1000 continued to be deferred). In that case the only solution is to temporarily move the 1000 messages out of the way by hand (Section 11.9.1) and clear the queue. The only way to detect this situation is to print the queue (Section 11.6).

V8.7 and above sendmail dynamically allocate memory to process the queue. If more than QUEUESIZE messages are found, sendmail will print the following notice and process them:

grew WorkQ for queue_directory to bytes

As an alternative to this dynamic behavior, V8.7 and above sendmail offer a hard limit that is somewhat like the old version but is site-tunable with the MaxQueueRunSize option (MaxQueueRunSize). After all the qf files have been gathered, they are sorted in order of cost. Messages with the lowest value on the P line have the highest priority (lowest cost) and are processed first.

Beginning with V8.7, sendmail also offers the QueueSortOrder option (QueueSortOrder), which allows you to sort by priority (as before), by priority and hostname, by date queued, or (beginning with V8.10) by filename or (beginning with V8.12) in random order. Once all the messages have been sorted, sendmail processes each in turn.

11.7.1 Processing a Single Message

A single queued message has a single sender but can have many recipients. When processing a queued message, sendmail attempts to deliver it to all recipients before processing the next queued message.

The first step in processing a queued message is to lock it so that concurrent runs of sendmail do not attempt to process it simultaneously (Section 11.2.3.1). Then the qf file is opened and read. The sender and all the recipients are gathered from the corresponding S and R lines.

For each recipient, delivery is attempted. If delivery is successful, that recipient's address is removed from the sendmail program's internal list of recipient addresses. If delivery fails, that address is either left in the list or bounced, depending on the nature of the error.

After all recipients have been either delivered, bounced, or left in the list, sendmail reexamines that list. If there are no recipients left in it, the message is dequeued (all the files in the queue directory that compose it are removed). If any recipients are left, each recipient results in an M line that is assigned the last error message for that recipient, and the qf file is rewritten with the list of the remaining recipients and a dot. Finally, the qf file is closed, thus freeing its lock.

Under V8 sendmail the CheckpointInterval option (CheckpointInterval) causes checkpointing of this process. When this option has a positive value, the qf file is rewritten after that value's number of recipients have been processed. For example, consider a mail message to five recipients. If the CheckpointInterval option is set to a value of 2, the qf file is rewritten after the first two recipients have been processed, then again after four, and again after they all have been processed. This keeps the qf file reasonably up to date as protection against sendmail being improperly killed or the machine crashing.