![]() ![]() |
11.8 Cause Queues to Be ProcessedThe sendmail program offers two different methods for processing its queues. It can be told to process them periodically or to process them once and then exit. 11.8.1 Periodically with -qThe -q command-line switch is used both to cause queues to be processed and to specify the interval between queue runs. A typical invocation of the sendmail daemon looks like this: /usr/sbin/sendmail -bd -q1h Here, the sendmail program is placed into listening mode with the -bd command-line switch. The -q1h command-line switch tells it to process the queue once each hour. Note that either switch puts sendmail into the background as a daemon. The -bd switch just allows sendmail to listen for incoming SMTP connections. Consider the following: /usr/sbin/sendmail -bd /usr/sbin/sendmail -q1h This runs two daemons simultaneously. The first listens for incoming SMTP connections. The second processes the queues once per hour. The time expression following the -q is constructed from an integer followed by a letter. The letters and the meaning of each are listed in Table 11-5. Integer and letter groups can be combined—for example, 5d12h means 5 days, 12 hours. If a letter is missing, the default is minutes.
At small sites, where mail messages are rarely queued, the time interval chosen can be small to ensure that all mail is delivered promptly. An interval of 15m (15 minutes) might be appropriate. At many sites an interval of one hour is probably best. It is short enough to ensure that delays in delivery remain tolerable, yet long enough to ensure that queue processing does not overlap (see Section 11.8.3 for a way to run a persistent queue runner that avoids overlapping runs). At large sites with huge amounts of mail and at sites that send a great deal of international mail, the interval has to be carefully tuned by observing how long it takes sendmail to process its queues and what causes that process to take a long time. Points to consider are the following:
11.8.2 From the Command LineThe -q command-line switch, invoked without a time interval argument, is used to run sendmail in queue-processing mode. In this mode, sendmail processes queues once and then exits. This mode can be run interactively from the command line or in the background via cron(8). Other command-line switches can be combined with -q to refine the way queues are processed. The -v (verbose) switch causes sendmail to print information about each message it is processing, and to process multiple queues sequentially. The -d (debugging) switch can be used to produce additional information about the queue. We'll discuss the -v switch as it applies to the queue later in this chapter. Those -d debugging switches appropriate to the queue can be found in Table 16-4 of Section 16.6. V8 sendmail allows variations on -q: -qI allows you to specify a specific message identifier for processing; -qR allows you to specify specific recipient addresses for processing; and -qS allows you to specify specific sender addresses for processing.[11]
11.8.2.1 Process the queue once: -qThe -q command-line switch, without an interval argument, tells sendmail to process the queue once, then exit. As such, this switch is a handy administrative tool. When the queue fills unexpectedly between queue runs of the daemon, for example, the -q command-line switch can be used to force an immediate queue run: # /usr/sbin/sendmail -q When multiple queues are run this way, they are all processed in parallel (Section 11.3.1.2). On machines that do not run the sendmail daemon, the -q command-line switch can be used in conjunction with cron(8) to periodically process the queue. The following crontab(5) file entry, for example, causes sendmail to be run once per hour, at five minutes past the hour, to silently process its queues and exit: 5 * * * * /usr/sbin/sendmail -q >/dev/null 2>&1 When used in conjunction with other switches (shown next), the -q switch allows many queue problems to be conveniently handled. 11.8.2.2 Combine -v with -qThe -q switch without an argument prevents sendmail from running in the background and detaching from its controlling terminal. But it also runs silently. To see what is going on, use the -v command-line switch in combination with the -q: % /usr/sbin/sendmail -v -q The -v command-line switch causes sendmail to print a step-by-step description of what it is doing. When running multiple queues, it also causes them to be processed in sequence. To illustrate, consider the following output produced by using both the -v and -q command-line switches: Running /var/spool/mqueue/dB9JBR106687 (sequence 1 of 2) <adams@dc.gov>... Connecting to dc.gov via ddn... Trying 123.45.67.8... Connection timed out during user open with DC.GOV <adams@dc.gov>... Deferred: Host DC.GOV is down Running /var/spool/mqueue/dB9JDWt06701 (sequence 2 of 2) <help@irs.dc.gov>... Connecting to irs.dc.gov via ddn... Trying 123.45.67.88... connected. 220 irs.dc.gov Sendmail 5.57/3.0 ready at Mon, 27 Jan 92 09:16:38 -0400 Here, two queued messages are being processed. The first fails because of a connection timeout and is requeued for a later queue run. The second succeeds (we omit the full SMTP dialog). After its delivery is complete, it is removed from the queue. 11.8.2.3 Process by identifier/recipient/sender: -q[ISR]With V8 sendmail you can process a subset of all queued messages. You can select which to process based on queue identifier, recipient address, or sender address: -qIident The -qI variation is followed by a queue identifier such as dB9JDWt06701. The -qR is followed by the address of a recipient. The -qS is followed by the address of a sender. In all three variations there must be no space between the uppercase letter and the identifier or address. These variations are used to limit the selection of queued files that are processed. For example: % /usr/sbin/sendmail -qSroot -qRbiff@here Here, the queue is processed once. Only messages from root are processed. Of those, only messages that have biff@here as one of the recipients are processed. In all three variations a partial specification of queueid, recipient, or sender is viewed by V8 sendmail as a substring. For example: -qSroot matches mail from all of the following: root ben@groots.edu ben@GROOTS.EDU The last line further illustrates that the substring match is a case-insensitive one. The substring match is literal. Wildcard characters (such as *) and regular expressions (such as .*@.*edu) won't work and might confuse the shell from which you run sendmail. Multiple specifications can be combined on the command line (as shown earlier), but they all AND together: % /usr/sbin/sendmail -qI123 -qSroot -qR@host.edu Here, the queue is processed only for messages with the number 123 anywhere in the queue identifier that are also from root and that are also addressed to anyone at host.edu. You can use the mailq command to preview the effect of these switches. For example, the following command will list (but not send) the messages that would be processed by the previous command line: % mailq -qI123 -qSroot -qR@host.edu 11.8.2.4 Process by negated identifier/recipient/sender (V8.12 and above)Beginning with V8.12 sendmail, you can prefix any of the I, S, or R specifications to -q with an ! character. The presence of an ! character prefix instructs sendmail to invert the logic of that particular test. For example: % mailq -q\!Sroot -qR@host.edu Here, we wish to process the queue for any message addressed to anyone at host.edu, just as we did in the previous section. But this time, we want to further limit that processing by including only messages with a sender that is not (the !) from root. Note that we prefix the ! with a backslash to protect it from the csh or tcsh shells (the backslash is not necessary for the Bourn shell and its derivatives). In summary, these specifications for how to limit the queue can be mixed and matched, specified and negated, in any combination that works for you: -qIident Only the ! character can be used to negate. Any other character will be interpreted as an argument to -q. The ! prefix must not follow the I, R, or S. If it follows, it will be interpreted as part of the expression to match. 11.8.2.5 Process by queue group with -qG (V8.12 and above)Beginning with V8.12 sendmail, you can use the -qG command-line switch to process queues based on selected queue groups. It is used like this: -qGgroupname Here, only mail queued in the qroupname directories will be processed. If the name specified is of an unknown group, the following error will print and log, and the queue run will fail: Queue group groupname unknown This command-line switch can be used in combination with all the other queue processing switches. Consider, for example, the following: % /usr/sbin/sendmail -qR@hostA.domain -qGslow Here, sendmail will deliver queued messages only to users at hostA.domain, if those messages were queued in the slow queue group. Multiple -qG command-line switches cannot be used at the same time. If you combine them, the following error will print and be logged: Cannot use multiple -qG options Unlike the -q[IRS] switches discussed earlier, the -qG command-line switch cannot be negated: % /usr/sbin/sendmail -q!Gmqueue Cannot use -q!G 11.8.2.6 Process the queue via ESMTP ETRNThe ESMTP ETRN command, based on RFC1985, causes V8.8 and above sendmail to asynchronously process its queue in a manner similar to the -qR command-line switch (Section 11.8.2.3). This command allows dial-on-demand sites to make an SMTP connection and to force the other side to process and send any mail that is queued for them. The form of this ESMTP command looks like this: ETRN host If host is missing, this error message will be returned: 550 Parameter required Otherwise, the queue will be processed just as if the following command-line argument were given: -qR@host In both cases a qf file will be processed if it has host anywhere in the host part (following the @) of one of its R lines. The only difference here is that the former (the ETRN) operates asynchronously. That is, sendmail forks a copy of itself, and the forked child processes the queue. Beginning with V8.12 sendmail, you can cause sendmail to process the queues by queue group. To do this, just replace the hostname with the name of the queue group, and prefix it with a literal # character: ETRN #groupname If the groupname has been defined, the queues for that queue group will be processed. Otherwise, the following error will be returned: 459 4.5.4 Queue badname unknown One way to use ETRN is with a perl(1) script supplied with the sendmail source. See the file: contrib/etrn.pl You might have to change the first line of this file to get it to work, depending on where you installed perl(1) on your system. To run this program, just give it the name of your MX server: % contrib/etrn.pl your.mx.server The etrn.pl script will connect to that server, and it will send an ETRN command to that server for each host you list with a Cw or Fw command in your configuration file. The etrn.pl script is also its own manual page, which you can read with a command such as this: % nroff -man contrib/etrn.pl | more 11.8.3 Persistent Queue Runners with -qpV8.12 sendmail introduced persistent queue runners as a solution to some of the problems caused by periodic queue runners. Periodic queue runners are the result of a normal -qinterval command-line switch, or a Runners= queue group equate. Either causes:
Persistent queue runners avoid these problems because a single process is dedicated to a queue, a queue group, or a grouping of queue groups (called a "workgroup"). A persistent queue runner is launched just like the periodic command-line queue runner, but with the addition of a p character: -qpinterval The p causes one or more persistent queue runners to be launched, one per queue group. One will be launched to handle your default queue group, and one more will be launched to handle each queue group defined by a QUEUE_GROUP mc option. Depending on the number of queue directories in each, these can be combined into a single workgroup. When you have many queue groups, you can end up with multiple workgroups controlling persistent runners. Each persistent queue runner will sleep for interval. When it awakes, it reads all the files in the queues that belong to its workgroup, and sorts all the envelopes it finds into the proper order needed for delivery. After it has finished ordering the envelopes, it launches one or more regular queue runners to perform delivery using that already processed list. This can significantly reduce the disk I/O compared to that needed by periodic queue runners. When the last of the regular queue runners has finished processing (and exited), the persistent queue runner goes back to sleep for interval. In general, persistent queue runners are valuable only at sites that normally have queues that are very full. When a queue is normally near empty, persistent queue runners can introduce unforeseen delays. Note that a persistent queue runner will sleep again only when all of its regular queue runners have finished. One regular queue runner, delivering to a very slow site, can appear to hang, and so can cause the persistent queue runner to also appear to hang. Subsequent queue runs will be delayed until the hung site times out, allowing the persistent queue runner to sleep interval again. At large sites, such delays will eventually smooth out due to the normal distribution of slow jobs. At small sites, such delays might be noticed and objected to. In general, persistent queue runners should be reserved for sites with full queues. If interval is omitted, the default interval becomes one second: -qp When the default interval is used (by omitting the interval), the persistent queue runner will sleep one second between queue runs, unless the prior queue run was empty, in which instance it will sleep five seconds. If you choose the default interval, we recommend you also set the MinQueueAge option (MinQueueAge). If interval is specified as zero, the effect is the same as if it were omitted. If interval is negative, the following error is logged and printed and sendmail exits: Invalid -q value If interval is nonnumeric (if you specify O when you mean zero) the following error is logged and printed, and sendmail exits: Invalid time unit `O' The process that was given the -qp command-line switch is the controlling process. It could be the listening daemon (if -bd or -bD were also used), or it could be a queue processing daemon (if only -qp and other queue processing limiters were specified). The controlling process has two special properties:
If a persistent queue runner core dumps, the following will be logged and that queue runner will not be restarted: persistent queue runner=number core dumped, signal=signal If a persistent queue runner exits because of a caught signal, the following is logged and that queue runner is restarted: persistent queue runner=number died, signal=signal If a persistent queue runner is restarted because of a SIGHUP, the following is logged: restart queue runner=number due to signal signal If the -dno_persistent_restart debugging command-line switch is specified, a failed persistent queue runner will not be restarted, and the following error will be logged: persistent queue runner=number, exited A persistent queue runner will not be restarted if it has already been restarted 10 times. Instead, the following error will be logged and that persistent queue runner marked as bad: ERROR: persistent queue runner=number restarted too many times, queue runner lost If this happens, examine your logs. Some nonmail-related process might be signaling your persistent queue runners, or you might have bad memory, or you might have made a mistake when building sendmail and should rebuild it, or you might have a junior system administrator who does not know how to correctly restart sendmail. Persistent queue runners look like executing periodic queue runners in process listings: root 22958 476 ? S 08:43 0:00 sendmail: accepting connections root 22947 512 ? S 08:32 0:00 sendmail: running queue: /var/spool/mqueues/ q.1/df Here, the first line shows the controlling process, and the second line shows a persistent queue runner. Note that, even though the second entry says "running," it might not be. ![]() |
![]() ![]() |