|
|
11.4 Queue Groups (V8.12 and Above)As of V8.12 sendmail it is possible to group queues according to selected criteria, and then to process each group with custom settings. This versatile ability is enabled and tuned with:
You can best tune queue groups by first understanding their limitations. We cover these topics in this section, but first we need to briefly discuss the default queue group. 11.4.1 The Default Queue GroupPrior to V8.12 sendmail there were no queue groups. Instead, every -q command and every queue option (such as QueueDirectory) applied to all the queue directories you had.[5]
Beginning with V8.12, sendmail offers a way to define both multiple queue directories and a way to group them by function or specialty. For compatibility with old versions, a special queue group named mqueue is the default queue group. It takes on all the properties of every -q command, and every queue option, just like before. When you later declare particular queue groups (as we show in the next section), those additional groups take all their properties from the default group, unless you override a particular property with a specific equate. Those equates and the command-line arguments or options they override are shown in Table 11-2. For example, the following declares two different queue directories: define(`QUEUE_DIR', `/var/spool/mqueue') QUEUE_GROUP(`regularmail', `') QUEUE_GROUP(`slowmail', `P=/var/spool/mqueue/slowqueue') The first line declares the queue used by the default group (always known as mqueue). Any other queue groups that are declared (such as regularmail) will use that same directory unless the directory is overridden by the P= equate, as shown in the third line. That is, the default queue group's queue directory and everything else that is set for the default queue group is inherited by the regularmail group. For the slowmail queue group, however, everything but the queue directory is inherited. (See Section 11.4.2.5 for a description of the P= equate, and for the reason queue group directories must be subdirectories under QUEUE_DIR.) 11.4.2 The Q Configuration CommandQueue groups are declared with the Q configuration command. That command can take a wide range of appearances, but in all guises it takes the name of the queue group and then a sequence of equates: Qgroupname, equates
The name of the queue group (here groupname) must follow the Q with no intervening spaces. If spaces are present, an error such as the following is printed and logged, and that Q line is ignored: file.cf: line line number: queue : `=' expected The equates are optional, but if they are present they must follow the queue group's name and a comma or whitespace, or both: Qgroupname, equates
The equates are formed by selecting one of the keywords shown in the leftmost column of Table 11-2, and following it with an equal sign and the value you wish to assign to that key-letter. Note that only the first letter is looked at by sendmail, so you can use the shorthand shown in parentheses if you wish. Also note that the first letter is case-sensitive—that is, R and r are different. For example, both of the following declare a queue directory (the Path= and P=), and a queue-processing interval of 10 minutes (the Interval= and I=): Qslowmail, Path=/disk1/mail/slowqueues, Interval=10m Qslowmail, P=/disk1/mail/slowqueues, I=10m A comma separates one equate from another. The comma can be optionally surrounded by whitespace characters (spaces and tabs). If the value following the comma is missing, an appropriate error will be printed and logged.
If an equate other than those shown in the table is used, an error such as the following is printed and logged, and that Q line is ignored: file.cf: line line number: Qgroupname: unknown queue equate bad equate here 11.4.2.1 The Flags= (F=) queue-group equateThe F= queue-group equate is used to set flags for the queue group. Currently there is only one flag, the f flag, which tells sendmail to fork multiple times to process the queue group in parallel (the exact opposite of the -qf command-line switch, which tells sendmail to not fork multiple times, but instead to run the queues serially in the foreground). When this F= flag is specified, sendmail forks one queue processor for each queue directory in the group. But note that the sendmail program will only fork up to the total number of parallel processors set by the R= queue group equate. If that limit is fewer than the number of queues, the remaining queues are handled during the next queue run, in round-robin fashion. When the fast processing of a queue group is required, we recommend you specify this F=f queue group flag. If speed is not of concern, you can reduce the system impact by omitting this flag. But if you omit it and then specify multiple runners with the R= queue group equate, the following message will print and be logged: Warning: Q=queuegroup name:R=number: multiple queue runners specified
but flag 'f' is not set
As a performance compromise, some parallelism can be attained and system impact reduced by setting this flag and limiting the number of runners specified with the R= queue group equate. 11.4.2.2 The Interval= (I=) queue-group equateThe I= queue-group equate specifies the time interval at which the queues in the queue group should be processed. The default interval is set by the -qinterval command-line switch, but can be overridden for a queue group using this I= queue group equate: I=interval The interval following the I= is constructed from an integer and a letter. The letters and the meaning of each are listed in Table 11-3. Integer and letter groups can be combined—for example, 5d12h means 5 days, 12 hours.
If the trailing letter is missing, the units default to minutes, thus the following defines an interval of 1 hour, 12 minutes: Interval=1h12 In general, the use of a trailing letter is recommended for clarity, and to avoid problems in the future should sendmail defaults change. 11.4.2.3 The Jobs= (J=) queue-group equateWhen a queue processor starts to process a queue directory, it first gathers a list of all the envelopes in that directory. It then sorts, or randomizes that list, and processes the envelopes in the resulting order. If no limit is imposed, all the envelopes will be processed before the queue run is complete. The default limit, if there is one, is defined by the MaxQueueRunSize option (MaxQueueRunSize). But a separate limit that will override the default can be set for a queue group using this J= equate. If the default is non-zero and if this equate specifies zero, the default queues will have the default limit imposed but this group will have none. This J= queue-group equate is used like this: Jobs=number If number is zero or negative, no limit is imposed. If number is positive, that will be the maximum number of envelopes processed. 11.4.2.4 The Nice= (N=) queue-group equateThe niceness of a process determines its priority to be run. The larger the nice value, the lower the priority. The default nice value varies from one version of Unix to another. In all cases, however, they generally begin with the same nice value, so all processes generally get an equal chance to run. With sendmail, the niceness of its queue processors is set by the NiceQueueRun option (NiceQueueRun). If that option specifies a positive value, the priority is reduced. If that option specifies a negative value, that priority is increased. In general, queue processors should run at a lower priority so as to minimize the adverse impact on other processes. On dedicated mail-sending machines, you might wish to increase the priority. Each queue group inherits its nice value from the NiceQueueRun option, unless this N= queue-group equate is specified. This N= equate is used like this: Nice=10 If the number is missing, nonnumeric, or negative, the niceness change is zero (no change). Otherwise, the niceness is increased (the priority is lowered) by the amount specified. 11.4.2.5 The Path= (P=) queue-group equateThe default location and name of the queue directory or directories is set by the QueueDirectory option (QueueDirectory). That option defines the default directory (for the default queue group mqueue) and the base path for all the other queue directories. The P= queue-group equate does not override the default (as the other equates do), but instead augments it. The path specified by the P= queue-group equate must be a full (absolute) path, and must contain the name of a subdirectory or subdirectories of the default path. To illustrate, consider the following mc file declarations: define(`QUEUE_DIR',`/var/spool/mqueues/q.*') Here, the fist line defines the default queues, which all begin with the characters q. and live under the path /var/spool/mqueues. The second line correctly sets the queue for the aolmail queue group. The base path, /var/spool/mqueues, is the same for both the default and this group. Note that queue group directories can also specify multiple queues (as with the /var/spool/mqueues/bob.* in the third line). The forth line shows that the path specified with P= must not be a relative pathname. If it is, sendmail will print and log the following error and exit: QueuePath hotmailnot absolute The last line shows that the path specified with P= must not use a base path different from the default. If it does, the following error will print and log, and sendmail will exit: QueuePath /var/spool/slowqueuenot subpath of QueueDirectory/var/spool/mqueues: No such file or directory Note, however, that symbolic links under the default queue path are OK. That is, you can declare the last line in the previous example like the following, and then simply make the path you specify a symbolic link to the real directory somewhere else: define(`QUEUE_DIR',`/var/spool/mqueues/q.*') Note, however, that the path pointed to by the symbolic line must be as trusted as the default path, with narrow ownerships and permission (QueueFileMode). 11.4.2.6 The recipients= (r=) queue-group equateThe MaxRecipientsPerMessage option (MaxRecipientsPerMessage) sets the default limit for the number of recipients allowed per envelope. If there are more recipients than that limit in an envelope, sendmail will split the envelope into two or more envelopes, each with the limit or fewer recipients. If the MaxRecipientsPerMessage option is zero, no limit is imposed. The r= queue-group equate allows you to override the default for each queue group. If the default allows unlimited recipients, or a large limit, you can use a smaller setting for your queue group. Or, if the default is too small, you can enlarge it. You use the r= equate like this: recipients=99 Note that a zero or negative expression sets the limit to unlimited. A nonnumeric expression, such as in the last line, also sets the limit to zero (unlimited). 11.4.2.7 The Runners= (R=) queue-group equateThe Runners= (R=) queue-group equate tells sendmail how many queue processors to launch each queue-processing interval. The queues are serviced in round-robin order. So, for example, if your queue group has three queues, and you set R= to 1, 2, 3, and 4, respectively, you will see the runs shown inTable 11-4.
The Runners= queue-group equate is declared like the following: Runners=12 If the number of queue-group runners specified by this equate is more than the number of queue children allowed by the MaxQueueChildren option (MaxQueueChildren), the number of queue-group runners is reduced to that amount, and the following error is logged and printed: Q=queuegroup: R=number exceeds MaxQueueChildren=limit, set to MaxQueueChildren If the MaxQueueChildren option is set to zero, there is no limit to how many queue-group runners you can declare. 11.4.3 How to Declare Queue Groups with the m4 TechniqueYou declare queue groups inside your mc configuration file with the QUEUE_GROUP mc configuration macro. As you have seen in the previous sections, it is used like this: QUEUE_GROUP(`group name', `equates') The queue group name can contain any characters except a comma or a whitespace character (a space or a tab).[6] It must not be surrounded (inside the quotes) with whitespace characters.
The equates form the second argument to the QUEUE_GROUP mc configuration macro. The equates are described in Section 11.4.2. To illustrate, consider the following QUEUE_GROUP mc configuration macro declaration: QUEUE_GROUP(`slowmail', `P=/var/spool/mqueues/slowqueue') Here, the name of the queue group is set to slowmail. The second argument is a single equate, the P= queue-group equate, which defines the queue directory or directories to be used by this queue group. If you want to define which queue group to use for certain delivery agents, you can use the Q= delivery agent equate (Q=) as set, for example, with the LOCAL_MAILER_QGRP mc macro. For example, the following tells sendmail to queue all local mail in the /queues/lq queue directory: QUEUE_DIR(`/queue') QUEUE_GROUP(`localgroup', `P=/queue/lq') define(`LOCAL_MAILER_QGRP', `localgroup') In the first line we set the default queue directory. In the second line we define the queue group localgroup, and set its queue directory to be /queue/lq. In the third line we declare that the Q= equate for the local delivery agent will be: Q=localgroup The fourth line declares support for the local delivery agent. Note that the definition of LOCAL_MAILER_QGRP must precede the MAILER(local), otherwise that definition will be silently ignored. Those four lines cause all mail for local users to be queued in the /queue/lq directory. Note that you can dedicate queue groups for other delivery agents. See Q= for a full description of this process. 11.4.4 The FEATURE(queuegroup) and the access DatabaseThe easiest way to select queue groups based on recipient addresses or recipient domains is by using the queuegroup feature. It is declared in your mc configuration file like this: FEATURE(`queuegroup')
FEATURE(`queuegroup', `default group')
The first line causes the queue group to default to mqueue if a queue group in the access database is missing or nonexistent. The second line allows you to set a different default queue group. For example, consider the following lines from an mc file: QUEUEGROUP(`localgroup', `/queue/lq') FEATURE(`queuegroup', `localgroup') This causes sendmail to use the group named localgroup instead of mqueue as the default if a queue group in the access database is missing or nonexistent. Once you have enabled the queuegroup feature, the next step is to add lines such as the following to the source file for your access database: QGRP:slow-poke.com slowgroup QGRP:root@notify.com fastgroup QGRP:your.domain localgroup Each line that selects queue groups must begin with the literal expression: QGRP: This prefix tells sendmail that you wish to map recipient addresses or domains to queue groups. The first line causes mail to the slow-poke.com domain to use the queue group called slowgroup. This shows that you can list just a domain in the lefthand column and it will work just as expected. The second line causes mail to the specific recipient root@notify.com to use the queue group named fastgroup. This line demonstrates that mail to an individual can be used in the lefthand column. The third line illustrates your local domain, which shows that mail to your domain, your.domain, will use the queue group named localgroup. If you omit the name of the queue group (not recommended), you will need to use the -e command-line switch with makemap to create the database. When you omit the name of the queue group the default queue group is used: QGRP:another.your.domain
Here, if you defined a default queue group when you declared the queuegroup feature, that group will be selected. Otherwise, the group mqueue will be selected for this domain. 11.4.5 Rule Set Queue Group SelectionNormally, the access database, described earlier, is the easiest way to select queue groups. There might be times, however, when selecting by recipient address or domain is not sufficient. Should such a situation arise, you could set up your own rule sets. But be forewarned that if you do, the queuegroup feature cannot be used. If you try to use both, you will get the following warning every time sendmail starts to run: WARNING: Ruleset queuegroup has multiple definitions The first step in declaring your own rules to select queue groups is to declare a special rule set called queuegroup. You do that in your mc configuration file using the LOCAL_RULESETS macro: LOCAL_RULESETS
Squeuegroup
The way this rule set works is simple. Any queue group for a recipient address that a rule selects is returned following the $# operator. For example, consider the following: R $* <@some.domain> $# somegroup Here, mail bound for any user at some.domain will be queued in the somegroup queue group. Normally queuegroup rule sets are used to select queue groups based on the recipient. If you wish to select based on the sender, you can do so using rules something like the following: LOCAL_RULESETS Squeuegroup R $* $: $>canonify $&f R $+ <@ lists.domain.> $# lists First, we fetch the sender address using $&f, and pass it through the canonify rule set 3 to focus on the host part. The second rule matches any user at the domain lists.domain, and selects the lists queue group. Because there are no more rules following the second one, this rule set returns without selecting a queue group. If the queuegroup rule set fails to select a queue group, the default queue group (mqueue) is used. Other possible uses for the queuegroup rule set might include:
Note that there are limitations on the use of this queuegroup rule set. First, this rule set is called directly from inside sendmail, so you should not call it from inside your own rules (if you do, the selected queue group will be ignored). And second, the queuegroup feature also uses this rule set, so you cannot share it with that feature.[7]
11.4.6 Queue Group LimitationsAs you saw in Section 11.4.1, the default queue group (mqueue) is defined by options and the command line. If any given Q configuration command is missing a given equate, that queue group inherits that property as defined by the default queue group. There are, however, properties for the default queue group which have no equivalent equates. These properties are inherited by all queue groups and cannot be overridden with a queue-group equate. They are:
 |
|
|
increase niceness by 10, lower priority
Nice=0 
a symbolic link to /var/spool/slowqueue