SA/Sieve Woes

Question

Hi all,
We have MS 2005Q6 core patch -58 installed. Below is the story and problems that we have encountered when setting up Spam Assassin.
SA is setup to label spam with addtag [SPAM] so that every spam email is tagged in the tcp_local channel. The spamfilter1_string_action looks like this:
require ["addheader","fileinto"];
addtag "[SPAM]";
addheader "$U";
fileinto "Spam";
stop;Problem 1. In a user level sieve script, the (header :matches "Subject"
"[SPAM]*") did match nothing even though the [SPAM] tag did appear in the
final subject line. It looks as if the addtag processing is only done
after the header match operations, or the header match uses the "original"
subject line for the match. Is there anything wrong with the sieve script?
After some time, we modified the script with the intention to give every scanned email detailed sa results even for non-spams, and at the same time add a new header for user leve sieve scripts.
#mode=2,field= in spamassassin.opt
require ["spamtest","relational","comparator-i;ascii-numeric","fileinto","addheader"];
spamadjust "$U";
addheader "Spam-Test: $U";
if spamtest :value "ge" :comparator "i;ascii-numeric" "5"{
  addheader "Spam: Y";
  addtag "[SPAM]";
  fileinto "Spam";
}else{
  keep;
}Problem 2. The above script runs well, except one thing that cost me a full day debugging. Note the 2nd addheader line, which reads addheader "Spam: Y". The problem is, it seems that we cannot have a 2nd addheader in one single script, which is longer than 7 characters in length including the space in between. If you say addheader "ABCDE: Y", you are busted. The filter will just refuse to run. Is this a bug or something?
OK, never mind, who care about a 7 byte header. We want to have a fine-grain control over the traditional email forwarding feature that uses mailForwardingAddress attribute. I suppose Sieve is more flexible when it comes to filtering and redirecting mail. So we disable the mailForwardingAddress, and use Sieve. So now, we try to setup a user level sieve to redirect important emails, even it is identified as spam like this:
require ["fileinto","copy"];
if header :match "Spam" "Y" {
  fileinto :copy "Spam";
if header :match "Priority" "Urgent" {
  redirect "[email protected]";
}Problem 3. When SA identifies a spam, it is marked as such with addtag [SPAM] (see code above). The tag should appear once. It is the case for the email left in the Spam folder of the server. But the email received by "[email protected]" contains 2 tags [SPAM] [SPAM] in the subject line. Clearly, SA is run twice, one for the incoming message, and one for the redirected message, and the log file confirms this. It seems that the redirect action cause the mail to be queued in the tcp_local channel, which causes SA to run. But it is interesting that the traditional emailForwardingAddress attribute kind of forwarding doesn't have this kind of behavior. Is there a way we can prevent SA from running twice in sieve to save resources and not to duplicate the tag?
Anyone can shed some lights on these matters?
Thanks in advance.
PY

Answer 1

Hi,
SA is setup to label spam with addtag [SPAM] so
that every spam email is tagged in the tcp_local
channel. The spamfilter1_string_action looks like
this:
require ["addheader","fileinto"];
addtag "[SPAM]";
addheader "$U";
fileinto "Spam";
stop;Problem 1. In a user level sieve script, the (header
:matches "Subject"
[SPAM]*") did match nothing even though the [SPAM]
tag did appear in the
final subject line. It looks as if the addtag
processing is only done
after the header match operations, or the header
match uses the "original"
subject line for the match. Is there anything wrong
with the sieve script?Please provide the actual spamfilter1_string_action you used - not just the 'cleaned' version.
Also I should point out that matching on a modified subject header can lead to unexpected behaviour e.g. forwarded spam emails getting filtered to spam folder because [SPAM] appears in the subject line. It is much better to use the spamadjust/spamtest if possible.
After some time, we modified the script with the
intention to give every scanned email detailed sa
results even for non-spams, and at the same time add
a new header for user leve sieve scripts.
#mode=2,field= in spamassassin.opt
require
["spamtest","relational","comparator-i;ascii-numeric",
"fileinto","addheader"];
spamadjust "$U";
addheader "Spam-Test: $U";
if spamtest :value "ge" :comparator "i;ascii-numeric"
"5"{
addheader "Spam: Y";
addtag "[SPAM]";
fileinto "Spam";
lse{
keep;Problem 2. The above script runs well, except one
thing that cost me a full day debugging. Note the
2nd addheader line, which reads addheader "Spam:
Y". The problem is, it seems that we cannot have a
2nd addheader in one single script, which is longer
than 7 characters in length including the space in
between. If you say addheader "ABCDE: Y", you are
busted. The filter will just refuse to run. Is this
a bug or something?There is one limit you may be hitting (I found this one yesterday) and that is that spamfilterX_string_action is currently limited to a maximum of 256 characters - that is AFTER the expansion of the $U btw. Please log a support call (if you haven't already) and ask to be added to bug #6485247.
There is a workaround to this limit and that is to use channel-level filtering, refer to my last reply in a forum question yesterday:
http://forum.sun.com/jive/thread.jspa?threadID=109607
OK, never mind, who care about a 7 byte header. We
want to have a fine-grain control over the
traditional email forwarding feature that uses
mailForwardingAddress attribute. I suppose Sieve is
more flexible when it comes to filtering and
redirecting mail. So we disable the
mailForwardingAddress, and use Sieve. So now, we try
to setup a user level sieve to redirect important
emails, even it is identified as spam like this:
require ["fileinto","copy"];
if header :match "Spam" "Y" {
fileinto :copy "Spam";
if header :match "Priority" "Urgent" {
redirect "[email protected]";Problem 3. When SA identifies a spam, it is marked
as such with addtag [SPAM] (see code above). The tag
should appear once. It is the case for the email
left in the Spam folder of the server. But the email
received by "[email protected]" contains 2 tags
[SPAM] [SPAM] in the subject line. Clearly, SA is
run twice, one for the incoming message, and one for
the redirected message, and the log file confirms
this. It seems that the redirect action cause the
mail to be queued in the tcp_local channel, which
causes SA to run. But it is interesting that the
traditional emailForwardingAddress attribute kind of
forwarding doesn't have this kind of behavior. Is
there a way we can prevent SA from running twice in
sieve to save resources and not to duplicate the
tag?
Anyone can shed some lights on these matters?
Thanks in advance.Well I'm not going to troubleshoot each of your issues as that will take me a ages - sieve scripts can be tricky. What is your overall goal of the filtering? From what you have mentioned so far it is:
At the system level:
-> add the [SPAM] tag to all spam emails
-> add the X-Spam: Y to all spam emails emails (you should use X- for custom headers btw.)
-> add the Spam-test: <score> to all scanned emails
At the user level:
-> filter all Spam emails to a spam folder
-> forward non-spam high priority emails to another address
Are you using separate front-end MTA/back-end store configuration and not using LMTP between the two? This will change whether spamadjust/spamtest can be used for the user level sieve rules.
Regards,
Shane.