MIME header missing "filename=" results in BAD messages

Question

I've been trying to track down several issues involving what appear to
be MIME handling issues. We are running GroupWise v7.01 (we have not
installed the "interim release" yet), and all components on the server
are at that release level (MTA, POA, GWIA, WebAccess).
One of the issues I am currently working on involves problems receiving
attachments from an outside sender. Their attachments are readable if
sent to an outside test account (i.e. - we've tested it with Yahoo
web-based mail, and a RoadRunner POP3 account using Outlook as the client).
The problem is that these e-mails either get dumped by the GWIA to the
GWPROB directory, or the e-mails go through, but the attachments are
corrupted (they do show up as attachments to the e-mail, but are
unreadable).
I haven't looked at why some e-mails go through and some don't, but I
did find out why the attachments are unreadable on the delivered
e-mails, and why the rest are just undeliverable. For whatever reason,
the sender's mail server (or AV gateway or whatever else had a chance to
touch the e-mail) is either not inserting, or is stripping out the
"filename=" from the MIME header just before the encoded attachment.
Adding this back into the message (by editing the original e-mail
w/header, and dropping the modified message into the GWIA "RECEIVE"
directory) fixes the attachment AND makes the e-mail deliverable.
Below I have an example of a bad e-mail which was originally sent to our
GWPROB directory, but was successfully delivered with a readable
attachment through a separate POP3 account (RoadRunner). Following this
"bad" e-mail example is the "fixed" version of that same e-mail, which
was delivered successfully to our user, with the attachment also being
readable.
*************** This is the original "BAD" e-mail ***************
MAIL From:<[email protected]> SIZE=73686
RCPT To:<[email protected]>
Received: from test.rr.com ([65.xx.xx.xx])
by smtp.ourhost.com with ESMTP; Wed, 21 Feb 2007 15:28:27 -0500
Received: from local.com (rrcs-24-xx-xx-xx.central.biz.rr.com [24.xx.xx.xx])
by test.rr.com (8.13.6/8.13.6) with ESMTP id l1LKSFTm009106
for <[email protected]>; Wed, 21 Feb 2007 15:28:24 -0500 (EST)
Message-Id: <[email protected]>
From: [email protected]
To: [email protected]
Date: Wed, 21 Feb 2007 15:25:54 -0500
MIME-Version: 1.0
X-Mailer: Telexis Gateway
Content-Type: multipart/mixed;
boundary="Mark=Num1_Lev1_2007221202554265_Tlx"
X-Virus-Scanned: Symantec AntiVirus Scan Engine
--Mark=Num1_Lev1_2007221202554265_Tlx
Content-Type: text/plain
Content-Transfer-Encoding: Quoted-Printable
This is a test
-Test Sender
--Mark=Num1_Lev1_2007221202554265_Tlx
Content-Type: application/vnd.ms-excel;
name="1st Q 2007 Income to Budget.xls"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
"1st Q 2007 Income to Budget.xls"
0M8R4KGxGuEAAAAAAAAAAAAAAAAAAAAAPgADAP7/CQAGAAAAAAAAAAAAAAABAAAAZgAAAAAAAAAA
EAAA/v///wAAAAD+////AAAAAGUAAAD/////////////////////////////////////////////
--Mark=Num1_Lev1_2007221202554265_Tlx--
************** END OF BAD E-MAIL **************
*************** This is the "fixed" e-mail ***************
MAIL From:<[email protected]> SIZE=73686
RCPT To:<[email protected]>
Received: from test.rr.com ([65.xx.xx.xx])
by smtp.ourhost.com with ESMTP; Wed, 21 Feb 2007 15:28:27 -0500
Received: from local.com (rrcs-24-xx-xx-xx.central.biz.rr.com [24.xx.xx.xx])
by test.rr.com (8.13.6/8.13.6) with ESMTP id l1LKSFTm009106
for <[email protected]>; Wed, 21 Feb 2007 15:28:24 -0500 (EST)
Message-Id: <[email protected]>
From: [email protected]
To: [email protected]
Date: Wed, 21 Feb 2007 15:25:54 -0500
MIME-Version: 1.0
X-Mailer: Telexis Gateway
Content-Type: multipart/mixed;
boundary="Mark=Num1_Lev1_2007221202554265_Tlx"
X-Virus-Scanned: Symantec AntiVirus Scan Engine
--Mark=Num1_Lev1_2007221202554265_Tlx
Content-Type: text/plain
Content-Transfer-Encoding: Quoted-Printable
This is a test
-Test Sender
--Mark=Num1_Lev1_2007221202554265_Tlx
Content-Type: application/vnd.ms-excel;
name="1st Q 2007 Income to Budget.xls"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
filename="1st Q 2007 Income to Budget.xls"
0M8R4KGxGuEAAAAAAAAAAAAAAAAAAAAAPgADAP7/CQAGAAAAAAAAAAAAAAABAAAAZgAAAAAAAAAA
EAAA/v///wAAAAD+////AAAAAGUAAAD/////////////////////////////////////////////
--Mark=Num1_Lev1_2007221202554265_Tlx--
************** END OF "FIXED" E-MAIL **************
The only change I made, which both allowed the above e-mail to be
delivered and made the attachment readable, was the addition of the
"filename=" just before the file name on the line after the
"Content-Disposition: attachment;".
I did the same test with an e-mail that had the same "filename="
missing, but somehow was successfully delivered to our user (again, the
attachment was unreadable). Adding the "filename=" into the resent
message did make the attachment readable.
I haven't looked to see if the sender's system is actually non-compliant
for excluding the "filename=" (I will be doing that later this evening),
or whether this is a GWIA problem for not being able to handle this. I
just wanted to throw this out there to see if anyone else has seen this.
Does the "interim release" update by chance fix this?
Thanks,
Greg Niese

Answer 1

Does this email come in directly to GWIA (or do you have a relay host in
between)? I must say I've never experienced this problem on any of the GW
systems I administer. Could you tell us what domain this is coming from?
Ted Kumsher
>>> On 2/23/2007 at 8:21 AM, in message
<[email protected]>, Greg
Niese<[email protected]> wrote:
> I agree that this shouldn't be accepted as a valid "filename parameter"
> line. There are two ways to "not accept" this invalid parameter: "stop
> processing" the e-mail because of the problem (discard the e-mail), or
> discard the bad parameter and continue processing the rest of the e-mail
> (this could only work when the parameter is optional, as in this case).
> The GWIA seems to take the route of "stop processing", and then moving
>
> the resultant "bad" e-mail to the GWPROB directory.
>
> I say "seems" because in many cases where this occurs the e-mail doesn't
> get sent to the GWPROB directory, but instead still gets through the
> GWIA to the user. The attachments are corrupted, but still show up
> correctly in the attachment window, so everything seems OK with the
> e-mail & attachment - until you try to open the attachment.
>
> If there is a semi-colon after the "Content-disposition" type, the GWIA
> should see a valid disposition parameter following. But if it sees an
> invalid parameter, as in this case, what is the GWIA doing? It must
> either be discarding the invalid parameter, or "uses" it by either
> incorrectly parsing it as part of the header, or lumping it in as part
> of the attachment. If it "uses" in any way the invalid line of data -
> that sounds like the definition of "accepting" it, isn't it?
>
> To test whether the GWIA is actually using, and therefore "accepting",
> this bad parameter, I simply removed the malformed parameter (having the
>
> line in or out shouldn't matter if the GWIA's not using it), and I tried
>
> this both with the semi-colon removed from the end of the
> "content-disposition" type, and with that semi-colon left in place. In
> both cases the GWIA properly processed the e-mail and the attachment was
> readable! What does that mean? If the GWIA was truely NOT
> using/accepting that invalid header parameter (i.e. - the GWIA discards
> it), then the e-mail and attachment should go through OK. Since this
> isn't what happens when the bad line is left in place (the attachments
> DO NOT come through OK, but the e-mail often does get to the user's
> mailbox), then the GWIA must be accepting that bad parameter as either a
>
> part of the header, or a part of the attachment.
>
> I suspect that in those cases that these e-mails are sent to the problem
> directory, instead of being delivered with a corrupt attachment, the
> GWIA isn't really recognizing that the optional disposition parameter is
>
> "bad". The reason for the e-mail going to the problem directory is
> probably due to an error while trying to decode the attachment if the
> bad parameter is treated as part of the attachment, or if part of the
> attachment is being lumped in with the malformed parameter as it is
> parsed.
>
> I agree that how the GWIA should go about "not accepting" bad header
> parameters may be debatable (assuming that the GWIA were actually
> halting processing of the e-mail when this type of problem is found, and
> then sending the e-mail to the problem directory every time). But in
> cases such as this, since the parameter itself is optional, it would
> seem that the better route would be for the GWIA to discard the
> parameter and continue processing, instead of "discarding" the whole
> e-mail (to the problem directory). "Discarding the bad parameter" seems
> to be what is being done by at least a few other competing SMTP agents,
> since the other mail systems I tested with did get the e-mail through
> without corrupting the attachment.
>
> I may suggest that the sender's company fix whatever is going on at
> their server, but it probably won't do any good (I've already been told
> that they don't experience this problem with anyone else). In the
> meantime, my user (yes...he is the CEO), will have to continue to have
> some important e-mails sent to one of his personal accounts, because
> those e-mail systems receive these e-mails fine, but our GroupWise
> system can't.
>
> :-(
>
> -Greg
>
>
>
> Michael Bell wrote:
>> No it shouldn't, IMO.
>>
>> It's a matter of interpretation, of course. But GWIA is 100% right in
> not
>> accepting this as vaid (that part is beyond any debate - it's in the
> RFC).
>>
>> The only part that can be debated is "should GWIA then pass the buck"?.
> In
>> olden days, many would have said yes.
>>
>> But In general, no. Moving away from Postel's law, which was formulated
> in a
>> pre-malware age, is pretty much needed. Tight specifications and
> pickiness
>> is good. Collateral damage is unfortunate, and of course one needs to be
>
>> flexible if this is a widespread issue (which frankly, it is not in this
>
>> case), but otherwise it is up to to sender to fix their issue.
>>
>> 90% of viruses and malware in the last 6 years (and I'm well aware of
> them,
>> working in the email security field) are from MALFORMED e-mail, that the
>> gateway and client misinterpreted. Many of these were MS issues, but
> that's
>> another point. Point is you have to be STRICT.
>>
>> In this case this violates the generic MIME RFCs which state that if you
> do
>> have parameters, they all gotta be separated with semicolons (PASS), and
>
>> have to be name=value pairs (FAIL).
>>
>>
>> "Greg Niese" <[email protected]> wrote in message
>> news:[email protected]...
>>> Well, according to RFC 2183, the "filename=" string has to be present if
>
>>> the Filename parameter is going to be used. So the sending host seems
> to
>>> be "broken" in that regard.
>>>
>>> My next question then is: Should the GWIA ignore the "stray" file name
> in
>>> these e-mail headers, instead of allowing it to be lumped in with the
>>> attachment (I'm assuming that's why the attachment is trashed)? It
>>> appears that other systems are able to do this.
>>>
>>> -Greg
>>>
>>> Greg Niese wrote:
>>>> I haven't looked to see if the sender's system is actually
non-compliant
>>>> for excluding the "filename=" (I will be doing that later this
evening),
>
>>>> or whether this is a GWIA problem for not being able to handle this. I
>>>> just wanted to throw this out there to see if anyone else has seen
this.
>>>>
>>>> Does the "interim release" update by chance fix this?
>>>>
>>>> Thanks,
>>>>
>>>> Greg Niese
>>>>
>>
>>