Re: [apps-discuss] FW: Comments on Malformed Message BCP draft

[Barry Leiba <[e-mail]>]

On reading all the comments about this, and thinking about it myself,
I'm of a very mixed mind.

First: I have no sympathy for the comments that we should fix this
stuff in 5322, and not in some "add-on".  This is *not* "fixing"
anything.  This is *not* saying that any of the "malformed" messages
are now valid.  This is not changing anything at all in 5322.  What
this is doing is acknowledging that senders often violate 5322, and
that those violations are *wrong*.  What it adds is that it also
acknowledges the reality that, as Nathaniel and others have said, we
can't just throw those wrong messages away, and there's some value in
agreeing how to handle them.  This document -- or its final version --
is an attempt to document that agreement.

Agents along the way -- MSAs, MTAs, MDAs, and MUAs -- will make their
guesses and fix-ups, and I do think it's in the best interest of
everyone for us to document less-harmful avenues to take, as well as
roads to hell.  So I support this document for that reason.

On the other hand, Ned's right that the "best" (or least bad, or
whatever) way to handle each situation *is* likely to change over
time, so locking the advice into a BCP might not work well.  It's also
clear that some malformations will do best with complicated heuristics
beyond what'll be recommended here.  The appearance of a non-header
line in a message header is a perfect example of that.  Consider these
two fragments:

1:
   Date: xxx
   Subject: this is a badly
   continued header line
   MIME-Version: 1.0

   This is the body of the message.

2:
   Date: xxx
   MIME-Version: 1.0
   Subject: this is the subject
   I've improperly terminated the header here.
   This is the rest of the body.

The right answer for the two is different.  In (1), we don't want to
assume the "continued header line" is the beginning of the body, and
in (2) we don't want to try to treat the "I've improperly" line as a
continuation of the subject.  A better answer will be to look ahead a
bit to try to re-establish context and make a better guess than can be
done simply by applying a rule.

And yet number three here will break that too:

3:
   Date: xxx
   MIME-Version: 1.0
   Subject: this is the subject
   I've improperly terminated the header here.
   Look: You know it's complicated.

   This is rest of the body.


On the other hand, we know that some of these issues are
straightforward.  Why make everyone figure it all out from scratch?

In the end, the best we can do is to make recommendations to try to
get some consistency.  I think it's worth doing a document like this,
but it's not at all straightforward, and we'll have to be very careful
at every step to make a few things clear:

1. The appearance of these broken messages is BAD, and the BEST thing
is to fix the software that's generating them.

2. Sometimes, it really IS the right thing to
reject/refuse/bounce/whatever-you-want-to-call-it the message.

3. We do or don't have a sense of a reasonably safe guess to make for
this particular malformation.

4. When we do have a reasonably safe guess, here's what it is.

Barry