Re: [apps-discuss] Comments on Malformed Message BCP draft

Ned Freed <> Mon, 18 April 2011 17:01 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 50846E0694 for <>; Mon, 18 Apr 2011 10:01:16 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id P9jUPqMRkONr for <>; Mon, 18 Apr 2011 10:01:15 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id 27E3DE0684 for <>; Mon, 18 Apr 2011 10:01:14 -0700 (PDT)
Received: from by (PMDF V6.1-1 #35243) id <> for; Mon, 18 Apr 2011 10:01:11 -0700 (PDT)
Received: from by (PMDF V6.1-1 #35243) id <>; Mon, 18 Apr 2011 10:01:09 -0700 (PDT)
Message-id: <>
Date: Mon, 18 Apr 2011 09:45:34 -0700 (PDT)
From: Ned Freed <>
In-reply-to: "Your message dated Mon, 18 Apr 2011 13:08:13 +0100" <>
MIME-version: 1.0
Content-type: TEXT/PLAIN
References: <> <> <> <> <3111.1302886222.968467@puncture> <>
To: Tony Finch <>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;; s=mauve; t=1303145635; bh=SeYSlScem8rghOvTqdOxgFlF1605VI4ApaoLqRomz+g=; h=Cc:Message-id:Date:From:Subject:In-reply-to:MIME-version: Content-type:References:To; b=JKmUsLpkwMcpDrXE3P35UoyE4+h18GlsyjfEiFhbFvHW9MAKq0dtebmoizT5KaZpy VFd4UBXZrYBpY8CygFzrXWM0wPlOuhXiDyDyU+Srp9ixPut++AqB0MWpjSYgafSLb/ 2OHzf71/wVH28u/ew4XfI/H1DdFFjR/9VaOc5O0M=
Cc: ietf-822 <>, General discussion of application-layer protocols <>
Subject: Re: [apps-discuss] Comments on Malformed Message BCP draft
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion of application-layer protocols <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 18 Apr 2011 17:01:16 -0000

> Dave Cridland <> wrote:
> >
> > It may be that the discussion suggests rejecting, in which case I suggest the
> > document should clearly explain why, and what the implications of not doing so
> > are, beyond "it makes some problems harder to diagnose".

> It does not make sense to have a uniform policy for dealing with corrupt
> messages. Some kinds of corruption are caused by common legitimate
> software, in which case you will want to treat it leniently; others are
> caused by malware or rare kinds of incompetence, in which case it makes
> more sense to reject. You can only determine which is which based on
> operational experience, and the least-worst response changes from time to
> time.

Bingo. This is precisely the concern I have with this effort - the IETF
consensus process operates in a time scale that will make it very difficult to
capture a sensible set of recommendations in a specification. And even if we do
manage to capture something useful, it will need to be updated regularly to
remain useful. Are we prepared to do that?

And it isn't a binary choice between rejection and fixup either. What sort of
fixup makes sense can change over time.

Consider the invalid header line case. There was a period a few years ago
when it seemed like half the email agents out there couldn't consistently
fold header lines. As a result the strategy of accepting such messages
and not terminating the header seemed to work best.

But as this issue was starting to disappear, a new one, where some group of
agents decided that the blank line between the header and body was  superfluous
and could be omitted, started to show up. A strategy of treating an invalid
header line as a stopping point works best in this case. But who's to say
what's going to happen next month?

This sort of behavior shifting over time is especially prevalent in certain
problem areas like CJK charset usage and labeling.

Another aspect to this that hasn't been discussed yet is where some
agent can only tolerate a subset of what the standards allow. Is it OK
to reformat messages to avoid using legitimate but neverthless problematic

My personal favorite of these was the one where, several years back, a very
popular product refused to accept MIME messages where the 
content-tranfer-encoding field appeared prior to the content-type in the
message header.