Re: [apps-discuss] Review of draft-melnikov-smtp-priority-14

Ned Freed <ned.freed@mrochek.com> Fri, 01 June 2012 20:37 UTC

Return-Path: <ned.freed@mrochek.com>
X-Original-To: apps-discuss@ietfa.amsl.com
Delivered-To: apps-discuss@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1246111E80B4; Fri, 1 Jun 2012 13:37:50 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level:
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[AWL=0.000, BAYES_00=-2.599]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uUaSXqjTDHlE; Fri, 1 Jun 2012 13:37:48 -0700 (PDT)
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by ietfa.amsl.com (Postfix) with ESMTP id 5768111E80B0; Fri, 1 Jun 2012 13:37:48 -0700 (PDT)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01OG6915ZXFK002Z6R@mauve.mrochek.com>; Fri, 1 Jun 2012 13:37:44 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01OG54121Q0W0006TF@mauve.mrochek.com>; Fri, 1 Jun 2012 13:37:41 -0700 (PDT)
Message-id: <01OG6914E4MS0006TF@mauve.mrochek.com>
Date: Fri, 01 Jun 2012 13:36:45 -0700
From: Ned Freed <ned.freed@mrochek.com>
In-reply-to: "Your message dated Fri, 01 Jun 2012 20:59:05 +0100" <4FC91F09.7070701@isode.com>
MIME-version: 1.0
Content-type: TEXT/PLAIN; format="flowed"
References: <6.2.5.6.2.20120521130747.0c219ab0@elandnews.com> <4FBDF199.2050300@isode.com> <4FC7204B.8020403@dcrocker.net> <01OG663WXFQA0006TF@mauve.mrochek.com> <4FC91F09.7070701@isode.com>
To: Alexey Melnikov <alexey.melnikov@isode.com>
Cc: draft-melnikov-smtp-priority-13.all@tools.ietf.org, Ned Freed <ned.freed@mrochek.com>, iesg@ietf.org, apps-discuss@ietf.org
Subject: Re: [apps-discuss] Review of draft-melnikov-smtp-priority-14
X-BeenThere: apps-discuss@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion of application-layer protocols <apps-discuss.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/apps-discuss>, <mailto:apps-discuss-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/apps-discuss>
List-Post: <mailto:apps-discuss@ietf.org>
List-Help: <mailto:apps-discuss-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/apps-discuss>, <mailto:apps-discuss-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 01 Jun 2012 20:37:50 -0000

OK, let me take a look at -16 and I'll see what else, if anything, needs to be
said. (I was looking at -15.)

				Ned

> Hi Ned,

> On 01/06/2012 17:18, Ned Freed wrote:
> > A few comments are in order here.
> >
> >> >    particularly important during emergencies for first responders and
> >> >    for environments such as military messaging, where messages have
> >> high
> >> >    operational significance, and the consequences of extraneous delay
> >> >    can be significant.
> >> >
> >> >    In order for an SMTP receiver to be able to send higher priority
> >
> >>     send -> relay
> >
> > Given the imprecise nature of this document overall, it's a stretch to
> > believe
> > that being precise here is really necessary, but if it is, this fixes
> > only part
> > of the problem. The message may have been received via SMTP, SUBMIT, or
> > something else. And the higher *or* *lower* priority may apply to
> > whatever the
> > next step is: SMTP, LMTP, or something else.
> >
> > If you want to be precise here, it needs to say something like:
> >
> >      In order for an MSA, MTA, or MDA to be able to relay, deliver, or
> >      otherwise process higher priority messages first, ...
> Yeah, I think this is less readable than what I have. I am sure readers
> understand the meaning of what is being said.
> >
> >> >    messages first, there needs to be a mechanism to communicate
> >> (during
> >> >    both Message Submission [RFC6409] and Message Transfer
> >> [RFC5321]) the
> >> >    priority of each message.  This specification defines this
> >> mechanism
> >> >    by specification of an SMTP [RFC5321] extension.
> >> >
> >> >    Implementors of this document might also consider implementing
> >> >    [PRIORITY-TUNNELING] which talks about tunneling of Message
> >> Transfer
> >> >    Priority information through Message Transfer Agents (MTAs) that do
> >> >    not support this extension, using a new message header field
> >> >    [RFC5322].
> >
> >> Consider replacing above paragraph with:
> >
> >>     In order to permit end-to-end use of this extension across email
> >> infrastructure that does not support it, a companion tunneling mechanism
> >> is defined in [PRIORITY-TUNNELING] through use of a new message header
> >> field [RFC5322].
> >
> > This is better. These specifications are not just for implementors.
> Yes, already used in -15.
> >> > 2.  Conventions Used in This Document
> >> >
> >> >    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
> >> >    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in
> >> this
> >> >    document are to be interpreted as described in [RFC2119] when they
> >> >    appear in ALL CAPS.  These words also appear in this document in
> >> >    lower case as plain English words, absent their normative meanings.
> >
> >> "when they appear in ALL CAPS".  sigh. In other words, this document is
> >> modifying RFC2119.
> >
> > On the contrary, all this is doing is being specific about the way RFC
> > 2119 has
> > actually been used ever since it was created. I can easily produce a
> > long list
> > of RFCs containing instances of these words in lower case where, even
> > though it
> > wasn't stated, the intent *clearly* was for lower case versions of
> > these words
> > to have their regular meanings.
> >
> > And even if I were to accept the notion that this document specifies a
> > novel
> > derivation of RFC 2119 (the notion that it *modifies* it being patently
> > absurd), that ship has also sailed. I can also easily point to RFCs
> > that do
> > things like specify ALLS CAPS versions of these words have one
> > meaning, Mixed
> > Case another, and lower case yet another.
> >
> >> Does anyone seriously expect this new nuance of usage to be read and
> >> remembered by average readers of this document?
> >
> > The better question is does anyone seriously expect anyone to assume,
> > given
> > abundant evidence to the contrary, that the lower case versions of
> > these words
> > will be taken to have RFC 2119 meaning?
> >
> >> FWIW, it took me about 3 minutes to make the relevant changes, below.
> >> Case-sensitivity in semantics invites comprehension problems here.
> >> Worse, there is absolutely no need or benefit in creating the problem
> >> for this document.  At a minimum, please do not try to modify IETF
> >> document writing policy on the fly.
> >
> > You're the one who is arguing for a modification to the *actual* IETF
> > document
> > writing policy here.

> Agreeing with that.

> >> >    The formal syntax use the Augmented Backus-Naur Form (ABNF)
> >> [RFC5234]
> >> >    notation including the core rules defined in Appendix B of RFC 5234
> >> >    [RFC5234].
> >> >
> >> >    In examples, "C:" and "S:" indicate lines sent by the client and
> >> >    server respectively.  Line breaks that do not start with a new "C:"
> >> >    or "S:" exist for editorial reasons and are not a part of the
> >> >    protocol.
> >> >
> >> >    This document uses the term "priority" specifically in relation to
> >> >    the internal treatment of a message by the server: messages with
> >> >    higher priorities may be given expedited handling, and those with
> >
> >>     may -> can
> >
> >
> >> >    lower priorities may be handled only as resources become available.
> >
> >>     may -> can
> >
> > Again, given the loosey-goosey nature of this document overall, I
> > don't think
> > precision is really required, but if it is, both of these changes are
> > somewhat
> > objectionable.
> >
> > Like it or not, "may" and "can" are *not* synonyms. "Can" is more
> > assertive,
> > and carries with it a flavor that's closer to "should", whereas "may" is
> > expresses a possibilit with no preference.
> >
> > We're talking about prioritization methodology here, specifically
> > expedited
> > handling and deferred processing. There can be overhead associated
> > with both of
> > these, and we should not be encourage the use of these techniques when
> > it isn't
> > absolutely necessary.
> > And yes, I'm being very pedantic here. That's what you're going to get
> > when
> > you elect to go down this path.
> :-)
> >
> >> >
> >> > 3.  Definition of the Priority SMTP Extension
> >> >
> >> >    The Priority SMTP service extension is defined as follows:
> >> >
> >> >
> >> >
> >> >
> >> > Melnikov & Carlberg     Expires November 25, 2012
> >> [Page 3]
> >> > 
> >> > Internet-Draft  Message Transfer Priority SMTP Extension        May
> >> 2012
> >> >
> >> >
> >> >    1.  The textual name of this extension is "Priority Message
> >> >        Handling".
> >> >
> >> >    2.  The EHLO keyword value associated with this extension is "MT-
> >> >        PRIORITY".
> >> >
> >> >    3.  The EHLO keyword has no parameters.  Parameters are reserved
> >> for
> >> >        possible future extensions and MUST be ignored by clients that
> >> >        don't understand them.
> >> >
> >> >    4.  No additional SMTP verbs are defined by this extension.
> >> >
> >> >    5.  One optional parameter ("MT-PRIORITY") is added to the MAIL
> >> FROM
> >> >        command.  The value associated with this parameter is a decimal
> >> >        integer number from -9 to 9 (inclusive) indicating the priority
> >> >        of the email message.  The syntax of the MT-PRIORITY
> >> parameter is
> >> >        described by the <priority-value> ABNF non-terminal defined in
> >> >        Section 6.  Higher numbers mean higher priority.
> >
> >> As a minor point, the form of -9 to 9 seems more complicated than
> >> useful.  Is there a need for 19 values?  I'd think that 0 to 9 would be
> >> sufficient.
> >
> > Well, to be fair, the specification only requires support for six
> > distinct
> > values:
> >
> >   A message priority is a decimal integer in the range from -9 to 9
> >   (inclusive).  SMTP servers compliant with this specification are not
> >   required to support all 19 distinct priority levels (i.e. to treat
> >   each priority value as a separate priority), but they MUST implement
> >   at least the following 6 distinct priority levels: -4, -2, 0, 2, 4,
> >   9.  I.e. an implementation that only supports the 6 priority levels
> >   will internally round up a syntactically valid level that isn't
> >   supported to the next higher supported number.  For example, such an
> >   implementation will treat priority values below and including -4 as
> >   priority -4, priority -3 as priority -2, and all priorities starting
> >   from 5 are treated as priority 9.  An SMTP server MAY support more
> >   than 6 different priority levels.  Decision about which levels to
> >   support in addition to the 6 mentioned above is a local matter.
> >
> > Now, I have no idea what it actually means to "support" six values, but I
> > assume it means that if you only make a processing distinction between
> > five or
> > fewer values total, if you support only one "lower than normal"
> > priority, or
> > you support fewer than three "higher than normal" priorities, you cannot
> > implement this specification. (Note I'm assuming 0 is "normal" here - see
> > below.)
> More or less, yes.
> >
> > Of course there's a trivial way around this: You simply state that, as
> > a matter
> > of implementation policy, you only support X number and here's the
> > mapping.
> > This would allow, say, a system that only does the old X.400 non-urgent,
> > normal, urgent thing to claim support.

> Right. And there is already such a mapping in Appendix B.

> >> For that matter, why are many values needed?  What is the basis for
> >> choosing to have a range of more than a few values?
> >
> > IMO this is one of the rare cases where X.400 was spot-on correct.
> >
> >> This appears to be the only place the provides semantics to the priority
> >> number.  As such, it should be more thorough.  The critical issue in
> >> assigning the number, I think, is trying to get independent originators
> >> to assign numbers according roughly the same criteria.  For example, if
> >> I label "average" messages I send as as 0 and you label them as 1, then
> >> your messages get preferential treatment over mine, although that was
> >> not your intent.  (Assuming you're not trying to game the service...)
> >
> >> This issue goes to the core of the usage model for the feature:  It
> >> really needs an environment tailored to its use, with all participants
> >> not only within the same trust system, but sharing the same priority
> >> assignment model/policy.  It's probably good for this document to permit
> >> a range of assignment models, but should note that an operational
> >> requirement for use of the extension is that an assignment policy be
> >> defined for all participants.
> >
> >> This document might, then, offer a candidate model, to be use by default
> >> and absent one specific to an environment.
> >
> > Well, this goes to the problem I have with the entire document - I
> > have no real
> > idea what the policy associated with these priorities is supposed to
> > be. We
> > have a prioritization capability in our product and I can tell you
> > what it
> > does, but s to whether or not it satisfies the intended target of this
> > specification, I realy couldn't say.

> I hope I clarified this in -16. Also look at Appendix A and Appendix B.

> > More generally, in a modern, high performance MTA that's capable of
> > processing
> > large numbers of messages simultaneously, there's a significant problem
> > actually implementing mechanisms that are guaranteed to preferentially
> > process
> > certain messages.
> >
> > For example, suppose I have several dozen threads/processes/whatever
> > connected
> > up to a given system, all busy transferring large messages. A higher
> > priority
> > message comes in. If I shove this message on the thread that is the
> > closest to
> > being done, there may still be a significant delay. OTOH, if I start a
> > new
> > connection to handle it, what if I hit the limit on the number of
> > connections
> > the remote MTA will accept from me?
> >
> > Put another way, if you assume a fairly strict policy is needed, your
> > point
> > about this only being applicable in an environment that's specifically
> > tailored
> > for it is, if anything, an massive understatement. For this to provide
> > a real
> > priority boost with high reliability it's actually necessary to closely
> > coordinate operational policies across all involved ADMDs. And not to
> > put too
> > fine a point on it, the level of technical acumen needed to actually
> > do this
> > sort of thing is something I've observed to be the exception and not
> > the rule.
> >
> > All that said, I think this actually argues for putting in some
> > statements
> > as to what this document does *not* do. It calls for best effort to
> > provide preferential processing, nothing more. It does *not* provide
> > any sort of guarantees.

> Fair enough, I will add some text about that. Best effort was certainly
> the intent.

> > And if that's not the case, then this needs to be experimental.
> >
> >> >    6.  The maximum length of a MAIL FROM command line is increased
> >> by 15
> >> >        octets by the possible addition of a space, the MT-PRIORITY
> >> >        keyword and value.
> >> >
> >> >    7.  The MT-PRIORITY extension is valid for the submission service
> >> >        [RFC6409] and LMTP [RFC2033].
> >> >
> >> > 4.  Handling of messages received via SMTP
> >> >
> >> >    This section describes how a conforming SMTP server should
> >> handle any
> >> >    messages received via SMTP.
> >> >
> >> > 4.1.  Handling of the MT-PRIORITY parameter by the receiving SMTP
> >> server
> >> >
> >> >    The following rules apply to SMTP transactions in a server that
> >> >    supports the MT-PRIORITY parameter:
> >> >
> >> >    1.  A conforming SMTP server MUST NOT refuse a MAIL FROM command
> >> >        based on the absence of the MT-PRIORITY parameter.
> >
> >> hmmm.   For some operational environments, it will be essential to have
> >> /all/ handling conform to the priority policy model in force for the
> >> environment.  In those cases, the absence of the parameter would be a
> >> violation of the spec.
> >
> > Interesting point. I'm not sure what, if anything, to do about it.
> >
> >> >    2.  If any of the associated <esmtp-value>s (as defined in Section
> >> >        4.1.2 of [RFC5321]) are not syntactically valid, or if there is
> >> >        more than one MT-PRIORITY parameter in a particular MAIL FROM
> >> >        command, the server MUST return an error, for example "501
> >> syntax
> >> >        error in parameter" (with 5.5.2 Enhanced Status Code
> >> [RFC2034]).
> >> >
> >> >    3.  When inserting a Received header field as specified in Section
> >> >        4.4 of [RFC5321], the compliant MTA/MSA SHOULD include the
> >> >        "PRIORITY" clause whose syntax is specified in Section 6.
> >> >
> >> >
> >> >
> >> > Melnikov & Carlberg     Expires November 25, 2012
> >> [Page 4]
> >> > 
> >> > Internet-Draft  Message Transfer Priority SMTP Extension        May
> >> 2012
> >> >
> >> >
> >> >    4.  If the sending SMTP client specified the MT-PRIORITY
> >> parameter to
> >> >        the MAIL FROM command, then the value of this parameter is the
> >> >        message priority.
> >> >
> >> >    5.  If no priority has been determined by the above, the server may
> >
> >>     may -> can
> >
> >
> >> >        use its normal policies to set the message's priority.  By
> >> >        default, each message has priority 0.
> >
> >> ouch.  This chooses a particular model for meaning of the numbers, and
> >> without having defined the model beforehand.
> >
> > I for one was assuming 0 was the normal, default priority. But now
> > that you
> > mention it, at least this needs to be made explicit.

> I believe the document already says that, but I will double check.

> >> That is, I think the problem with this default is that it really is
> >> assuming a particular policy for meaning of the values, namely that
> >> average mail is mid-priority, assuming the numbers have linear
> >> semantics.
> >
> > That sure was what I was assuming.

> Yes.

> >> >    The SMTP server MUST NOT allow upgraded priorities from untrusted
> >> >    (e.g. unauthenticated) or insufficiently trusted sources.  (One
> >> >    example of an "insufficiently trusted source" might be an SMTP
> >> sender
> >> >    which authenticated using SMTP AUTH, but which is not explicitly
> >> >    whitelisted to use the SMTP MT-PRIORITY service.)  The server MAY,
> >
> >> It is good that this issue is raised, but it really requires an earlier,
> >> separate and careful discussion of the need for this extension to be
> >> used within an administered trust environment.  Given such an
> >> introductory section, the above text would be sufficient.
> >
> > Agreed.

> Ok.

> >> In its absence, the reference to a whitelist is based on the very large
> >> assumption that participants in the extension are whitelisted.  This is
> >> not document elsewhere.
> >
> > True.

> Hmm, Ok.

> >> >    however, allow an untrusted source to lower its own message's
> >> >    priorities -- consider, for example, an email marketer that
> >> >    voluntarily sends its marketing messages at low priority.
> >
> >> To beat the point a bit deader:  this assumes a particular policy for
> >> the meaning of the priority values.  Worse, it also appears to
> >> contradict the earlier default of 0, but perhaps I've misunderstood
> >> that.
> >
> > I think that's a misunderstanding.

> I already replied in another message that it is.

> > I generally agree with the remainder of your comments, although I
> > think some
> > of them become less important as long as it's clear this doesn't provide
> > guaranteees, but they still should be addressed.

> I believe I addressed most of them in -15 and -16. I might need to add a
> couple of clarifying sentences as per above.