Re: [apps-discuss] Review of draft-melnikov-smtp-priority-14

Ned Freed <ned.freed@mrochek.com> Fri, 01 June 2012 19:13 UTC

Return-Path: <ned.freed@mrochek.com>
X-Original-To: apps-discuss@ietfa.amsl.com
Delivered-To: apps-discuss@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B516311E80B4; Fri, 1 Jun 2012 12:13:45 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level:
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dUT9G0JtiPAe; Fri, 1 Jun 2012 12:13:44 -0700 (PDT)
Received: from mauve.mrochek.com (mauve.mrochek.com [66.59.230.40]) by ietfa.amsl.com (Postfix) with ESMTP id 1EFB311E80A3; Fri, 1 Jun 2012 12:13:44 -0700 (PDT)
Received: from dkim-sign.mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01OG663YE9AO002X1O@mauve.mrochek.com>; Fri, 1 Jun 2012 12:13:41 -0700 (PDT)
Received: from mauve.mrochek.com by mauve.mrochek.com (PMDF V6.1-1 #35243) id <01OG54121Q0W0006TF@mauve.mrochek.com>; Fri, 1 Jun 2012 12:13:39 -0700 (PDT)
Message-id: <01OG663WXFQA0006TF@mauve.mrochek.com>
Date: Fri, 01 Jun 2012 09:18:43 -0700
From: Ned Freed <ned.freed@mrochek.com>
In-reply-to: "Your message dated Thu, 31 May 2012 09:39:55 +0200" <4FC7204B.8020403@dcrocker.net>
MIME-version: 1.0
Content-type: TEXT/PLAIN; Format="flowed"
References: <6.2.5.6.2.20120521130747.0c219ab0@elandnews.com> <4FBDF199.2050300@isode.com> <4FC7204B.8020403@dcrocker.net>
To: Dave Crocker <dhc@dcrocker.net>
Cc: draft-melnikov-smtp-priority-13.all@tools.ietf.org, iesg@ietf.org, apps-discuss@ietf.org
Subject: Re: [apps-discuss] Review of draft-melnikov-smtp-priority-14
X-BeenThere: apps-discuss@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: General discussion of application-layer protocols <apps-discuss.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/apps-discuss>, <mailto:apps-discuss-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/apps-discuss>
List-Post: <mailto:apps-discuss@ietf.org>
List-Help: <mailto:apps-discuss-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/apps-discuss>, <mailto:apps-discuss-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 01 Jun 2012 19:13:45 -0000

A few comments are in order here.

> >    particularly important during emergencies for first responders and
> >    for environments such as military messaging, where messages have high
> >    operational significance, and the consequences of extraneous delay
> >    can be significant.
> >
> >    In order for an SMTP receiver to be able to send higher priority

>     send -> relay

Given the imprecise nature of this document overall, it's a stretch to believe
that being precise here is really necessary, but if it is, this fixes only part
of the problem. The message may have been received via SMTP, SUBMIT, or
something else. And the higher *or* *lower* priority may apply to whatever the
next step is: SMTP, LMTP, or something else.

If you want to be precise here, it needs to say something like:

      In order for an MSA, MTA, or MDA to be able to relay, deliver, or
      otherwise process higher priority messages first, ...

> >    messages first, there needs to be a mechanism to communicate (during
> >    both Message Submission [RFC6409] and Message Transfer [RFC5321]) the
> >    priority of each message.  This specification defines this mechanism
> >    by specification of an SMTP [RFC5321] extension.
> >
> >    Implementors of this document might also consider implementing
> >    [PRIORITY-TUNNELING] which talks about tunneling of Message Transfer
> >    Priority information through Message Transfer Agents (MTAs) that do
> >    not support this extension, using a new message header field
> >    [RFC5322].

> Consider replacing above paragraph with:

>     In order to permit end-to-end use of this extension across email
> infrastructure that does not support it, a companion tunneling mechanism
> is defined in [PRIORITY-TUNNELING] through use of a new message header
> field [RFC5322].

This is better. These specifications are not just for implementors.

> > 2.  Conventions Used in This Document
> >
> >    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
> >    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
> >    document are to be interpreted as described in [RFC2119] when they
> >    appear in ALL CAPS.  These words also appear in this document in
> >    lower case as plain English words, absent their normative meanings.

> "when they appear in ALL CAPS".  sigh. In other words, this document is
> modifying RFC2119.

On the contrary, all this is doing is being specific about the way RFC 2119 has
actually been used ever since it was created. I can easily produce a long list
of RFCs containing instances of these words in lower case where, even though it
wasn't stated, the intent *clearly* was for lower case versions of these words
to have their regular meanings.

And even if I were to accept the notion that this document specifies a novel
derivation of RFC 2119 (the notion that it *modifies* it being patently
absurd), that ship has also sailed. I can also easily point to RFCs that do
things like specify ALLS CAPS versions of these words have one meaning, Mixed
Case another, and lower case yet another.

> Does anyone seriously expect this new nuance of usage to be read and
> remembered by average readers of this document?

The better question is does anyone seriously expect anyone to assume, given
abundant evidence to the contrary, that the lower case versions of these words
will be taken to have RFC 2119 meaning?

> FWIW, it took me about 3 minutes to make the relevant changes, below.
> Case-sensitivity in semantics invites comprehension problems here.
> Worse, there is absolutely no need or benefit in creating the problem
> for this document.  At a minimum, please do not try to modify IETF
> document writing policy on the fly.

You're the one who is arguing for a modification to the *actual* IETF document
writing policy here.

> >    The formal syntax use the Augmented Backus-Naur Form (ABNF) [RFC5234]
> >    notation including the core rules defined in Appendix B of RFC 5234
> >    [RFC5234].
> >
> >    In examples, "C:" and "S:" indicate lines sent by the client and
> >    server respectively.  Line breaks that do not start with a new "C:"
> >    or "S:" exist for editorial reasons and are not a part of the
> >    protocol.
> >
> >    This document uses the term "priority" specifically in relation to
> >    the internal treatment of a message by the server: messages with
> >    higher priorities may be given expedited handling, and those with

>     may -> can


> >    lower priorities may be handled only as resources become available.

>     may -> can

Again, given the loosey-goosey nature of this document overall, I don't think
precision is really required, but if it is, both of these changes are somewhat
objectionable.

Like it or not, "may" and "can" are *not* synonyms. "Can" is more assertive,
and carries with it a flavor that's closer to "should", whereas "may" is
expresses a possibilit with no preference.

We're talking about prioritization methodology here, specifically expedited
handling and deferred processing. There can be overhead associated with both of
these, and we should not be encourage the use of these techniques when it isn't
absolutely necessary. 

And yes, I'm being very pedantic here. That's what you're going to get when
you elect to go down this path.

> >
> > 3.  Definition of the Priority SMTP Extension
> >
> >    The Priority SMTP service extension is defined as follows:
> >
> >
> >
> >
> > Melnikov & Carlberg     Expires November 25, 2012               [Page 3]
> > 
> > Internet-Draft  Message Transfer Priority SMTP Extension        May 2012
> >
> >
> >    1.  The textual name of this extension is "Priority Message
> >        Handling".
> >
> >    2.  The EHLO keyword value associated with this extension is "MT-
> >        PRIORITY".
> >
> >    3.  The EHLO keyword has no parameters.  Parameters are reserved for
> >        possible future extensions and MUST be ignored by clients that
> >        don't understand them.
> >
> >    4.  No additional SMTP verbs are defined by this extension.
> >
> >    5.  One optional parameter ("MT-PRIORITY") is added to the MAIL FROM
> >        command.  The value associated with this parameter is a decimal
> >        integer number from -9 to 9 (inclusive) indicating the priority
> >        of the email message.  The syntax of the MT-PRIORITY parameter is
> >        described by the <priority-value> ABNF non-terminal defined in
> >        Section 6.  Higher numbers mean higher priority.

> As a minor point, the form of -9 to 9 seems more complicated than
> useful.  Is there a need for 19 values?  I'd think that 0 to 9 would be
> sufficient.

Well, to be fair, the specification only requires support for six distinct
values:

   A message priority is a decimal integer in the range from -9 to 9
   (inclusive).  SMTP servers compliant with this specification are not
   required to support all 19 distinct priority levels (i.e. to treat
   each priority value as a separate priority), but they MUST implement
   at least the following 6 distinct priority levels: -4, -2, 0, 2, 4,
   9.  I.e. an implementation that only supports the 6 priority levels
   will internally round up a syntactically valid level that isn't
   supported to the next higher supported number.  For example, such an
   implementation will treat priority values below and including -4 as
   priority -4, priority -3 as priority -2, and all priorities starting
   from 5 are treated as priority 9.  An SMTP server MAY support more
   than 6 different priority levels.  Decision about which levels to
   support in addition to the 6 mentioned above is a local matter.

Now, I have no idea what it actually means to "support" six values, but I
assume it means that if you only make a processing distinction between five or
fewer values total, if you support only one "lower than normal" priority, or
you support fewer than three "higher than normal" priorities, you cannot
implement this specification. (Note I'm assuming 0 is "normal" here - see
below.)

Of course there's a trivial way around this: You simply state that, as a matter
of implementation policy, you only support X number and here's the mapping.
This would allow, say, a system that only does the old X.400 non-urgent,
normal, urgent thing to claim support.

> For that matter, why are many values needed?  What is the basis for
> choosing to have a range of more than a few values?

IMO this is one of the rare cases where X.400 was spot-on correct.

> This appears to be the only place the provides semantics to the priority
> number.  As such, it should be more thorough.  The critical issue in
> assigning the number, I think, is trying to get independent originators
> to assign numbers according roughly the same criteria.  For example, if
> I label "average" messages I send as as 0 and you label them as 1, then
> your messages get preferential treatment over mine, although that was
> not your intent.  (Assuming you're not trying to game the service...)

> This issue goes to the core of the usage model for the feature:  It
> really needs an environment tailored to its use, with all participants
> not only within the same trust system, but sharing the same priority
> assignment model/policy.  It's probably good for this document to permit
> a range of assignment models, but should note that an operational
> requirement for use of the extension is that an assignment policy be
> defined for all participants.

> This document might, then, offer a candidate model, to be use by default
> and absent one specific to an environment.

Well, this goes to the problem I have with the entire document - I have no real
idea what the policy associated with these priorities is supposed to be. We
have a prioritization capability in our product and I can tell you what it
does, but s to whether or not it satisfies the intended target of this
specification, I realy couldn't say.

More generally, in a modern, high performance MTA that's capable of processing
large numbers of messages simultaneously, there's a significant problem
actually implementing mechanisms that are guaranteed to preferentially process
certain messages.

For example, suppose I have several dozen threads/processes/whatever connected
up to a given system, all busy transferring large messages. A higher priority
message comes in. If I shove this message on the thread that is the closest to
being done, there may still be a significant delay. OTOH, if I start a new
connection to handle it, what if I hit the limit on the number of connections
the remote MTA will accept from me?

Put another way, if you assume a fairly strict policy is needed, your point
about this only being applicable in an environment that's specifically tailored
for it is, if anything, an massive understatement. For this to provide a real
priority boost with high reliability it's actually necessary to closely
coordinate operational policies across all involved ADMDs. And not to put too
fine a point on it, the level of technical acumen needed to actually do this
sort of thing is something I've observed to be the exception and not the rule.

All that said, I think this actually argues for putting in some statements
as to what this document does *not* do. It calls for best effort to 
provide preferential processing, nothing more. It does *not* provide
any sort of guarantees. 

And if that's not the case, then this needs to be experimental.

> >    6.  The maximum length of a MAIL FROM command line is increased by 15
> >        octets by the possible addition of a space, the MT-PRIORITY
> >        keyword and value.
> >
> >    7.  The MT-PRIORITY extension is valid for the submission service
> >        [RFC6409] and LMTP [RFC2033].
> >
> > 4.  Handling of messages received via SMTP
> >
> >    This section describes how a conforming SMTP server should handle any
> >    messages received via SMTP.
> >
> > 4.1.  Handling of the MT-PRIORITY parameter by the receiving SMTP server
> >
> >    The following rules apply to SMTP transactions in a server that
> >    supports the MT-PRIORITY parameter:
> >
> >    1.  A conforming SMTP server MUST NOT refuse a MAIL FROM command
> >        based on the absence of the MT-PRIORITY parameter.

> hmmm.   For some operational environments, it will be essential to have
> /all/ handling conform to the priority policy model in force for the
> environment.  In those cases, the absence of the parameter would be a
> violation of the spec.

Interesting point. I'm not sure what, if anything, to do about it.

> >    2.  If any of the associated <esmtp-value>s (as defined in Section
> >        4.1.2 of [RFC5321]) are not syntactically valid, or if there is
> >        more than one MT-PRIORITY parameter in a particular MAIL FROM
> >        command, the server MUST return an error, for example "501 syntax
> >        error in parameter" (with 5.5.2 Enhanced Status Code [RFC2034]).
> >
> >    3.  When inserting a Received header field as specified in Section
> >        4.4 of [RFC5321], the compliant MTA/MSA SHOULD include the
> >        "PRIORITY" clause whose syntax is specified in Section 6.
> >
> >
> >
> > Melnikov & Carlberg     Expires November 25, 2012               [Page 4]
> > 
> > Internet-Draft  Message Transfer Priority SMTP Extension        May 2012
> >
> >
> >    4.  If the sending SMTP client specified the MT-PRIORITY parameter to
> >        the MAIL FROM command, then the value of this parameter is the
> >        message priority.
> >
> >    5.  If no priority has been determined by the above, the server may

>     may -> can


> >        use its normal policies to set the message's priority.  By
> >        default, each message has priority 0.

> ouch.  This chooses a particular model for meaning of the numbers, and
> without having defined the model beforehand.

I for one was assuming 0 was the normal, default priority. But now that you
mention it, at least this needs to be made explicit.

> That is, I think the problem with this default is that it really is
> assuming a particular policy for meaning of the values, namely that
> average mail is mid-priority, assuming the numbers have linear semantics.

That sure was what I was assuming.

> >    The SMTP server MUST NOT allow upgraded priorities from untrusted
> >    (e.g. unauthenticated) or insufficiently trusted sources.  (One
> >    example of an "insufficiently trusted source" might be an SMTP sender
> >    which authenticated using SMTP AUTH, but which is not explicitly
> >    whitelisted to use the SMTP MT-PRIORITY service.)  The server MAY,

> It is good that this issue is raised, but it really requires an earlier,
> separate and careful discussion of the need for this extension to be
> used within an administered trust environment.  Given such an
> introductory section, the above text would be sufficient.

Agreed.

> In its absence, the reference to a whitelist is based on the very large
> assumption that participants in the extension are whitelisted.  This is
> not document elsewhere.

True.

> >    however, allow an untrusted source to lower its own message's
> >    priorities -- consider, for example, an email marketer that
> >    voluntarily sends its marketing messages at low priority.

> To beat the point a bit deader:  this assumes a particular policy for
> the meaning of the priority values.  Worse, it also appears to
> contradict the earlier default of 0, but perhaps I've misunderstood that.

I think that's a misunderstanding.

I generally agree with the remainder of your comments, although I think some
of them become less important as long as it's clear this doesn't provide
guaranteees, but they still should be addressed.

				Ned