Re: RFC 5321bis / 2821ter

[David MacQuigg <[e-mail]>]

Hi John,

Good discussion.  I appreciate your time and effort on this.  I'll try to avoid sloppy statements that generate unnecessary discussion.

At 09:47 AM 1/24/2009 -0500, John C Klensin wrote:
>--On Saturday, January 24, 2009 6:23 -0700 David MacQuigg wrote:
>
>>> 
>>>> ...
>>>> I have yet to see any sensible use-case that would be
>>>> complicated by a requirement that the HELO/EHLO name end in a
>>>> registered, verifiable domain name.
>>>> ...
>>> 
>>> David,
>>> 
>>> Today, under 5321, EHLO clearly requires a fully-qualified
>>> domain name that can be resolved unless the host doesn't know
>>> its name and has to use an IP address, a situation that the
>>> spec doesn't encourage.  My reading of 5321 (but not
>>> necessarily 821) says that HELO does too.
>> 
>> Perhaps it is just a matter of emphasis.  As I read the spec
>> it *does* seem to encourage using HELO/EHLO names that are
>> useless for authentication.  For example, the language in
>> 4.1.4 "if the verification fails, the server MUST NOT refuse
>> to accept a message" puts the burden on the receiver to deal
>> with mis-configured transmitters.  Receivers are expected to
>> accept mail from transmitters that say "HELO this is Jupiter".
>
>Sigh.  Read more carefully...
>
>(1) It says, quite deliberately, "MUST NOT refuse to accept a
>message on that basis".  That is _very_ different from the
>excerpt you quote.

Sorry for being sloppy in implying that section 4.1.4 requires that receivers accept "HELO this is Jupiter".  The connection between  cause and effect is less direct.  I should have said that this requirement (no rejection when verification fails) makes the HELO name unreliable for authentication.  It means that receivers MUST accept mail from transmitters that provide a fraudulent ID, i.e. one that does not correspond to the IP address of the TCP connection.  The fact that receivers can't rely on the HELO name leads to little use of that name for authentication, and attempts to find some other name (MAIL FROM) that will more reliably ID the transmitter.  This may be the reason so many transmitters provide useless HELO names like Jupiter.

The Jupiter example came from a long ago discussion in the SPF group, in which the conclusion was we have to accept this, even if it is obviously invalid.  The reason was that some legitimate senders have HELO names like this, and we would be rejecting legitimate mail.

Why not say:

  An SMTP server MAY reject a mail session if a domain name argument in
  the EHLO command does not actually correspond to the IP address of the
  client.  Clients that are unable to establish this correspondence MUST
  NOT provide an invalid domain name argument.

How the correspondence is established should be left to another spec (CSV, SPF, whatever).  There is no change in the syntax of the HELO command, no change that would affect receivers following the old MUST NOT rule, just a change in emphasis that should have a beneficial effect on the reliability of HELO names as IDs.

I see very little downside.  The examples of the digital camera and the roaming laptop don't justify degrading the reliability of HELO IDs for everyone else.  Maybe I'm not understanding these examples.  See below for discussion of the roaming laptop.


>(2) The place where the names get "used", i.e., the SMTP client,
>is covered by the strongest type of statement that we can make
>in a standard:
>
>        "The SMTP client MUST, if possible, ensure that the
>        domain parameter to the EHLO command is a primary host
>        name as specified for this command in Section 2.3.5."

Then the strength of this statement is nullified with a MUST NOT in the next paragraph.

>The key case that justifies "if possible" is mentioned in the
>next sentence, but there are others.  2.3.5 is very explicit,
>arguably to the point of tedium -- the string must be
>fully-qualified (no local names or incomplete strings permitted)
>and it must be resolvable (by any plausible reading "in the
>public DNS" -- and I wrote myself a note in December to ask
>where things had gotten weird enough that we had to add that
>phrase).
>
>(3) "HELO this is Jupiter" fails the syntax rule
>
>   helo = "HELO" SP Domain CRLF
>
>in several ways.  If you receive such a string, you are
>perfectly justified, conformant, and in line with the spec to
>reject it with a 501 code.   Indeed, you would be outside the
>spec if you did not do so.  You wouldn't even have to figure out
>whether "this" was a resolvable FQDN or not.

The prohibition against an unauthorized domain name should be just as strong as the prohibition against bad syntax.  The fix is equally simple in both cases, even for digital cameras with built in transmitters.

>My imagination doesn't stretch nearly far enough to get from
>those three points to "seems to encourage using HELO/EHLO names
>that are useless for authentication".
>
>That prohibition in 4.1.4 goes back to RFC 1123, which predates
>2821 by nearly a dozen years.  There are a number of reasons for
>it.  Some of them are subtle, some of them involve cases that we
>would now prefer to see handled via submission servers and prior
>agreements.  But let me give you a contemporary example that
>might make the situation clear.   Suppose there is a laptop with
>a full service, conforming, SMTP client on it.  It sends mail,
>manages queues, etc. -- no submission server involved.  It has a
>permanent name, say roaming.example.com.  However, it is not
>connected at a fixed point to the public Internet.  It roams,
>and may have different addresses on different networks on
>different days.  If its owner is being really careful she even
>uses dynamic DNS to be sure that, at any time the host is
>online, its domain names points to its actual address.   Or, if
>she knows that the host is used at only a half-dozen locations
>and has a usual address at each, she might set up static DNS
>entries that list each of them as its address.     Now suppose a
>message is sent to you from that host, after which it is shut
>down.  When it reappears on the network a few hours later, it is
>at a different address.  If you resolve the domain name at that
>time, you get a valid address for the host, but it isn't the
>same address the host had when it initiated sending the message.
>Do you really want to reject the message?  If you do, please
>don't justify the decision on the basis that that you have done
>a careful and adequate job of authentication -- _nothing_ you
>can do with HELO/EHLO as they are defined in 5321 is going to
>get you to a point at which that is possible.

Let us call this the "roaming laptop" problem.  There are several solutions.
1) The laptop user can leave her setup as is, relaying through the same server as when she is in her home town.
2) She can publish a DNS record authorizing a collection of IP addresses, or even entire blocks that she might be using.
3) She can use an address literal, saying in effect - please accept this session request without a HELO check.

For the typical "road warrior" running Windows and Outlook, {1) is by far the most likely solution.  Even if she changes the default settings for SMTP server, she will still need to keep the same POP or IMAP server, and the same password to access her home mailbox.  Same for the other popular email programs.  They are just not expecting to use a local SMTP server.

Solution 2 is not yet available, but if senders were to start taking HELO names seriously, a solution would quickly develop.  It would be something like SPF, without the risky or ambiguous terms that have led to lack of universal acceptance.

Solution 3 is no worse than the status quo.  There is no degradation in service for those using this method, if others chose a more reliable method.


>> The purpose of changes should be to avoid lost mail, as
>> opposed to rejected mail, which doesn't have any damaging
>> effects on reliability.  SMTP REJECT should be encouraged as
>> the proper response to doubtful requests.  (Sorry, we don't
>> accept mail from unidentified transmitters.)
>
>2821/5321 were carefully written to give you the option of
>making that decision... or the decision to not accept mail from
>even-numbered IP addresses for that matter.  I personally cannot
>recommend it unless you have a much better notion of
>"identified" than anything you can get out of HELO/EHLO.  I'd
>also suggest to you that the typical user who is waiting for a
>message to arrive is not likely to distinguish between
>"rejected" and "lost".  Unless the former causes the sender to
>get on the phone, the only issue is "didn't arrive".  Your users
>and experience may differ.

The difference between rejected and lost is critical to the reliability of our email system.

>>> So what are you asking for that isn't there already?
>>> Elimination of the IP address literal option?  Some criteria
>>> for "verifiable" that would presumably lie well outside the
>>> scope of SMTP?
>> 
>> If you can deprecate HELO, surely you can do the same for
>> address literals.
>
>Independent of some practical issues and the fact that address
>literals can sometimes actually provide useful information, that
>would accomplish what?

Eliminate what seems to be useless information, and motivate senders to provide useful information instead.  What is in an address literal that we can't get more reliably from the TCP address?

>    You can already make a policy decision
>to reject any message that starts with EHLO and an address
>literal; doing so would be consistent with your position that
>false rejects are ok as long as one doesn't drop the message.
>
>I can't recommend that we try to dig into the question in the
>standard, but consider whether you would prefer:
>
>   EHLO [#.#.#.#]
>
>where "#.#.#.#" is a valid, public, IP address that might even
>be static and stable and have a legitimate reverse mapping to a
>host name,     and
>
>   EHLO comment.example.org
>
>where "comment.example.org" is a valid, registered, resolvable
>FQDN but the only information at its DNS node is
>
>     IN TXT This is a mobile host and I can't really tell you
>what or where it is without compromising my privacy.
>
>You don't have much "authentication" information either way; you
>might be able to extract as much or more contact information
>from the first.

I would split the mail into two streams.  99% of legitimate messages are sent from transmitters with verifiable HELO IDs.  1% have an address literal, or some more explicit way of saying "I can't ID just yet, please accept this mail session anyway."

Idea: The HELO name could actually specify what authentication method the sender offers in lieu of a robust HELO check.
    EHLO  _dkim.hostname.example.com

>>> Do keep in mind that we know from experience that the bad guys
>>> have absolutely no trouble obtaining perfectly valid domain
>>> names.
>> 
>> I agree that reputation requirements should be outside the
>> scope of 5321.  Same for the details of any particular
>> authentication method.  What I am suggesting is not that level
>> of detail.
>> 
>> I know you can't turn back the clock and make radical changes
>> to SMTP, but in revising the spec, we should do everything we
>> can to make up for the original deficiency in the HELO command
>> and the lack of attention to security issues.
>
>I think that, had you made this point at the time (others did,
>more or less), the DRUMS WG would have told you that the changes
>from the language of 821 about that field to the requirement for
>a resolvable FQDN _and_ the addition of the IP address literal
>option for SMTP clients that didn't reliably know their own
>names, was "everything we can to make up for...".  It seems to
>me that you are looking for something else entirely, either to
>push the Domain name that is expected to appear in that field
>beyond its plausible limits (unless you are willing to start
>prohibiting some legitimate cases) or to replace the Domain name
>with an authentication token or identifier.  And that is why I
>suggested in an earlier note that, if you or others really think
>that would be useful, you start writing an extension spec for an
>authenticator that would supply the information you want, rather
>than trying to overload it onto EHLO.  Of course, you'd probably
>then have to adopt a policy of rejecting (or assigning a very
>low score to) any message that came from a source that didn't
>adopt and support your extension, but that is how it goes with
>twenty-odd years of installed base.

Writing a new draft now would be a waste of time.  A discussion like this would be the first step, and I am not seeing in this group any desire to change the status quo, or even any recognition that there is a problem.  That is the major hurdle with all attempts to fix the identity fraud problem on the Internet.  The status quo is profitable for many, and at least acceptable to most others.  It's not a technical problem, but a problem of motivation and organization.  The technology is there if we really want it.

I appreciate your time also, and I will try to keep this discussion focused on the HELO ID.

-- Dave