Re: [ietf-smtp] EHLO domain validation requirement in RFC 5321

Keith Moore <moore@network-heretics.com> Mon, 28 September 2020 01:26 UTC

Return-Path: <moore@network-heretics.com>
X-Original-To: ietf-smtp@ietfa.amsl.com
Delivered-To: ietf-smtp@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C8E5D3A0B4C for <ietf-smtp@ietfa.amsl.com>; Sun, 27 Sep 2020 18:26:18 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.108
X-Spam-Level:
X-Spam-Status: No, score=-2.108 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, NICE_REPLY_A=-0.213, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=messagingengine.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gPTxDKkcP1SH for <ietf-smtp@ietfa.amsl.com>; Sun, 27 Sep 2020 18:26:17 -0700 (PDT)
Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 50AC03A00D2 for <ietf-smtp@ietf.org>; Sun, 27 Sep 2020 18:26:17 -0700 (PDT)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 76E15847 for <ietf-smtp@ietf.org>; Sun, 27 Sep 2020 21:26:16 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Sun, 27 Sep 2020 21:26:16 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=fP6skR XR+fCSo0O52FYDpT4VWIsnnDXhpLi64HbQhCI=; b=RrGlgr4rB2Icuwxa71mlL9 leUGBrIEodHHpax+nAhjmLpY4W0sU6ojZBJhdU8JRtQVDkMTY8LiXP8vRLTfU4CT 1nOB+GMm64lozy1IO30AevSNAbNvDu+v+boZy1EJbOtptRjTWal8Ppu0vekE8sOm nP2VKKjAcNTAmiS8Yl928AskGEJ26hLdL7wOUnIfT21rL4VSmyg0VOdQi3ST0bK/ yaiNoNWcWUUXj/0T9zl5MAjY102OYROnx4Fxw4qj5HEmY+UHi2aDZZ3VJPzeQXRw Ws9+Prb7XRE8ts7Gw1gnTJ8IknRGTQHCmKYiHAthhNS8QDBVIHAcuCMleWC/eZJg ==
X-ME-Sender: <xms:tztxX7tYyGROlpfvgFOA6ATSrsPnDXCJXUhUpR7O2-OCSRGQVjMvDg> <xme:tztxX8flXyfgpCE3mvcCuETF5MFEwBaf6rujg4mKhKsdK2OWXunu6NqNhwQm24G0K __9Myqvtjwq1A>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedujedrvdehgdegjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepuffvfhfhkffffgggjggtsegrtderre dtfeejnecuhfhrohhmpefmvghithhhucfoohhorhgvuceomhhoohhrvgesnhgvthifohhr khdqhhgvrhgvthhitghsrdgtohhmqeenucggtffrrghtthgvrhhnpeevfeetudeigedtle dvvddtudefjeejffdvfeetjeeiueelgfdtgfegtdffkeetudenucfkphepuddtkedrvddv uddrudektddrudehnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilh hfrhhomhepmhhoohhrvgesnhgvthifohhrkhdqhhgvrhgvthhitghsrdgtohhm
X-ME-Proxy: <xmx:tztxX-z3GUmcZi3cDXaZaufkWCPTqZOi3imUrPRdmYhKnDM2ieKgFw> <xmx:tztxX6MDQO_ZPiS9cisl5X8CkPfeernZAP-jsg8-Xb-v4d2xcGhh0A> <xmx:tztxX7_CDVIyixP9ldMYaU1oPWpPySgeQw6jP_2DOUsxzWBqpXNMLw> <xmx:uDtxX5dlyhTQnIeAcxTefG0hraTVOb0I7j96dcDYZQGfSLFKBB9lYw>
Received: from [192.168.1.85] (108-221-180-15.lightspeed.knvltn.sbcglobal.net [108.221.180.15]) by mail.messagingengine.com (Postfix) with ESMTPA id 1CD763064610 for <ietf-smtp@ietf.org>; Sun, 27 Sep 2020 21:26:15 -0400 (EDT)
To: ietf-smtp@ietf.org
References: <402e7482-394f-e077-48b9-c9e47047c49d@dcrocker.net> <55218bbd-b001-ae3f-1afd-e4328ec7ba35@network-heretics.com> <c6b5dae5-d20a-c876-ce5a-86e1d073cf8f@dcrocker.net> <c4a66db8-e3f5-3f6f-acb0-afe01a69a27a@network-heretics.com> <cone.1601250950.437858.35945.1004@monster.email-scan.com> <ac132a1a-ec83-1ec6-dd34-85fd3bba95c5@network-heretics.com> <cone.1601252021.530626.35945.1004@monster.email-scan.com> <6330c607-5ede-4766-1823-5c8be8a9097b@network-heretics.com> <s1Gob6BEOTcfFAg3@highwayman.com>
From: Keith Moore <moore@network-heretics.com>
Message-ID: <3b1279c2-ce25-2c74-cfe4-89fe31075c06@network-heretics.com>
Date: Sun, 27 Sep 2020 21:26:14 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <s1Gob6BEOTcfFAg3@highwayman.com>
Content-Type: multipart/alternative; boundary="------------867398FB58B242FFDD563C9A"
Content-Language: en-US
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf-smtp/wLjWP4IyUZzQDAudjAEKrlwY4bo>
Subject: Re: [ietf-smtp] EHLO domain validation requirement in RFC 5321
X-BeenThere: ietf-smtp@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of issues related to Simple Mail Transfer Protocol \(SMTP\) \[RFC 821, RFC 2821, RFC 5321\]" <ietf-smtp.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf-smtp>, <mailto:ietf-smtp-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf-smtp/>
List-Post: <mailto:ietf-smtp@ietf.org>
List-Help: <mailto:ietf-smtp-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf-smtp>, <mailto:ietf-smtp-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 28 Sep 2020 01:26:19 -0000

On 9/27/20 8:51 PM, Richard Clayton wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> In message <6330c607-5ede-4766-1823-5c8be8a9097b@network-heretics.com>om>,
> Keith Moore <moore@network-heretics.com> writes
>
>> But this is a silly discussion.
> It seems backwards from where it started ... which effectively came down
> to what would be good advice to proffer to a client to ensure that their
> email deliverability improved.  The good advice "why don't you make sure
> that forward and reverse DNS match up and say EHLO in a consistent
> manner" is difficult (as has been pointed out) for some clients to
> follow ... and that's why IMO it ends up as a SHOULD rather than a MUST.

I thought it was about advice to the server which is currently that the 
server MUST NOT refuse to accept a message based on failure of EHLO 
argument verification.

My argument is that EHLO verification is, in the long run, poor practice 
and should not be encouraged by 5321bis /even if it seems like an 
effective spam countermeasure today./   There are various reasons for 
this, but probably the most salient is that the need to support mail 
sending from some IPv6-only networks, and eventually to no public IPv4 
Internet, makes any expectation that the client knows what DNS name will 
result from a PTR lookup of the client's source address as seen by the 
server, to be dubious at best and perhaps completely meaningless.   
Since we expect 5321bis, as a Full Standard, to be long-term stable, it 
doesn't make sense to promote a practice that seems dubious or even 
harmful to mail reliability in the long run.

(I'm open to other ways of rewording the language currently in 5321, as 
long as it takes into account the fact that in a world where 
communications between IPv4 hosts and IPv6 hosts is mediated by NATs 
supplied by carriers, a client no longer has a reasonable expectation of 
knowing what its source IP address will be as seen by the server, or of 
having control over the PTR records in the corresponding DNS zone to 
that source address.   What I don't think is reasonable in the long run 
is to expect all SMTP clients to have direct access (and their own 
static addresses) on both the public IPv4 network and the public IPv6 
network.)

>> I certainly acknowledge that spam
>> filtering is hard, and that the state of the art is to use unreliable
>> heuristics.
> I would disagree ... state of the art is ML clustering algorithms using
> a wide range of signals, where even the people who developed the systems
> find it fairly hard to reliably predict beforehand which of those
> signals are going to be of real significance.

Thanks for the clarification.   I agree that it's useful to distinguish 
ML algorithms from heuristics.

This does bring a tangential question to mind, though:

> Since the only practical way of tuning these algorithms is end-user
> free-back

To me it seems like a very important case where reliability is concerned 
is when a willing sender (who has not previously received mail from this 
recipient) tries to send their first email to a willing recipient.    If 
that doesn't work, the sender and recipient may simply give up, but 
whatever they do, they're shooting in the dark.

If end-user feedback is already difficult to get (and I acknowledge that 
it is), it seems like measuring this critical case and tuning ML 
algorithms based on this case would be extremely difficult.

Keith