[ietf-smtp] SMTP server reply extensions

Timo Sirainen <timo@sirainen.com> Wed, 08 April 2020 14:00 UTC

Return-Path: <timo@sirainen.com>
X-Original-To: ietf-smtp@ietfa.amsl.com
Delivered-To: ietf-smtp@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2FB7A3A0970 for <ietf-smtp@ietfa.amsl.com>; Wed, 8 Apr 2020 07:00:32 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tBuJZ-b0t2GT for <ietf-smtp@ietfa.amsl.com>; Wed, 8 Apr 2020 07:00:30 -0700 (PDT)
Received: from sirainen.com (mail.sirainen.com [94.237.26.55]) by ietfa.amsl.com (Postfix) with ESMTP id B27343A0C68 for <ietf-smtp@ietf.org>; Wed, 8 Apr 2020 07:00:29 -0700 (PDT)
Received: from [172.20.10.2] (unknown [85.76.150.94]) by sirainen.com (Postfix) with ESMTPSA id D51212B3C8B for <ietf-smtp@ietf.org>; Wed, 8 Apr 2020 14:00:27 +0000 (UTC)
From: Timo Sirainen <timo@sirainen.com>
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3608.60.0.2.5\))
Message-Id: <8CF389F4-7BD3-44D0-85F4-91E66120A59B@sirainen.com>
Date: Wed, 8 Apr 2020 17:00:27 +0300
To: ietf-smtp@ietf.org
X-Mailer: Apple Mail (2.3608.60.0.2.5)
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf-smtp/eXgEDcHMXbKvjtQUBqFjn_lSH7I>
Subject: [ietf-smtp] SMTP server reply extensions
X-BeenThere: ietf-smtp@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of issues related to Simple Mail Transfer Protocol \(SMTP\) \[RFC 821, RFC 2821, RFC 5321\]" <ietf-smtp.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf-smtp>, <mailto:ietf-smtp-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf-smtp/>
List-Post: <mailto:ietf-smtp@ietf.org>
List-Help: <mailto:ietf-smtp-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf-smtp>, <mailto:ietf-smtp-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 08 Apr 2020 14:00:32 -0000

Hi,

I've recently found several reasons to add some kind of extended response metadata to SMTP/LMTP's RCPT and DATA replies. Similar to IMAP's resp-codes. Since I'm not so familiar with SMTP protocol, I was thinking I'd ask here if anyone happens to know already existing solutions for this or has some recommendations? Maybe there is already some RFC solving a similar problem?

IMAP uses "[resp-code parameters]" strings, but this doesn't seem to be used with SMTP protocol. Another option is to invent new SMTP enhanced code numbers, but especially for private extensions it seems a bit dangerous since there could be numbering conflicts. Also I'm not sure if SMTP clients would become confused when they see codes they don't understand? Also another problem with these numeric codes is how to add multiple different replies, especially since multi-line replies can't have different enhanced codes. But perhaps a single new extensible enhanced code could be invented? Something like:

RCPT TO:<user@example.com>
250-2.1.100 key1 metadata1
250-2.1.100 key2 metadata2
250 2.1.100 OK with some extra metadata
DATA
354 OK
.
250-2.0.100 key1 metadata1
250-2.0.100 key2 metadata2
250 2.0.100 <user@example.com> Saved and returned some metadata

To maybe clarify things, below are the 3 use cases I have so far. The first two are likely to be completely internal to Dovecot and I'm not trying make them standard.

1. IMAP-like login referrals (RFC 2221)

This RFC isn't really used nowadays, but I'm now thinking of using referrals internally in Dovecot clusters. Dovecot backend could send a referral and Dovecot proxy would reconnect to that referral instead. But the same thing is needed also for LMTP, which means that it would have to be sent in a RCPT TO reply since the redirection can be different for different users. Perhaps something like:

RCPT TO:<user@example.com>
250 2.1.5 OK
RCPT TO:<user2@example2.com>
450 4.4.4 [REFERRAL smtp://backend2/] Redirected
RCPT TO:<user3@example3.com>
450 4.4.4 [REFERRAL smtp://backend3/] Redirected

2. Single instance storage for deliveries between backends in a cluster

The idea is that if backends share a storage (NFS, object storage) and there are multiple recipients, the proxy could deliver mail first to backend1, get back some metadata and use that to deliver mail to backend2, which would use the metadata to store a copy of the original mail. So for example:

RCPT TO:<user@example.com>
250 2.1.5 OK
DATA
354 OK
.
250 2.0.0 <user@example.com> [MAILDATA /nfs/mail/user1@example.com/INBOX/Maildir/file1] Saved

(In reality the path would likely be a base64 encoded blob)

And then in backend2:
RCPT TO:<user2@example2.com> MAILDATA /nfs/mail/user1@example.com/INBOX/Maildir/file1

Now backend2 can create a hard link from user1@example.com's mail to user2@example2.com's INBOX, saving some disk space. (Obviously only if the mails would be identical.)

3. We published a "submission token" draft a while ago: https://www.ietf.org/archive/id/draft-slusarz-extra-smtp-submission-token-00.txt

It's going to need a bunch of changes, but that draft is also trying to solve this metadata problem by inventing new enhanced status codes, e.g.:

     S: 250 2.1.13 <user@foo.com> etLBdTj1iG NRaaVe83QN Saved

The client is expected to understand 2.1.13 and to parse the reply appropriately.

The 2 and 3 cases are also an example of when it would be necessary to combine these metadata to the same DATA reply.