[ietf-smtp] SMTP server reply extensions

[Timo Sirainen <[e-mail]>]

Hi,

I've recently found several reasons to add some kind of extended response metadata to SMTP/LMTP's RCPT and DATA replies. Similar to IMAP's resp-codes. Since I'm not so familiar with SMTP protocol, I was thinking I'd ask here if anyone happens to know already existing solutions for this or has some recommendations? Maybe there is already some RFC solving a similar problem?

IMAP uses "[resp-code parameters]" strings, but this doesn't seem to be used with SMTP protocol. Another option is to invent new SMTP enhanced code numbers, but especially for private extensions it seems a bit dangerous since there could be numbering conflicts. Also I'm not sure if SMTP clients would become confused when they see codes they don't understand? Also another problem with these numeric codes is how to add multiple different replies, especially since multi-line replies can't have different enhanced codes. But perhaps a single new extensible enhanced code could be invented? Something like:

RCPT TO:<[e-mail]>
250-2.1.100 key1 metadata1
250-2.1.100 key2 metadata2
250 2.1.100 OK with some extra metadata
DATA
354 OK
.
250-2.0.100 key1 metadata1
250-2.0.100 key2 metadata2
250 2.0.100 <[e-mail]> Saved and returned some metadata

To maybe clarify things, below are the 3 use cases I have so far. The first two are likely to be completely internal to Dovecot and I'm not trying make them standard.

1. IMAP-like login referrals (RFC 2221)

This RFC isn't really used nowadays, but I'm now thinking of using referrals internally in Dovecot clusters. Dovecot backend could send a referral and Dovecot proxy would reconnect to that referral instead. But the same thing is needed also for LMTP, which means that it would have to be sent in a RCPT TO reply since the redirection can be different for different users. Perhaps something like:

RCPT TO:<[e-mail]>
250 2.1.5 OK
RCPT TO:<[e-mail]>
450 4.4.4 [REFERRAL smtp://backend2/] Redirected
RCPT TO:<[e-mail]>
450 4.4.4 [REFERRAL smtp://backend3/] Redirected

2. Single instance storage for deliveries between backends in a cluster

The idea is that if backends share a storage (NFS, object storage) and there are multiple recipients, the proxy could deliver mail first to backend1, get back some metadata and use that to deliver mail to backend2, which would use the metadata to store a copy of the original mail. So for example:

RCPT TO:<[e-mail]>
250 2.1.5 OK
DATA
354 OK
.
250 2.0.0 <[e-mail]> [MAILDATA /nfs/mail/[e-mail]/INBOX/Maildir/file1] Saved

(In reality the path would likely be a base64 encoded blob)

And then in backend2:
RCPT TO:<[e-mail]> MAILDATA /nfs/mail/[e-mail]/INBOX/Maildir/file1

Now backend2 can create a hard link from [e-mail]'s mail to [e-mail]'s INBOX, saving some disk space. (Obviously only if the mails would be identical.)

3. We published a "submission token" draft a while ago: https://www.ietf.org/archive/id/draft-slusarz-extra-smtp-submission-token-00.txt

It's going to need a bunch of changes, but that draft is also trying to solve this metadata problem by inventing new enhanced status codes, e.g.:

     S: 250 2.1.13 <[e-mail]> etLBdTj1iG NRaaVe83QN Saved

The client is expected to understand 2.1.13 and to parse the reply appropriately.

The 2 and 3 cases are also an example of when it would be necessary to combine these metadata to the same DATA reply.