Re: [imap5] Feature set? - was Re: Designing a new replacement protocol for IMAP

[Bron Gondwana <[e-mail]>]

On Mon, Feb 20, 2012 at 12:00:05PM +1300, Adrien de Croy wrote:
> that depends on how your system is structured.  The MTAs I'm
> familiar with put files into their send queues, and specify
> envelopes separately, either in a separate file or in a DB or
> something..

Unless you're putting the pre-exiting file from store into
the queue by reference rather than by copying, you would do
the rewrite as you copy.

Putting it in the queue by reference means you will have to
guarantee that the original mailbox copy doesn't get
expunged until the send has finished.
 
> What is the concern about sending the parameters?

Basically the \Sent things below.

> the main use-case I can think of where the recipients and
> return-path are not normally in the message are in lists.  So with
> your system, you won't be able to have an efficient list processor
> attach to an IMAP message store.  It will need to make copies of the
> message for each recipient.

I wouldn't expect an efficient list processor to be a user-facing
client.  Hence it would obviously talk directly, probably LMTP, to
the MTA.

> Another problem is auto-responders, where commonly you specify a
> blackhole or empty return path to avoid loops.

Not a user client.

> You'll at least need a way to update certain headers into the
> message efficiently.  This would make for much more efficient
> forwarding as well... copy, rewrite To, Subject and send.  Rather
> than download, edit, upload send.  Maybe store the headers
> separately.

Lemonade style CATENATE.

> Honestly I haven't fully convinced myself either way yet.  I am
> quite nervous about such a wide-reaching change though.

So am I.  But your arguments are convincing me more that I'm right.
The cases you're coming up with are all "agents running on the
server doing non-user-client things" or "we don't support a way to
strip headers from a message when sending".

The "we don't support a way to strip headers" is the closest to a
viable argument I've heard so far, but that's an egg I think is
worth breaking in exchange for a really clear model about who a
message was sent to.  You read the headers, you know.  The \Sent
flag has a really specific meaning - and it takes an actively
malicious client to break that.  The theoretical "moron" will
get it right, because getting it wrong is a lot more work.

> I must be missing something.  AFAIK the common case that happens all
> the time is that you specify forward and reverse paths separately
> from the message content.

Yes, you do.

> You're proposing removing this.

Yes, I am.

> I'm still open to the option, but there are a myriad of consequences
> that haven't been touched on.
> 
> Most MUAs can and will still ensure the parameters match the
> headers.  Other software may have other requirements.  You're just
> saying they will need to use SMTP.  Fine, but that doesn't move us
> forward.

I'm saying this is a protocol for MUAs to talk to mail stores.  That's
all.  It does not replace SMTP, it does not replace LMTP.  It provides
a submission path for MUAs that can and will and want to make the
parameters match because the message on the server is a record of what
was sent.

> >Make the easy things EASY and the
> >hard things possible, not everything equally hard.
> sure.  I just don't think it's hard to have a command like
> 
> SEND UID forward-path reverse-path
> 
> In fact I'd be keen on something like
> 
> SENDFLAGANDMOVE UID +\Sent "Sent Items" forward-path reverse-path
> 
> to avoid magic which you're otherwise doing with your Sent flag.

The magic is a latch.  The latch is a protection against replay.
I'd rather see "SENDFLAGANDMOVE" as a composition of simpler commands
which can be composed than one funny-shaped lego block, so when you
want to add something else to your chain, you don't need to do
anything else.

Which would make it "MOVE (IFNOT (FLAGS (\Sent)) (REPLACEHEADERS (Envelope-To: forward-path Envelope-From: return-path) +FLAGS (\Sent) DELIVER) UID "Sent Items"

Looks a bit nasty in this syntax for sure.  But that's non-magic
then, it's a transaction including the various changes.  I would
infinitely prefer this to actual magic.  But I think side effects
on taking an action are significantly preferable to side effects
on reading.  The same way that a "STORE +FLAGS" has an implicit
side-effect of bumping the MODSEQ on the message, and you can do
an UNCHANGEDSINCE to latch that too.

> The final point I'd make about obtaining the forward path from the
> message vs a parameter.
> 
> If you're parsing the To, CC, BCC headers, you need to be able to
> parse all character sets (there are many) in a completely
> bullet-proof fashion.

Now that's a better argument than anything else I've seen so far.

The flip side is, you're going to have to parse them to search and
sort on them as well - so the code already exists in your server
if it's even vaguely complete.

Bron.