Re: [imap5] Feature set? - was Re: Designing a new replacement protocol for IMAP

[Adrien de Croy <[e-mail]>]

On 20/02/2012 11:08 a.m., Bron Gondwana wrote:
> On Mon, Feb 20, 2012 at 09:31:03AM +1300, Adrien de Croy wrote:
>> not everyone is on sendmail
>>
>> we're talking about adding 2 parameters, a return path and forward
>> path.    Hardly complex.
>>
>> this allows us to
>> - not have to parse the headers
>> - not have to re-write the entire message when stripping bcc
> Most MTAs seem to do header rewriting by appending or stripping
> parts of the header on the fly.  They don't rewrite the entire
> message, they print the headers down the wire, then blat the
> body.  That's not expensive unless you're doing it wrong.

that depends on how your system is structured.  The MTAs I'm  familiar 
with put files into their send queues, and specify envelopes separately, 
either in a separate file or in a DB or something..

So a message file re-write would be required to strip bcc before putting 
it into the queue, or a re-factor of the SMTP sending code to allow the 
option to parse the headers on the fly (the option would need to be 
communicated out-of-band).  That is of course do-able.  But it forces a 
model on the SMTP delivery agent.

What is the concern about sending the parameters?

That they can be different to the headers in the message?

the main use-case I can think of where the recipients and return-path 
are not normally in the message are in lists.  So with your system, you 
won't be able to have an efficient list processor attach to an IMAP 
message store.  It will need to make copies of the message for each 
recipient.

Another problem is auto-responders, where commonly you specify a 
blackhole or empty return path to avoid loops.

You'll at least need a way to update certain headers into the message 
efficiently.  This would make for much more efficient forwarding as 
well... copy, rewrite To, Subject and send.  Rather than download, edit, 
upload send.  Maybe store the headers separately.

>> these things are not cheap.  they will affect service capacity.
> Seriously?  We do it to millions of emails in Postfix every day
> on a single box and the CPU barely breaks a sweat.  What sort of
> capacities are you considering serving on a single machine here?

I always save CPU when I can.  There's always a bigger client out there 
who will need more capacity.

> Micro-optimising this... if you can save a single fsync somewhere
> you'll get much better optimisation than avoiding a header removal.
> Remember you're probably ADDING a header anyway to note the sending
> time and user/ip of the sender at the time it happened.

Senders don't normally stamp Received if that's what you're referring 
to, that happens at the SMTP receiver.  our sender doesn't write to the 
message.

>
>> Why would you incur such penalty simply to avoid transmission of a
>> couple of parameters?
> Because I don't believe you when you make it out to be a big penalty.
> You're not rewriting the header on disk, purely on what gets written
> to the wire.

Sure if you do it all your way... but not all mail systems work that way.

>
>> It's trivially simple for an MUA to supply this information when it
>> wants to send the message.
> Trivially simple, yes - not so simple to track and repeat.

Sure, displaying the information later is not something currently supported.

Your way allows existing sent items mailbox to continue to perform its 
current function in the current way.

But there may be a better way.

Honestly I haven't fully convinced myself either way yet.  I am quite 
nervous about such a wide-reaching change though.

>
>> There's no reason to assume this information would be lost either.
>> It's up to the server how to record such things and make them
>> visible.  Whether that's in the sent items folder or something
>> associated with the message.
> Significantly more complex than a simple keyword.  For a rare case.
> Parsing "Resent-To" instead handily covers the entire use-case that
> you have raised, with no loss of functionality, and without bloody
> raising the edge cases to higher importance than the simple common
> case which happens all the time.

I must be missing something.  AFAIK the common case that happens all the 
time is that you specify forward and reverse paths separately from the 
message content.

You're proposing removing this.

I'm still open to the option, but there are a myriad of consequences 
that haven't been touched on.

Most MUAs can and will still ensure the parameters match the headers.  
Other software may have other requirements.  You're just saying they 
will need to use SMTP.  Fine, but that doesn't move us forward.

I think if we're to drop the concept of an SMTP envelope, and say you 
can't send a message where the to and from in the message don't match 
the envelope (which I'm sure many people would like) we need tomake sure 
we don't break so much stuff as to make the new protocol impossible for 
people to adopt is all.  Just seems that policy can be used to enforce 
such a requirement as desired, rather than mandating it in the protocol 
and removing everyone's choice.

> Make the easy things EASY and the
> hard things possible, not everything equally hard.
sure.  I just don't think it's hard to have a command like

SEND UID forward-path reverse-path

In fact I'd be keen on something like

SENDFLAGANDMOVE UID +\Sent "Sent Items" forward-path reverse-path

to avoid magic which you're otherwise doing with your Sent flag.

The final point I'd make about obtaining the forward path from the 
message vs a parameter.

If you're parsing the To, CC, BCC headers, you need to be able to parse 
all character sets (there are many) in a completely bullet-proof fashion.

Adrien

>> Distinction between envelope and content is a key principle of SMTP.
>> One could argue it's also a cause of certain issues, but I don't
>> think it should be discarded without some fairly serious thought.
> Happy to have serious thought about it, but in this case the initial
> envelope should IMHO be part of the content, because you're sending
> something out of your "\Sent" box, near enough, and what's there
> should be a record of what you sent.
>
> SMTP will still exist for the 0.1% cases.  Resent-To will handily
> cover the normal "user operated client" use-case.  This is a method
> for "submission by a user operated client", that's all.
>
> Bron ( getting bogged down in side issues, whee )

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com
WinGate 7 is released! - http://www.wingate.com/getlatest/