Re: [imap5] Feature set? - was Re: Designing a new replacement protocol for IMAP

[Adrien de Croy <[e-mail]>]

On 20/02/2012 12:39 p.m., Bron Gondwana wrote:
> On Mon, Feb 20, 2012 at 12:00:05PM +1300, Adrien de Croy wrote:
>> that depends on how your system is structured.  The MTAs I'm
>> familiar with put files into their send queues, and specify
>> envelopes separately, either in a separate file or in a DB or
>> something..
> Unless you're putting the pre-exiting file from store into
> the queue by reference rather than by copying, you would do
> the rewrite as you copy.
>
> Putting it in the queue by reference means you will have to
> guarantee that the original mailbox copy doesn't get
> expunged until the send has finished.

i'm just talking about good old file system file copy, without parsing.

>
>> What is the concern about sending the parameters?
> Basically the \Sent things below.
>
>> the main use-case I can think of where the recipients and
>> return-path are not normally in the message are in lists.  So with
>> your system, you won't be able to have an efficient list processor
>> attach to an IMAP message store.  It will need to make copies of the
>> message for each recipient.
> I wouldn't expect an efficient list processor to be a user-facing
> client.  Hence it would obviously talk directly, probably LMTP, to
> the MTA.
>
>> Another problem is auto-responders, where commonly you specify a
>> blackhole or empty return path to avoid loops.
> Not a user client.

check out RFC 6409 Sec 3.2 para 5 re NULL return paths.

Clients do commonly auto respond.

>
>> You'll at least need a way to update certain headers into the
>> message efficiently.  This would make for much more efficient
>> forwarding as well... copy, rewrite To, Subject and send.  Rather
>> than download, edit, upload send.  Maybe store the headers
>> separately.
> Lemonade style CATENATE.

I need to take another look, last time it gave me a headache. Is anyone 
supporting this?

we may be putting the cart before the horse talking about submission 
before we even discussed message generation.

>
>> Honestly I haven't fully convinced myself either way yet.  I am
>> quite nervous about such a wide-reaching change though.
> So am I.  But your arguments are convincing me more that I'm right.
> The cases you're coming up with are all "agents running on the
> server doing non-user-client things" or "we don't support a way to
> strip headers from a message when sending".
>
> The "we don't support a way to strip headers" is the closest to a
> viable argument I've heard so far, but that's an egg I think is
> worth breaking in exchange for a really clear model about who a
> message was sent to.  You read the headers, you know.  The \Sent
> flag has a really specific meaning - and it takes an actively
> malicious client to break that.  The theoretical "moron" will
> get it right, because getting it wrong is a lot more work.
>
>> I must be missing something.  AFAIK the common case that happens all
>> the time is that you specify forward and reverse paths separately
>> from the message content.
> Yes, you do.
>
>> You're proposing removing this.
> Yes, I am.
>
>> I'm still open to the option, but there are a myriad of consequences
>> that haven't been touched on.
>>
>> Most MUAs can and will still ensure the parameters match the
>> headers.  Other software may have other requirements.  You're just
>> saying they will need to use SMTP.  Fine, but that doesn't move us
>> forward.
> I'm saying this is a protocol for MUAs to talk to mail stores.  That's
> all.  It does not replace SMTP, it does not replace LMTP.  It provides
> a submission path for MUAs that can and will and want to make the
> parameters match because the message on the server is a record of what
> was sent.

does anyone except ISPs use LMTP?

>
>>> Make the easy things EASY and the
>>> hard things possible, not everything equally hard.
>> sure.  I just don't think it's hard to have a command like
>>
>> SEND UID forward-path reverse-path
>>
>> In fact I'd be keen on something like
>>
>> SENDFLAGANDMOVE UID +\Sent "Sent Items" forward-path reverse-path
>>
>> to avoid magic which you're otherwise doing with your Sent flag.
> The magic is a latch.  The latch is a protection against replay.
> I'd rather see "SENDFLAGANDMOVE" as a composition of simpler commands
> which can be composed than one funny-shaped lego block, so when you
> want to add something else to your chain, you don't need to do
> anything else.
>
> Which would make it "MOVE (IFNOT (FLAGS (\Sent)) (REPLACEHEADERS (Envelope-To: forward-path Envelope-From: return-path) +FLAGS (\Sent) DELIVER) UID "Sent Items"
>
> Looks a bit nasty in this syntax for sure.  But that's non-magic
> then, it's a transaction including the various changes.  I would
> infinitely prefer this to actual magic.  But I think side effects
> on taking an action are significantly preferable to side effects
> on reading.  The same way that a "STORE +FLAGS" has an implicit
> side-effect of bumping the MODSEQ on the message, and you can do
> an UNCHANGEDSINCE to latch that too.

however you do it, it needs to be transactional.  A single command for 
this is one way to do it. Or you could create a new syntax that 
explicitly does transactions.

Coding my SENDFLAGANDMOVE function would be quite simple.  Your's with 
the conditional would be less simple.  An alternative is providing 
explicit locking methods to clients (a la webDAV).

But frankly I don't like giving control of locks to clients.  So, maybe 
the best way is

START
   SEND UID
   UID STORE UID +\Sent
   MOVE ...
END

Sent as one block.  But that's just splitting my previous example into a 
few more lines.  Gives you more combinations/flexibility I guess.

Actually I don't have a problem with a side-effect of Send being to set 
a \Sent flag.  But prohibiting subsequent re-sends I'm not so convinced 
about.  You'd need to clear that flag if the message were to be copied 
(more magic).

>> The final point I'd make about obtaining the forward path from the
>> message vs a parameter.
>>
>> If you're parsing the To, CC, BCC headers, you need to be able to
>> parse all character sets (there are many) in a completely
>> bullet-proof fashion.
> Now that's a better argument than anything else I've seen so far.
>
> The flip side is, you're going to have to parse them to search and
> sort on them as well - so the code already exists in your server
> if it's even vaguely complete.

failure is quite different though - failure to deliver vs failure to search.

Having had to deal with things like KoiR, Shift-JIS etc etc, it's non 
trivial.  Frankly I'd avoid parsing wherever possible.

An alternative would be to take another approach, where envelope is 
stored separately from the message itself, and combined with it at 
submission time when the copy is put into the sent folder.

I don't see a strong reason to prohibit sending the same message more 
than once.  Resending from my sent items folder is something I do quite 
often.  They end up as another message in there.  So It's implicitly a 
copy, edit, send.

Adrien

>
> Bron.

-- 
Adrien de Croy - WinGate Proxy Server - http://www.wingate.com
WinGate 7 is released! - http://www.wingate.com/getlatest/