Re: [Jmap] simpler future release & unsend without outbox (was Re: Best vs Good enough - adoption of JMAP)

["Chris Newman" <[e-mail]>]

On 3 May 2017, at 0:57, Neil Jenkins wrote:
> On Mon, 1 May 2017, at 09:53 PM, Chris Newman wrote:
>> Here's my guess at the root cause:
>> * You're trying to mirror a particular client UI model in the mail
>>   store server design.
> I'm trying to ensure that the API exposes the data clients need to
> present common UI patterns in an efficient way, while also allowing 
> the
> most flexibility in server implementation. I think we're both aiming 
> for
> the same thing!
>> 1. I believe there are four message types involved in this 
>> discussion:>   * draft messages
>>   * client-side outbox
>>   * future release RFC 4865 submitted
>>   * other submitted messages
>
> I agree, although I don't think the client-side outbox is particularly
> relevant to this discussion, as it should not affect the API design.

An important factor in protocol/API design is to avoid misleading the 
consumers. So in this respect the distinction between client-side outbox 
and server-side future release messages is important. We don't want 
protocol/API consumers to confuse those two classes of messages because 
they behave differently when the client is offline.

> I'm sorry I brought this up as it side-tracked the discussion 
> somewhat. My
> main point was that many existing systems assign submitted "future
> release" messages to a separate mailbox on the *server*, and clients
> present them to the user distinctly to those that have been released.
> What I misunderstood from your proposal, is that rather than marking
> messages that can be recalled/cannot be recalled, you propose a
> client just look at the "release date" annotation instead. Although
> you also say this:
>> Actually, my proposal makes this server design approach
>> simpler. There's> just an internal flag in the sent folder to 
>> indicate if the
>> message has> been passed forward. The mail queue daemon that operates 
>> on the mail
>> store sent folder, can transfer the message to the next hop MTA and
>> set> that flag at the future release time. No need to move the 
>> message
>> between folders.
>
> So the mail queue daemon *does* perform an action on the message in 
> the
> mail store when it is sent? I previously thought you were advocating 
> for
> a model where these can be completely separate and so this is not
> possible. Could you please clarify?

I advocate an implementation model where the mail queue and mail store 
can be separate in the server implementation. I accept that some server 
implementations may choose not to do this and the protocol design 
shouldn't constrain potentially reasonable implementation choices. In 
the quote above, I was explaining one way a server implementation that 
mixes mail queue and mail store could work with the protocol model I 
proposed.

> I think we've got various issues conflated a bit over these 
> discussions,
> so I'll clarify what I think are the separate questions:
> How, if at all, are "future release" messages (that have yet to be
> released) marked over the API? Is there a change in the state when 
> they
> are released? I agree this may not correspond exactly with whether a
> message can be unsent, especially for internal corporate mail systems,
> so a related question is: how, if at all, are messages marked that can
> be unsent? (This is potentially a tristate: can (almost) definitely
> unsend, can definitely not unsend, may be able to unsend).
> I think if we can agree on answers to these questions, the rest will
> fall out fairly easily.

I prefer a protocol model where it is possible for the client to easily 
identify "future release date is in the future" messages and to apply an 
"unsend" action to those messages. I further believe the model shouldn't 
limit the "unsend" action so it can only be used with that set of 
messages. And I'd prefer a model where there is no explicit state change 
in the mail store when a "future release" message is released as that 
keeps the mail queue and mail store components independent.

I'm ok if the base JMAP spec requires implementation of future release 
and unsend for future release messages. I'd prefer to leave details 
about message tracking (including whether a message can unsent) and 
unsend for other messages to a JMAP extension. Things get complicated 
for the other cases. First, it's actually a four state problem: "future 
release in the future", "unsend not possible", "unsend maybe possible", 
"no information about whether unsend is possible". And there's the 
question of whether tracking is an action (as it is for RFC 3887), or we 
require an MTA to mail store notification path for all messages to 
indicate whether a sent message can be unsent. The latter probably 
results in a better UI but I think it's too much to require in the JMAP 
base spec if we want JMAP to deploy quickly.

>> 3. I view "submit" and "unsend" as MTA actions that should not be
>> confused or conflated with mail store operations. I'm opposed to
>> protocol designs where a message move, delete or flag changes 
>> triggers> MTA actions as a side-effect.
>> Instead, I propose JMAP have explicit "submit" and "unsend" commands
>> that first perform the MTA action but can> also trigger mail store 
>> actions (like move message or flag change) if
>> the MTA action succeeds.
> Yes, this is a perfectly fine approach; I'm not opposed to adding
> explicit sendMessages/unsendMessages calls. If the call can optionally
> set which mailbox(es)/keywords to set upon successful send, then this 
> is
> functionally equivalent.

Great, we have an approach we both support.

>> My proposal is to have future release sent messages tagged with 
>> their> future release date. The MUA can create a virtual folder for 
>> future
>> release messages with a future date. Whether the server caches a 
>> count> for the messages in that virtual folder is a server design
>> optimization> choice. If enough clients request that count that it 
>> helps server
>> scalability to cache that count, then servers will cache that
>> count. The> count of future release messages is thus not relevant to 
>> the
>> issue under> debate.
>
> What's relevant is how this count is exposed over the API. If there
> is a mailbox, then the synchronisation of mailbox objects will ensure
> the client is "pushed" the count; because count changes are common
> and clients need to know them to present the most common UI paradigm
> for mail, synchronisation of mailbox counts is optimised for in the
> JMAP protocol.

Virtual/smart mailboxes are a common mail UI feature. As a mail user, 
I'd like my virtual mailboxes to look the same in different mail clients 
to the extent possible. So the way to get more consistent behavior is to 
have a server-side-virtual mailbox model. I'd very much like JMAP to 
have the ability to create a virtual/smart mailbox that behaves like a 
regular mailbox in most ways (including optimized counts). That's a 
significant value-add that could be implemented in the JMAP layer. IMAP 
has primitives that can help (like RFC 5466, and RFC 5267) but nothing 
that enables smart mailbox behavior consistency across clients.

> One proposal that might satisfy everyone: an "outbox" or 
> "futurerelease"
> mailbox role is added (this is the IMAP SPECIAL-USE representation in
> JMAP).  Servers can *optionally* have a mailbox with this role; if it
> does, clients SHOULD place future-release messages in here, but like
> "sent" *can* put the message in any mailbox(es). Servers that have 
> this
> mailbox MUST move messages from this mailbox to the sent mailbox at
> future release time (which may just be based upon the future-release
> date, or may be actually based on releasing it to the mail queue).

This is something I could implement easily as long as there's no 
requirement that the mail store verify the message was actually released 
from the mail queue at the future release time. But I'm not sure it's a 
good design as it raises some issues... Messages in the futurerelease 
mailbox have been submitted/sent, so should that mailbox also have the 
\Sent special use? And if we then end up with more than one mailbox with 
\Sent special use, will mail clients need code to prefer the one without 
the "futurerelease" special use for sent messages that aren't future 
release?

I think a futurerelease virtual mailbox (based on the sent mailbox) is a 
cleaner way to do this (and optionally a sentandreleased virtual mailbox 
for clients desiring that).

		- Chris