Re: [Jmap] Best vs Good enough - adoption of JMAP

["Chris Newman" <[e-mail]>]

On 25 Apr 2017, at 16:46, Neil Jenkins wrote:
> I think there's been some misconception over the intended semantics of
> the outbox proposal. The outbox was never intended to be the 
> submission
> queue, with mail queue semantics. It is intended as a standard 
> mailbox,
> with mailstore semantics, because it is where messages go **before**
> they are passed on to submission. The server would validate the
> message/envelope at the time of moving it to the outbox to ensure it 
> can
> be accepted for submission later of course. If you have existing
> mailstore + SMTP infrastructure, implementation is not much more than
> having some small demon that triggers to take the message from the
> mailstore and submit it to the SMTP queue at the appropriate time.

And as soon as you add that "small daemon" (also known as a mail queue 
processing daemon), you've now required the mailstore to also have mail 
queue semantics. This includes key semantics such as: the mail queue 
daemon has full read / delete access to those messages even when the 
user is offline. If the delivery doesn't happen when the user expects, 
the user is going to make a support call, so we now need all the 
monitoring, logging and robustness that comes with a mail queue. And 
addresses that were valid at the time the message entered this mail 
queue can become invalid later, so there has to be bounce-processing for 
when that happens, etc, etc.

The only way to have a server-side outbox that is not a mail queue would 
be if messages sit in the outbox indefinitely unless the user is online 
and their client decides to issue a submit command for a message in that 
fictitious outbox.

> From a user experience point of view, this model gives us an
> understandable conceptual place for messages that are *going* to be
> sent but have not yet been picked up by the postman, to use a real
> world analogy.

This analogy only works with a desktop MUA where the "outbox" is on the 
desktop and thus owned by the "postal customer" (and incidentally, the 
outbox won't empty if the desktop is shut down). The hardware running 
JMAP is not owned but the "postal customer"; it's effectively owned by 
the postman. So an "outbox" on the JMAP server means the message has 
been given to the postman already; the postman is just waiting for the 
right time to release it to the next hop.

> Because it is a standard mailbox, we automatically have
> an API for the client to fetch and show to the user all the currently
> scheduled messages. I think this is important functionality as a user
> if we have delayed send (and of course we also have the ability to 
> then
> cancel any of the future submissions, by just moving the messages back
> out of the outbox).

The problem with server-side-outbox in JMAP is that it constrains the 
server implementation to three options:

1. The mail store includes a mail queue for each user.
2. The mail queue, at least for delayed messages, can be accessed with 
full JMAP semantics. Including things like body structure, mod sequence, 
flags, etc.
3. The JMAP outbox is always empty.

I really want a fourth option for server implementation:

4. The mail store and mail queue are completely separate. The mail queue 
daemon never accesses the mail store. The set of actions JMAP performs 
on objects in the mail queue is limited to only those that are strictly 
necessary to achieve a useful goal (envelope extensions like future 
release at submission time, unsend for messages still in local queue, 
and status feedback tagging on sent messages).

> With my client author hat on, I like this model because it makes it
> easier to implement full offline support. If I am offline, the outbox
> represents messages waiting to go out from the client. From an
> implementation perspective it is much easier if my offline sync code
> doesn't need a separate path for keeping track of messages to be sent
> vs. normal messages moved to another folder.

A client-side outbox makes sense to me. It's a server-side outbox that I 
find problematic. Let me ask you this:

Suppose JMAP implements a server-side outbox (aka mail queue). The JMAP 
client synchronizes the server-side outbox and goes offline. While the 
client is offline, the message gets submitted and the user decides they 
don't want to submit the message and delete it from the local cached 
outbox. Now what? Have you really simplified things by adding a 
server-side outbox to the mix?

> The *only real difference* between a "sendMessage" command and using
> "move to outbox" as the equivalent command, is that clients can get
> the list of "delayed" messages waiting for submission. I think this is
> valuable, and pretty much required for "scheduled send", although not
> so important for "undo send". The fact that this is really the only
> difference is why servers that can't or don't want to implement
> delayed send or expose messages waiting to be sent can use the "empty
> outbox" model.

My product has already implemented future release, already implemented 
unsend from local mail queue and implemented message tracking so we can 
get status of a message in the local queue (and potentially beyond 
that). I'm willing to expose these in JMAP if the protocol is designed 
in such a way that we preserve clean functional separation between mail 
store and mail queue. I'm very much not interested in all the complexity 
a server-side outbox adds on top of those three features.

I'd argue that "scheduled send" is a mail client feature and "future 
release" is a mail server feature. If there's a daemon that can move the 
message to the next hop even when the client is offline, you're talking 
about "future release" and the message has semantically been submitted 
already. So I think JMAP should put "future release" messages in the 
Sent folder to avoid confusion with a client-side outbox. I'd like JMAP 
to include a way to "unsend" future release messages so JMAP clients 
have access to the unsend feature we've already implemented.

		- Chris