Re: [imapext] [ietf-smtp] Fwd: Request to form a new WG: JMAP

[Ned Freed <[e-mail]>]

Apologies for the late response on this one.

> A few questions about some of your comments (again, just
> questions);

> --On Wednesday, November 09, 2016 07:15 -0800 Ned Freed
> <[e-mail]> wrote:

> >> The complexity of the requests and replies are the issue. If
> >> its mostly converting IMAP to JSON, the same problems are
> >> going to exist.

> > Yes and no. Part of the problem with IMAP is that it was
> > designed for a very different sort of client than what we have
> > now.

> Which design model are you talking about?   It seems to me that
> the original specs (through and including RFC 1064 and IIR 1176)
> had only what we now call online mode and was optimized for
> "kiosk" uses.  With the exception of the "walk up to someone
> else's shared machine" part, that seems very similar to what we
> have today with the assumption that everything is on the server,
> the client keeps messages only on a transient basis, and one
> cannot access messages unless one is online.    It seems to me
> that is not very different from where we are headed now, whether
> via "dumb MUA" or "MUA implemented on a web interface".   It may
> be more complex than needed because it contains some
> optimizations are what, by today's standards, are low-bandwidth
> environments but some of those features also have security
> advantages (that we haven't talked about very much recently).

Pretty much the entire design model is canted away from what I see being needed
by modern clients.

One assumption the old model makes is that bandwidth is cheap enough that
there's no need to highly optimize metadata transfer. And to some extent this
also applies to message data, e.g., the lack of support for message summaries
and thumbnails, but that's a more complex topic.

This matters because with mobile client view timliness is critical, but not at
the expense of bandwidth. And push notifications only get you half of the way
there - once notified you have to connect and update.

Sit down with a mobile client designer sometime, and you'll quickly see that in
order to build the sorts of message displays they want without transferring far
more data than they need or can afford, you need to engage a large number of
IMAP extensions, and even then you're unlikely to achieve the exact result they
would prefer.

And then there's search. A powerful search capability is now essential, but
IMAP search assumes a particular set of semantics that are simultaneously not
what a mobile client needs (nobody is going to spend time entering complex
search terms on their phone) as well as being misaligned with modern text
search engines, making it difficult to implement on the back end. (What
typically happens is implementations vary substantially from what the
specifications call for, and vary in different ways.)

Then there's the stuff that nobody cares about that you have to support, like
subcriptions.

I could go on and list more stuff, but I think this makes the point that
there's a mismatch. And really, given how the landscape has changed and how old
some of these specifications are, I see no reason for surprise.

> Now, when we added offline (simulated POP3 on steroids) and
> disconnected modes with IMAP4, things got more complex.  From my
> personal perspective, disconnected mode (more or less the
> ancient PCMAIL model) is the only thing that makes sense for
> people with poor or intermittent connectivity, especially if
> they need to work offline sometimes and use multiple machines.
> However, it has never been broadly and well-supported in MUAs
> (there are, IMO, a few "good enough" MUA implementations but no
> really good ones) so maybe the marketplace has told us there
> aren't enough users to count.

Offline support is another thing that's increasingly superfluous now. Mobile
devices are expected to always be online and to present an up-to-date view of
the world at all times.

Of course the sync capbilities designed to support offline mode can also be
used to minimize data transfer for a connected client, but even so, it's not
quite the same problem.

> Now, I've never been a fan of the LISP-like command and response
> syntax, but that is much more a matter of taste than
> functionality.

Unfortunately, syntax matters more than it really should. I previously pointed
out that the complexity of HTTPS in this context is irrelevant because "there's
a library for that". The same is true for JSON - the automapping mapping of
JSON to and from data structures that many languages offer is very convenient.

> > (I note that the problem SMTP is intended to solve hasn't
> > changed nearly as much.)

> I agree with you but have heard claims that the multiple relay
> and alternate destination (mail exhannge) models have outlived
> their usefulness in today's highly-connected Internet.

Even if it's true, this is all external to SMTP itself, and I don't think
it's relevant here.

> I don't
> see things as that highly connected (even looking to the future,
> SMTP would work much better in the presence of a DTN than, e.g.,
> HTTP[S} and for much the same reason it worked well for delivery
> servers that were only connected a few hours a week) and I think
> the robustness those features enable as important.  However,
> there is no disputing that mail transport could be considerably
> simplified if we allowed only endpoint-to-endpoint connections
> with no alternatives.

Uh, no, not really. To the extent that implementation of message transfer is a
complex problem, it's mostly in the splitting of messages and the handling of
errors. These problems don't go away when you limit things to one hop. In fact
one of the nice things about SMTP is that you don't know and don't need to know
how many hops there are ahead of you.

The complexity of transitions through multiple ADMDs grots up the security and
spam situation, but those are for the most part externalities to the SMTP
implementation.

Additionally, this notion of getting rid of multiple hops across the Internet
ignores the present day reality that as the size of a "typical" mail system
deployment has increased, so has the number of hops messages often take inside
the deployment.

> Such connections would also permit
> straightforward feature negotiation.  That would be a big help
> for, e.g., EAI/SMTPUTP8 and similar arrangements and might even
> help blur the boundary between transport-level and end0t9-end
> encryption.

Perhaps, but again, I don't see the relevance to the matter at hand.

> >  Getting rid of historical baggage will definitely simplify things.

> But that is my question.  I don't see IMAP disconnected mode as
> historical baggage.  We've ended up with some redundant ways to
> do things as a result of newer and more powerful options and
> commands and the old ways could be cleared out.   We could
> probably make some different choices about data structures and
> maybe even standardize some things that today just add to the
> complexity of the options.  But I'm not sure how significant
> those actually would be -- what are you thinking of?

IMAP disconnected mode support isn't baggage because it provides capabilities
that are useful to mobile connected clients, albeit in a somewhat suboptimal
fashion.

That said, there absolutely is substantial baggage in IMAP.

> ...

> > My guestimate is that - and here I'm speaking only of the
> > message access space - that a HTTPS/JSON approach wins in
> > terms of effective implementation complexity, although it is
> > far from clear to me by how much.

> But, unless we can get rid of IMAP (and some of the
> functionality, like disconnected mode, that can't obviously be
> supported over a purely HTTPS/JSON interface) entirely, it seems
> to me that an HTTPS/JOSN approach is additive, requiring the
> mail environment to support both it and IMAP for (at least) a
> very long time.  So, while I agree with your guess, I don't see
> the need to support an additional as a net gain, especially in
> the short to medium term.

Just a guess on my part, but I rather suspect that disconnected mode is
easier to do on top of a protocol designed to optimize mobile sync than
the other way around.

Of course this ignores the cost of conversion. But I don't think you can
necessarily make the case based on protocol features alone.

				Ned