Re: [Jmap] Best vs Good enough - adoption of JMAP

["Adrien de Croy" <[e-mail]>]

------ Original Message ------
From: "Chris Newman" <[e-mail]>
To: "Adrien de Croy" <[e-mail]>
Cc: "[e-mail]" <[e-mail]>
Sent: 25/04/2017 12:39:12 PM
Subject: Re: [Jmap] Best vs Good enough - adoption of JMAP

>On 24 Apr 2017, at 15:43, Adrien de Croy wrote:
>>On the topic of best vs good enough.
>
>This is a misleading dichotomy for an engineering discussion. My 
>immediate response when you say "best" is "best for whom?" If you 
>design the best possible protocol for end-user experience, it will 
>place unreasonable requirements on servers and won't deploy. Good 
>engineering involves tradeoffs, cost/benefit analysis, deployability 
>analysis, extensibility design, etc.
>
>>I agree they are perhaps opposed, you could say that good enough is 
>>the enemy of best however, since once you have something that is good 
>>enough, your incentive to do any better is greatly diminished.
>>
>>However, we are up against several obstacles when it comes to adoption 
>>of JMAP.
>>
>>1. Server authors won't want to add support unless there are clients
>>2. Client authors won't want to add support unless there are servers
>>3. Client or server authors won't want to add support unless they can 
>>make things significantly better for mail users.
>
>I've never heard of a mail client author implementing a feature before 
>the server exists. Server authors are the ones who have moved first in 
>the email community.
there are a few vendors who provide both, and they are in a special 
situation where they can really push the envelope on features because 
they have control of both sides of the conversation.

This is why Google was able to push SPDY, and why HTTP/2 is like it is.


>So the key to JMAP deployability is to make it not unnecessarily 
>hostile to server developers (like myself) and server operators. But I 
>agree we also want JMAP to provide significant benefits to client 
>authors and their users. There are many design choices JMAP can make 
>that meet all of these goals.
>
>>4. We are losing customers to alternate messaging systems.
>>
>>If our target is just going to end up providing no benefits to users 
>>over what we already have with IMAP+SMTP, then nobody will bother.
>>
>>Where it gets interesting for me personally at least is if we can push 
>>the boundary of mail experience.  In many cases this can be done in 
>>the MUA but in some cases it requires support from the protocols.
>>
>>I hear the arguments that the more complex we make the spec, the less 
>>likely it will be implemented, at least correctly. I'm not a fan of 
>>complexity either, and neither of my suggestions do I consider to be 
>>particularly complex.
>>
>>My argument is that if the design doesn't hold out a carrot for better 
>>mail experience for users, then nobody will bother, and we're wasting 
>>our time here, which would be a shame because we have the potential to 
>>achieve a lot here.
>>
>>At the moment, there is a benefit on the table, mostly being the 
>>ability to only require configuration of access to 1 service (that's 
>>if JMAP does submission).
>>
>>I believe we should be looking for opportunities to make mail the best 
>>experience possible, come up with ideas, and we may decided to defer 
>>some or not implement others, but we should keep them in mind so that 
>>perhaps we don't close off the path to some of them.  We don't need 
>>another protocol that is hamstrung by lack of foresight.
>>
>>there are plenty of ideas around, look at the competition.
>
>Email is the most widely deployed multi-vendor messaging system on the 
>Internet.
This one is rapidly moving.  I'd agree it's most widely deployed, but I 
don't know if it's most widely used.  Since mobiles have overtaken 
desktops as internet endpoints, I'd imagine Facebook messaging is 
getting close in terms of number of people using it, if it hasn't 
surpassed mail already.


>The major value of doing JMAP as a standard is to get multi-vendor 
>deployment. A walled garden will always have features that an open 
>multi-vendor system can't achieve (simply for privacy reasons). Yes, we 
>should look at the features that are popular in walled garden messaging 
>systems and do a cost/benefit analysis. If the cost is low and benefit 
>is high, let's consider it. But if the cost is high or the feature will 
>involve a lot of careful design work and is outside the charter scope, 
>then you'll get opposition to adding that feature to the JMAP base 
>spec.
understood.  I'm not proposing creating any more walled gardens.  
However walled gardens with open protocols, and which avoid privacy 
issues are fair game.

>A multi-vendor standard needs to support both small and large 
>deployments and support both cloud and on-premises deployments. There 
>are hundreds of deployed IMAP and Submission server implementations on 
>the Internet and JMAP will deploy a lot faster if it "plays well" with 
>those implementations. And by "play well", I mean it can be added to an 
>existing deployment without major infrastructure changes.
yes.  It has to be deployable on all levels (e.g. meet policy and 
commercial requirements as well as technical).

>
>For example, if I'm operating a million-user deployment, it's critical 
>to monitor the mail queues for problems. The proposed JMAP "outbox" 
>model means I'm adding one million new mail queues to the monitoring 
>system. That's a major deployment change; and thus a significant 
>barrier to JMAP deployability.
>
>Suppose we remove the "outbox" model from JMAP and replace it with a 
>submission command with envelope extensions. Now suppose we want to 
>require JMAP servers to support BINARYMIME submission (and/or append) 
>as a boon to JMAP submission client authors? Well that's no problem 
>from an infrastructure viewpoint as long as the JMAP server is 
>permitted to perform the base64-encoding on binary message parts at 
>submission/append time in order to remain compatible with IMAP mail 
>stores (and non-BINARYMIME submission servers). Want to have the JMAP 
>submission command handle both the \Sent folder copy and the Submission 
>without duplicate data transmission? I think we need to do that. Want 
>to have the base JMAP submission or append command include IMAP 
>CATENATE functionality (assemble a message including server-stored 
>attachments in order to "forward without download")? I have no problem 
>with that.
Yes, I think clients need to be able to build the message on the server, 
in binary.

>
>There's quite a bit of room to improve the JMAP client author's 
>experience without forcing major infrastructure changes.

I guess my point here is that I don't believe we can force major 
infrastructure changes even if we decided we wanted to.  I didn't think 
I was proposing anything that would do that.


>The WG exists to discuss the cost/benefit of various features and make 
>an engineering judgment call. The proposals that are most likely to 
>succeed are ones that can be implemented in a JMAP proxy without 
>significant changes to IMAP/Submission. And proposals that would 
>require reasonably straightforward or well-understood IMAP/Submission 
>extensions are worth considering (e.g., if I need a per-user 
>unique/persistent message identifier for IMAP, I understand the 
>implications of that).
OK, so we have an IMAP server and SMTP server.  I'm intending (if it 
works out) to implement JMAP, but I would be making whatever changes 
needed to the IMAP and SMTP servers required to provide whatever added 
benefit I can give to a user by way of JMAP.

I don't really think of JMAP as a proxy.  I think of it on an even par 
with IMAP + SUBMIT servers.  I'm not intending to proxy JMAP back via 
IMAP, we abstracted the store from IMAP in any case (so it could be 
shared with POP3), so the JMAP server would be another user of that mail 
store.


>
>So the opposition you're seeing is to making fundamental architecture 
>changes to the Internet email infrastructure or adding complex new 
>mandatory features that have a poor cost/benefit. If you want to do 
>that, go design your own walled-garden protocol. But if you want to 
>make the client author experience better in a way that can deploy 
>across email systems from multiple vendors, then this is the place to 
>make and refine proposals that strike a deployable balance.
>
>Sorry for the meta-discussion, but I got the impression you had 
>misinterpreted opposition to infrastructure disruption as opposition to 
>improving the client experience.
No problem, I appreciate your response.

Maybe I haven't been clear in my suggestions, but I'm not proposing 
making all these suggestions mandatory, but I am keen that they remain 
possible, and not get prohibited by some design decisions now.

Being able to do something prior to submission relating to submission as 
a general concept I think has some interesting possibilities, and early 
mail verification I think has a lot more value than some are seeing.  
When you've had your umpteenth mail bounced that took you hours to 
write, you end up wishing something told you the mail address was bad at 
the beginning.

I still think as a general principle, that failing fast and early is 
preferable to slow and late.  that's where the email address 
pre-validation idea came from.

As for the other one of message notification.  I also think keeping the 
human in the loop has all manner of benefits.  With HTTP it may be 
easier to see, since people hit the refresh button over and over if the 
page doesn't load.  We built a variant of Chrome that worked with our 
HTTP status response codes during AV scanning, and the usability was 
greatly improved.

I think as a general principle, keeping people informed of what's going 
on, at a much greater level than we currently do, can save people a lot 
of wasted effort and worry.

Cheers

Adrien

>
>
>   - Chris