Re: [Jmap] Review of draft-ietf-jmap-core-00.txt and draft-ietf-jmap-mail-00.txt

[Bron Gondwana <[e-mail]>]

On Fri, 14 Jul 2017, at 23:10, Chris Newman wrote:
> 2.x [ ... ]

I haven't created any tickets for this as the discussion is mostly superseded by "split authentication out".  If we do go ahead with our own authentication document within JMAP, I'll come back to this for notes.
> 3.2. API structure
> 
> This model only allows extensibility by adding methods or altering
> method parameters. There's no way to attach properties to the entire
> API request. There are a number of potentially interesting
> request-level properties such as "fail on error" (stop on first error)> or "transaction"/"atomic" (either perform all methods successfully or> none of them). If you made the top-level an object and the array of
> methods a property of the top-level object, you'd get request-level
> extensibility.

https://github.com/jmapio/jmap/issues/115

> 3.3. Errors
> 
> "transport level" => This seems wrong because TCP is typically the
> transport level for HTTP-based protocols, but returning a TCP-level
> error is not desirable here. Why not just simplify this to one
> sentence saying the error is returned at the HTTP level.

https://github.com/jmapio/jmap/issues/116

> 3.6 Concurrency
> 
> For potentially long-running operations like search and
> copy/move-messages, the creates an inappropriate and unnecessary
> implementation burden on the server. In general, the goal is that the> server should present a consistent view of data to the client, even
> if it's slightly out-of-date by the time the method completes. It's
> fine to state this and even say the ability of a server to present a
> consistent view to clients is a quality-of-implementation issue for
> the server. But don't force the presence of a hard locking mechanism
> into the server implementation unless it's absolutely necessary as
> that harms server scalability.
> 
> Normative restrictions on concurrency should be specified for those
> methods that need them and be minimally restrictive. For example, a
> multi-message move operation should either move all messages or none
> of them, and the messages should retain their relative ordering in the> destination folder. But additional restrictions are not needed to
> present a good client experience.

https://github.com/jmapio/jmap/issues/117

I would probably implement a lot of these things as an MVCC-style snapshot view for read operations.  Write is trickier depending how you do your state management, do you do either need to lock against changes entirely so you can give back a new state string at the end, or you wind up in cases where creating a state string is tricky.
> 3.8 Date datatype
> 
> Are you sure you want to pin the timezone of timestamps to UTC? This
> potentially loses information.

https://github.com/jmapio/jmap/issues/118

> 3.10.1 getFoos
> 
> state: need to specify scope and duration of this string. I'd guess it> is scoped to the specific getFoos request, so there's no requirement
> for portability of this state token to other users or types. But
> it'd be better if it's quite persistent (e.g., lasts a month or until> the client issues an update).
> 
> 3.10.2 getFooUpdates
> 
> fetchRecords & fetchRecordProperties
> 
> Instead of two separate parallel properties, how about defining
> fetchRecords as having the same structure as a getFoos object, but
> with appropriate defaults for accountId and ids. This way,
> extensions to the getFoos object naturally apply to the fetchRecords
> value object.
> 
> If the cannotCalculateChanges error occurs too frequently, the server> becomes problematic for client performance. A quality server should
> strive to persist state tokens for several clients per user for at
> least a month.
> 
> 3.10.3. setFoos
> 
>> Each create, update or destroy is considered an atomic unit.
> I suggest:
> Each create, update or destroy list item is considered an atomic unit.> 
>> record, so it can substitute in the correct value if necessary in
>> later method calls. The type
> dangling end of paragraph.
> 
>> a separate creation id -> id map MUST be kept for each type.
> 
> I assume these must only be kept for the duration of a single API
> request? It would be helpful to say that explicitly.

https://github.com/jmapio/jmap/issues/119 (state string SHOULD persist e.g. 1 month)https://github.com/jmapio/jmap/issues/120 (setFoos and idmap duration)
https://github.com/jmapio/jmap/issues/121 (fetchRecords as getFoos with defaults)
> 3.11.1. getFooList
> 
> position: it's very useful to specify position relative to the end of> the list. For mail, the following search is common: "the 50 most
> recent not-deleted messages". I suggest making -50, the 50th position> from the end of the list (the way Python list slicing works). Failure> to add support like this makes RFC 5267 partial results far less
> useful than it could have been. For the response, always using
> non-negative positioning makes sense.

This already got created as:

https://github.com/jmapio/jmap/issues/114

> 5. Uploading binary data
> 
> Can you make the header "JMAP-AccountId" instead of "X-JMAP-AccountId"> per RFC 6648?

This is already covered by:

https://github.com/jmapio/jmap/issues/92

> 5.1 file upload
> 
> The 'type' field is not well defined. Is this the Content-Type header> field with CFWS collapsed to SP and RFC 2231/2047 encoding removed? Or> should the content-type parameters be split out into a separate
> property with an object as a value?
> 
> size: For IMAP compatibility, this needs to allow sizes up to 2^32-1,
Size is an interesting area of discussion, what with LITERAL- and various server restrictions.  I'm feeling lazy so I just made one ticket for both of these:
https://github.com/jmapio/jmap/issues/122

> 6.2 Web hook
> 
>> The JMAP server MUST follow any redirects.
> 
> Maybe I'm a bit paranoid, but I suggest: The JMAP server
> implementation MUST be capable of following at least one redirect, but> MAY choose not to do so for configured policy reasons.
> 
> 6.2.1. setPushCallback
> 
> I think this needs another error code:
> 
> policyDenied: The server is not willing to connect to the specified
> URL for policy reasons.

https://github.com/jmapio/jmap/issues/123

> ====
> 
> draft-ietf-jmap-mail-00
> 
> 2. Mailboxes
> 
> Need to distinguish JMAP message id from RFC 5322 Message-ID
> header. Duplicate 5322 message ids occur, so it's helpful to have a
> JMAP message ID that's unique within an account.

This got made into a ticket during the meeting notes regarding naming:

https://github.com/jmapio/jmap/issues/102

> role: I support proposal to either remove 'outbox' (or make it
> optional) and replace with MessageDelivery/MessageSubmission object.
> I'd also like to see this more closely aligned with RFC 6154 (at least> use the same names). We should discuss issue of multiple folders with> same role; I like the simplicity of not having that, but RFC 6154 was> written as it is for a reason; we should make sure that reason doesn't> apply to JMAP before changing that design.

Wow, for all that we've talked about this, we don't actually have a ticket for it!
https://github.com/jmapio/jmap/issues/124


> sortOrder: don't hand-wave the locale-specific character order problem> as that breaks interoperability. I suggest abstracting the problem by> associating a collation (RFC 4790) with each account. If there's a
> collation associated with the account, that produces a deterministic
> order. Otherwise, permission has been given to be
> non-deterministic. Server operators can derive the collation algorithm> name from the user's language when they register their account; then
> it will work even on a client with the wrong locale configured.

https://github.com/jmapio/jmap/issues/125

> mustBeOnlyMailbox: I don't understand the distinction. Is it
> important?

I haven't created anything for this, maybe Neil can just reply directly.  This is for servers which don't support having the same message in multiple mailboxes.
> "may*": Can we just replace this with a "myrights" per RFC 4314? If
> you want to drop the rights grouping model of RFC 4314 to be simpler,> that's fine with me, but I'd prefer to keep the set of rights the same.> 
> 2.3.3. updating mailboxes
> 
> myrights (may*) isn't immutable; it's server-generated (possibly based> on an IMAP ACL from RFC 4314) so the client can't directly update
> it. I think it's fine to leave ACLs as an extension to base JMAP but
> having the equivalent of myrights in the base spec is a good decision.
https://github.com/jmapio/jmap/issues/126


> 3.1.1 filtering
> 
> This really needs to be semantically closer to IMAP. In particular,
> support for extensible flags is needed.
> 
> Does this need an extensibility mechanism? A way to filter the search> that's ignored if not understood by the server (like non-critical LDAP> search controls)?

Extending filtering would be done with a "using jmapesearch" or something, and that spec would define exactly how it extended filtering, so clients would opt in to search methods that the server claimed to support.
> 3.1.2. Sorting
> 
> We have a collation model (RFC 4790) that covers sorting. I think
> servers need to implement either RFC 5051 or the Unicode Collation
> Algorithm as a minimum/default. Allowing a collation to be specified
> would be good for interoperabiltiy/predictability.

updated and renamed #125 :)

> 5. Messages
> 
> Overall, I find this to be almost as problematic as the sending model of> the specification. I understand the goal of trying to make a minimal
> JMAP client work without having to implement 5322/MIME/2047/2231. But> the problem with creating a data format gateway is you've got the union> of the problems and the intersection of the functionality. And while
> having a way to fetch the raw blob is nice, that's not great for clients> that want some of the 5322/MIME/2047/2231 functionality that's missing> from JMAP but don't want to be forced to fetch the full message and do a> full parse of it.
> 
> I suggest a different approach. First, define a set of properties that> are largely un-adulterated (similar to IMAP header fetch, IMAP body part> references, etc). Then define properties that try to
> convert/simplify/flatten that data (losing information in the process).> Then clients can choose the level of precision they want. You'll need to> describe the algorithms used to convert/simplify/flatten the data in
> detail. And if you're going to simplify some address fields, you need> the ability to simplify all of them, including ones like
> Message-Disposition-To.
> 
>> or RFC 2047 encoding of non-ASCII characters, MUST be fully decoded
>> into standard UTF-8.
> 
> If you say this, you need to identify the charsets that MUST be
> supported; otherwise the requirement is impossible. You'll also want to> mention RFC 2231.

This is all covered by https://github.com/jmapio/jmap/issues/104 but I've copied this text into a comment.
>> hasAttachment: Boolean Does the message have any attachments?
> 
> You need to define what you mean by an attachment. Do you mean a MIME> part with a Content-Disposition of "attachment", or do you mean
> something else? Be precise.

https://github.com/jmapio/jmap/issues/127

> textBody & htmlBody: I assume these are converted to UTF-8? This model> is not good. I generally want to be presented the plain text body if
> present and the HTML body if the text body is not present. Are you
> unfolding RFC 3676 for this? What if it's S/MIME signed?

Threw this into 104.

> date: the timezone of the sender is relevant information, so if you're> parsing out the date, you should parse out that field as well. For
> submission you need a way to provide the timezone.

and this :)

> 8. Vacation Response
> 
> Can this be moved to an extension? The question of how this interacts> with Sieve, ManageSieve, etc, is tricky.

We talked this one out at dinner on Monday night.  Let me know if you still want a ticket created for it.
> ====
> 
> Missing stuff:
> 
> * email address internationalization
> => the protocol could be defined in such a way that this will mostly
> "just work" (to the degree that's possible). Needs changes to Emailer> object, Message object, etc.
> 
> I don't have time to do it now, but I'll run through the list of IMAP> extensions to come up with more 'missing stuff'.

And likewise with this bit.

Do you have a github user?  Might be easier to add you to the org so you can create/edit tickets yourself!
Bron.

--
  Bron Gondwana, CEO, FastMail Pty Ltd
  [e-mail]