Re: [Jmap] new JMAP server for prototyping

Neil Jenkins <neilj@fastmailteam.com> Fri, 28 May 2021 04:48 UTC

Return-Path: <neilj@fastmailteam.com>
X-Original-To: jmap@ietfa.amsl.com
Delivered-To: jmap@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C8CB73A1714 for <jmap@ietfa.amsl.com>; Thu, 27 May 2021 21:48:01 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.096
X-Spam-Level:
X-Spam-Status: No, score=-2.096 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_BLOCKED=0.001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=fastmailteam.com header.b=SQDsK/m5; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=sZ3H2CpS
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VOPuWg8LWWIp for <jmap@ietfa.amsl.com>; Thu, 27 May 2021 21:47:56 -0700 (PDT)
Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C06253A1717 for <jmap@ietf.org>; Thu, 27 May 2021 21:47:56 -0700 (PDT)
Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 562F85C017D for <jmap@ietf.org>; Fri, 28 May 2021 00:47:55 -0400 (EDT)
Received: from imap43 ([10.202.2.93]) by compute3.internal (MEProxy); Fri, 28 May 2021 00:47:55 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= fastmailteam.com; h=mime-version:message-id:in-reply-to :references:date:from:to:subject:content-type; s=fm2; bh=gy/zOCZ 3vkGCJBxB1L6q2xlus2YqnUBTeTuKqHP5s1M=; b=SQDsK/m5EudnUIQGjQVWMkd oGnM99LZsymg3ZlOHC5spjUyWVFD/VNDp0swPix3sX0J8T+72fM31e6o+vzoCzl1 LgAaX0oMfHKqrhZ0LaMjeKxa5pHkU4glXrvIvy0ai2vSPkxcCv9YacGnjTXOzPnS SRdqsPZEJ9rFn6t9vXFPEIazu/yhqvZVnRFAjjXviYMfqazSF7J5PxfvE3eodrYF aUwIuAp+UsspwvPALwqVeE+ib3M4bRZ4FzZKXbCH8XswC5WkaNNxpPjXRYZeWZuI 4lICJG6gmStohsyynLoXkFyEy4aIXL/1gZXp2saZx1syf8EwKLZrlpCuBov1Upg= =
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=gy/zOC Z3vkGCJBxB1L6q2xlus2YqnUBTeTuKqHP5s1M=; b=sZ3H2CpSXWUhnPCkBffv+G SfGbg6ovaM6km3a9aywIdTxn6Fpq3sfcgQN6V7ucyCH67T6+qcQUCrU2ZidMhPJP gH9J7V1oQof7dX0WvJAM7EHSRnFh9VrBhUr8p2DMIzrAejQM0D7WRqTDjL9U8JEV JVdXXFQn3rWlhgBQB/p9Y3yx5VkoPj/swC+fqqiMcNgFtgw+WH7VtjNPClwjEhzV IJf6bYP8j/u42ssLMTN9OCjbdUkJMh7o5Qf5SYULrrFHV5b/dhwZJNRSBjMF6Hsw 55xSu+B+d+ALN8pXWaf1aUsvDD0ArSXMzNueiYi9RaA26E21b6xZVCWiSEeyFZGA ==
X-ME-Sender: <xms:-nWwYFpE_ocohfTD6k1ZN-UC3dDuVFylxGzRXZviaoYu8wrX5R7NLQ> <xme:-nWwYHoSmq_MoyKFRdXVJWcrQbnIjOkvEpaO58rv99I-YCXvKuRYz2jthIw8gffyP gSWlajw9qgngw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvdekiedgkeeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepofgfggfkjghffffhvffutgesrgdtreerreerjeenucfhrhhomhepfdfpvghi lhculfgvnhhkihhnshdfuceonhgvihhljhesfhgrshhtmhgrihhlthgvrghmrdgtohhmqe enucggtffrrghtthgvrhhnpeehuefhudejtdeiveekvdfhfffgleeflefhfeekhefhkeel kefhfeeufeevffejieenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrih hlfhhrohhmpehnvghilhhjsehfrghsthhmrghilhhtvggrmhdrtghomh
X-ME-Proxy: <xmx:-nWwYCPSC_3DcarC2-fBi2KRwaKnM4_fLg37AZc9q3VizZULoOc3-g> <xmx:-nWwYA5X-7MLGi-1l_sNGjFaVBn5DdbYbJ_4q-Y1THPKWMKVALvaHQ> <xmx:-nWwYE5pZhNzB2AfLxxxnZ3z8OGSRb-zFxA830OnqI9EP_o6eQmgnA> <xmx:-3WwYGEgjccpSUd0KxcUcXSJEyHqP-n4ByyjoUgEWwvCzRV4keJfEw>
Received: by mailuser.nyi.internal (Postfix, from userid 501) id DDF5CAC005B; Fri, 28 May 2021 00:47:54 -0400 (EDT)
X-Mailer: MessagingEngine.com Webmail Interface
User-Agent: Cyrus-JMAP/3.5.0-alpha0-701-g78bd539edf-fm-ubox-20210517.001-g78bd539e
Mime-Version: 1.0
Message-Id: <590c785d-7e1a-420e-9f2a-5989e6a76c04@dogfood.fastmail.com>
In-Reply-To: <20210526222836.GA1792@eh>
References: <20210526222836.GA1792@eh>
Date: Fri, 28 May 2021 14:47:14 +1000
From: "Neil Jenkins" <neilj@fastmailteam.com>
To: "IETF JMAP Mailing List" <jmap@ietf.org>
Content-Type: multipart/alternative; boundary=5a85fbaa7f6d4eb5ae2e1fa04b768865
Archived-At: <https://mailarchive.ietf.org/arch/msg/jmap/N0iGFncv2wsPCyQ3zrTb5xCjY1A>
Subject: Re: [Jmap] new JMAP server for prototyping
X-BeenThere: jmap@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: JSON Message Access Protocol <jmap.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/jmap>, <mailto:jmap-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/jmap/>
List-Post: <mailto:jmap@ietf.org>
List-Help: <mailto:jmap-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/jmap>, <mailto:jmap-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 28 May 2021 04:48:02 -0000

On Thu, 27 May 2021, at 08:28, Jamey Sharp wrote:
> I suppose it's not unreasonable, but I can't find any hint in the 
> specification that the keys of `update` or the entries of `destroy` can 
> be creation id references. The most relevant text I can find is this:
> 
>     Some records may hold references to other records (foreign keys). 
>     That reference may be set (via create or update) in the same request 
>     as the referenced record is created.  To do this, the client refers 
>     to the new record using its creation id prefixed with a "#".
> 
> My reading is that creation id references can only appear within a 
> record, either inside the `Foo` in `create`, or inside the `PatchObject` 
> in `update`. Am I missing something?

Hmm, yes, it looks like we never made it explicit in the spec. It's a rare edge case, but possibly worth an errata too to be explicit.

> That's a great example for me to keep in mind but it didn't make me less 
> confused. I've figured out what's bothering me though.
> 
> Let's say the client has cached a prefix of the results, and then some 
> of those items are destroyed.
> 
> old: ["A", "B", "C", "D", ...]
> new: ["B", "D", ...]
> 
> Say upToId is "D"; you said I should look for it in the new result, 
> where it is at index 1. Then 'any ids that were removed but have a 
> higher index than "upToId" SHOULD be omitted.' Which means the removed 
> list is permitted to contain only "A", right?

That was not the intention, but I agree the spec is misleading here, to the point I think it is worth doing an errata; I'll do that too.

> Here's what I think are the minimum necessary rules for upToId; I'd 
> appreciate any comments on whether I've got this right:
> 
>      If an "upToId" is supplied and existed in the old results, any ids 
>      that were removed but had a higher index than "upToId" previously 
>      did SHOULD be omitted. All others MUST NOT be omitted.
> 
>      If an "upToId" is supplied and existed in the old results, the last 
>      id which was not removed, and which previously had an index no 
>      higher than "upToId" did, is the client's last cached id. Any ids 
>      which were added but now have a higher index than the client's last 
>      cached id does SHOULD be omitted. All others MUST NOT be omitted.
> 
> This text might be more clear if written from the point of view of the 
> client's requirements rather than restrictions on the server:
> 
>      If an "upToId" is supplied, any ids which the server is certain did 
>      not appear at or before "upToId" in the old results SHOULD be 
>      omitted.
> 
>      If an "upToId" is supplied, any ids which are not necessary to 
>      return the client's cache to a valid prefix of the new results, 
>      after removing the ids given in "removed", SHOULD be omitted.

That's absolutely correct, and was the intention here.

(The description in the spec is kind-of true, but only because it's grounded in the idea of a particular implementation where you have "tombstone" records for the deleted items that you can sort into the new query, so it's the index of *those* that are lower than the upToId. It's not that important, the spec is wrong and we need an errata, but I know why we thought it made sense when it was written!)

> I think any mention of sort/filter mutability here is an unnecessary 
> constraint on server behavior, as is requiring upToId to be present in 
> the new results.

I agree. That's particular implementation details leaking into the spec again.

Thanks again for the great feedback! As an Area Director put it to me, if there's no erratas it doesn't mean a spec is perfect, it just means no one's reading it. 🙂

Neil.