Re: [Jmap] new JMAP server for prototyping

Neil Jenkins <> Fri, 28 May 2021 04:48 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id C8CB73A1714 for <>; Thu, 27 May 2021 21:48:01 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.096
X-Spam-Status: No, score=-2.096 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_BLOCKED=0.001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key) header.b=SQDsK/m5; dkim=pass (2048-bit key) header.b=sZ3H2CpS
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id VOPuWg8LWWIp for <>; Thu, 27 May 2021 21:47:56 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id C06253A1717 for <>; Thu, 27 May 2021 21:47:56 -0700 (PDT)
Received: from compute3.internal (compute3.nyi.internal []) by mailout.nyi.internal (Postfix) with ESMTP id 562F85C017D for <>; Fri, 28 May 2021 00:47:55 -0400 (EDT)
Received: from imap43 ([]) by compute3.internal (MEProxy); Fri, 28 May 2021 00:47:55 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=; h=mime-version:message-id:in-reply-to :references:date:from:to:subject:content-type; s=fm2; bh=gy/zOCZ 3vkGCJBxB1L6q2xlus2YqnUBTeTuKqHP5s1M=; b=SQDsK/m5EudnUIQGjQVWMkd oGnM99LZsymg3ZlOHC5spjUyWVFD/VNDp0swPix3sX0J8T+72fM31e6o+vzoCzl1 LgAaX0oMfHKqrhZ0LaMjeKxa5pHkU4glXrvIvy0ai2vSPkxcCv9YacGnjTXOzPnS SRdqsPZEJ9rFn6t9vXFPEIazu/yhqvZVnRFAjjXviYMfqazSF7J5PxfvE3eodrYF aUwIuAp+UsspwvPALwqVeE+ib3M4bRZ4FzZKXbCH8XswC5WkaNNxpPjXRYZeWZuI 4lICJG6gmStohsyynLoXkFyEy4aIXL/1gZXp2saZx1syf8EwKLZrlpCuBov1Upg= =
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=gy/zOC Z3vkGCJBxB1L6q2xlus2YqnUBTeTuKqHP5s1M=; b=sZ3H2CpSXWUhnPCkBffv+G SfGbg6ovaM6km3a9aywIdTxn6Fpq3sfcgQN6V7ucyCH67T6+qcQUCrU2ZidMhPJP gH9J7V1oQof7dX0WvJAM7EHSRnFh9VrBhUr8p2DMIzrAejQM0D7WRqTDjL9U8JEV JVdXXFQn3rWlhgBQB/p9Y3yx5VkoPj/swC+fqqiMcNgFtgw+WH7VtjNPClwjEhzV IJf6bYP8j/u42ssLMTN9OCjbdUkJMh7o5Qf5SYULrrFHV5b/dhwZJNRSBjMF6Hsw 55xSu+B+d+ALN8pXWaf1aUsvDD0ArSXMzNueiYi9RaA26E21b6xZVCWiSEeyFZGA ==
X-ME-Sender: <xms:-nWwYFpE_ocohfTD6k1ZN-UC3dDuVFylxGzRXZviaoYu8wrX5R7NLQ> <xme:-nWwYHoSmq_MoyKFRdXVJWcrQbnIjOkvEpaO58rv99I-YCXvKuRYz2jthIw8gffyP gSWlajw9qgngw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvdekiedgkeeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepofgfggfkjghffffhvffutgesrgdtreerreerjeenucfhrhhomhepfdfpvghi lhculfgvnhhkihhnshdfuceonhgvihhljhesfhgrshhtmhgrihhlthgvrghmrdgtohhmqe enucggtffrrghtthgvrhhnpeehuefhudejtdeiveekvdfhfffgleeflefhfeekhefhkeel kefhfeeufeevffejieenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrih hlfhhrohhmpehnvghilhhjsehfrghsthhmrghilhhtvggrmhdrtghomh
X-ME-Proxy: <xmx:-nWwYCPSC_3DcarC2-fBi2KRwaKnM4_fLg37AZc9q3VizZULoOc3-g> <xmx:-nWwYA5X-7MLGi-1l_sNGjFaVBn5DdbYbJ_4q-Y1THPKWMKVALvaHQ> <xmx:-nWwYE5pZhNzB2AfLxxxnZ3z8OGSRb-zFxA830OnqI9EP_o6eQmgnA> <xmx:-3WwYGEgjccpSUd0KxcUcXSJEyHqP-n4ByyjoUgEWwvCzRV4keJfEw>
Received: by mailuser.nyi.internal (Postfix, from userid 501) id DDF5CAC005B; Fri, 28 May 2021 00:47:54 -0400 (EDT)
X-Mailer: Webmail Interface
User-Agent: Cyrus-JMAP/3.5.0-alpha0-701-g78bd539edf-fm-ubox-20210517.001-g78bd539e
Mime-Version: 1.0
Message-Id: <>
In-Reply-To: <20210526222836.GA1792@eh>
References: <20210526222836.GA1792@eh>
Date: Fri, 28 May 2021 14:47:14 +1000
From: "Neil Jenkins" <>
To: "IETF JMAP Mailing List" <>
Content-Type: multipart/alternative; boundary=5a85fbaa7f6d4eb5ae2e1fa04b768865
Archived-At: <>
Subject: Re: [Jmap] new JMAP server for prototyping
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: JSON Message Access Protocol <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 28 May 2021 04:48:02 -0000

On Thu, 27 May 2021, at 08:28, Jamey Sharp wrote:
> I suppose it's not unreasonable, but I can't find any hint in the 
> specification that the keys of `update` or the entries of `destroy` can 
> be creation id references. The most relevant text I can find is this:
>     Some records may hold references to other records (foreign keys). 
>     That reference may be set (via create or update) in the same request 
>     as the referenced record is created.  To do this, the client refers 
>     to the new record using its creation id prefixed with a "#".
> My reading is that creation id references can only appear within a 
> record, either inside the `Foo` in `create`, or inside the `PatchObject` 
> in `update`. Am I missing something?

Hmm, yes, it looks like we never made it explicit in the spec. It's a rare edge case, but possibly worth an errata too to be explicit.

> That's a great example for me to keep in mind but it didn't make me less 
> confused. I've figured out what's bothering me though.
> Let's say the client has cached a prefix of the results, and then some 
> of those items are destroyed.
> old: ["A", "B", "C", "D", ...]
> new: ["B", "D", ...]
> Say upToId is "D"; you said I should look for it in the new result, 
> where it is at index 1. Then 'any ids that were removed but have a 
> higher index than "upToId" SHOULD be omitted.' Which means the removed 
> list is permitted to contain only "A", right?

That was not the intention, but I agree the spec is misleading here, to the point I think it is worth doing an errata; I'll do that too.

> Here's what I think are the minimum necessary rules for upToId; I'd 
> appreciate any comments on whether I've got this right:
>      If an "upToId" is supplied and existed in the old results, any ids 
>      that were removed but had a higher index than "upToId" previously 
>      did SHOULD be omitted. All others MUST NOT be omitted.
>      If an "upToId" is supplied and existed in the old results, the last 
>      id which was not removed, and which previously had an index no 
>      higher than "upToId" did, is the client's last cached id. Any ids 
>      which were added but now have a higher index than the client's last 
>      cached id does SHOULD be omitted. All others MUST NOT be omitted.
> This text might be more clear if written from the point of view of the 
> client's requirements rather than restrictions on the server:
>      If an "upToId" is supplied, any ids which the server is certain did 
>      not appear at or before "upToId" in the old results SHOULD be 
>      omitted.
>      If an "upToId" is supplied, any ids which are not necessary to 
>      return the client's cache to a valid prefix of the new results, 
>      after removing the ids given in "removed", SHOULD be omitted.

That's absolutely correct, and was the intention here.

(The description in the spec is kind-of true, but only because it's grounded in the idea of a particular implementation where you have "tombstone" records for the deleted items that you can sort into the new query, so it's the index of *those* that are lower than the upToId. It's not that important, the spec is wrong and we need an errata, but I know why we thought it made sense when it was written!)

> I think any mention of sort/filter mutability here is an unnecessary 
> constraint on server behavior, as is requiring upToId to be present in 
> the new results.

I agree. That's particular implementation details leaking into the spec again.

Thanks again for the great feedback! As an Area Director put it to me, if there's no erratas it doesn't mean a spec is perfect, it just means no one's reading it. 🙂