Re: [Jmap] Hacker News Discussion - getMailboxes single data set vs tree primitives or paging

Bron Gondwana <brong@fastmail.fm> Tue, 09 May 2017 00:29 UTC

Return-Path: <brong@fastmail.fm>
X-Original-To: jmap@ietfa.amsl.com
Delivered-To: jmap@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AE724126DEE for <jmap@ietfa.amsl.com>; Mon, 8 May 2017 17:29:13 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.32
X-Spam-Level:
X-Spam-Status: No, score=-1.32 tagged_above=-999 required=5 tests=[BAYES_05=-0.5, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=fastmail.fm header.b=gaLduroC; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=gkENJvH7
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id O6ffkw3Yj99U for <jmap@ietfa.amsl.com>; Mon, 8 May 2017 17:29:12 -0700 (PDT)
Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 61011126BF0 for <jmap@ietf.org>; Mon, 8 May 2017 17:29:12 -0700 (PDT)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id D379A2037D for <jmap@ietf.org>; Mon, 8 May 2017 20:29:11 -0400 (EDT)
Received: from web2 ([10.202.2.212]) by compute6.internal (MEProxy); Mon, 08 May 2017 20:29:11 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.fm; h= content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; bh=cAZbVPpLWJpeS3vxYtM/6HXFRZPVn eW1a/gmeomliM0=; b=gaLduroChUQxbIzEzIk2fmZo98NcDa4hz0rm1yxfGn7Gl YweuJBaehXCawY7ruqumrslUjwGLj+VL6KRhgcOsn+6B4NQYKwOQl9TPCUUjPrPH tavnfSdIPwFDn7LJDESoRPVu/k8RI35sop3P44HFUvVkW2RxAxrjDIVdlocTaB0/ Dfox98AVcjtcFg10XNju9XTOgkneCom4MgySHsniEqUvOSkGoPwnHLI9jYj4XqNw vk+i015aptDg1sAs+7V88W0lf72qqmJga2a9IqP7vPNX/zZvcKIhw4/73Hi9mWuZ 7sNRGYQJw4xxoIUNCgz7zb4zgM3S1AIjle9QbGg+g==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=cAZbVP pLWJpeS3vxYtM/6HXFRZPVneW1a/gmeomliM0=; b=gkENJvH7Qa+chDBGtJpegY vMJW1qUC3f0w5Uuqbs0+8foBGnWN5rioZSei/sRpHFgbILRoTpZfgu6ZLDJRY8K0 N1K2V0EM1VLD8PjROiW7IE73xi6pETqIft8VXllo98XuhRXIXOX4rYU3yHkNabgh sp7jdQ2YOrNUhEV1y9c2TxFAsdOC0r0We3rLWXkDAw/b0kU+2wi0PmHxZWTYiYDJ xo7giV0iUfN1jrsZ/HxrcRqf1RXOu2Mvidu4Xus9GfoW/wc8tfoEZlgXXesx5JO2 KbAyc6Tnf1/z+RK7Vbv12mrIJ5eZmKHxGuFEix5uRKxNDudHZYiRy7qQT+JLJGMA ==
X-ME-Sender: <xms:Vw0RWXT_SKKzQriSTGQxS-mSBxIEKo0lDh-meMpXQdLQqbs5QfgQsw>
Received: by mailuser.nyi.internal (Postfix, from userid 99) id AC67862704; Mon, 8 May 2017 20:29:11 -0400 (EDT)
Message-Id: <1494289751.131921.970152157.2A520058@webmail.messagingengine.com>
From: Bron Gondwana <brong@fastmail.fm>
To: jmap@ietf.org
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="utf-8"
X-Mailer: MessagingEngine.com Webmail Interface - ajax-6cc55fe1
References: <20170507123912.GA22520@snake.grepular.com> <FF749B75-173D-4086-BAE6-AFB897D5F676@fugue.com> <1494224344.1447245.969038872.78D5E02D@webmail.messagingengine.com> <1494249269.88461.969397104.25D88F0A@webmail.messagingengine.com> <f43499ea-cc8d-4104-b820-d3144affb685@gulbrandsen.priv.no>
In-Reply-To: <f43499ea-cc8d-4104-b820-d3144affb685@gulbrandsen.priv.no>
Date: Tue, 09 May 2017 10:29:11 +1000
Archived-At: <https://mailarchive.ietf.org/arch/msg/jmap/gaEhQ50pmqzR68agrhOJIKo72ZE>
Subject: Re: [Jmap] Hacker News Discussion - getMailboxes single data set vs tree primitives or paging
X-BeenThere: jmap@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: JSON Message Access Protocol <jmap.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/jmap>, <mailto:jmap-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/jmap/>
List-Post: <mailto:jmap@ietf.org>
List-Help: <mailto:jmap-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/jmap>, <mailto:jmap-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 09 May 2017 00:29:14 -0000

On Tue, 9 May 2017, at 00:14, Arnt Gulbrandsen wrote:
> It's not clear that mailboxes really need to be hierarchical, and we've 
> seen that the hierarchy stuff wasn't really well implemented by IMAP 
> clients.
> 
> It seems plausible that polling the complete list oftenish is unacceptable 
> for at least some users, but an IMAP-like hierarchy is not the only way to 
> cut bandwidth usage. Retrieving it once and then asking for updates would 
> work too, like JMAP's in-mailbox updates.
> 
> Servers that don't have a global modseq or some other generation counter 
> would have to fall back to sending all/many mailboxes.

So if we have just a single sort (fullpath) and an optional filter on parentId plus windowing we can handle either nested or flat folder structures very easily and handle upteen million mailboxes comfortably as well.

It seems viable to implement this even if you're doing depth-first traversal of a folder listing on a filesystem with no other smarts required, you can always use the path of the folder as the "id", though it will obviously not survive renames, so you miss that benefit.

Bron.

-- 
  Bron Gondwana
  brong@fastmail.fm