IETF Logo Mail Archive

Re: [Jmap] recommendations for gigantic mailboxes

Bron Gondwana <brong@fastmailteam.com> Fri, 26 June 2020 10:54 UTC

Return-Path: <brong@fastmailteam.com>
X-Original-To: jmap@ietfa.amsl.com
Delivered-To: jmap@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 15E093A1166 for <jmap@ietfa.amsl.com>; Fri, 26 Jun 2020 03:54:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Level:
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=fastmailteam.com header.b=nNCCubPT; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=PyoJc1uU
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kKGBoK4nk1hd for <jmap@ietfa.amsl.com>; Fri, 26 Jun 2020 03:54:57 -0700 (PDT)
Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com [64.147.123.21]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CF84B3A103F for <jmap@ietf.org>; Fri, 26 Jun 2020 03:54:57 -0700 (PDT)
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 1359D828 for <jmap@ietf.org>; Fri, 26 Jun 2020 06:54:57 -0400 (EDT)
Received: from imap28 ([10.202.2.78]) by compute1.internal (MEProxy); Fri, 26 Jun 2020 06:54:57 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= fastmailteam.com; h=mime-version:message-id:in-reply-to :references:date:from:to:subject:content-type; s=fm3; bh=b0kmUD7 aOVkvvhdRACgzRx+PVAgynPB7rp78/VfjSww=; b=nNCCubPTYesJIpGgNgZO5j/ N3i+lEQXRlA3N5xdLHCQ+brknXVDW4qWQm3W8yDNpPZMcXJSvu74V8PTZmjfgJK6 gNfslqObLYFsXQBiWUKy+XGGh6bDuhPmhr4gYvv2hjbVLdGj2NtFSW1/vjS2ldVQ OqCryRldjhYKo/gbgo2CAqpeZT5VQwx8ISRiG46BW3lbsYUpZq05LV15fU7jZGYx skQ38M7k8hw03FRfb7ltFgZLXVnfnrLX5wcP9KrgURO6DXvH8dTUsfyR1jcz6THO nT1xQGoXKtxbP5bsJc7BaD9+eQ3gx3zFwpA6x0uNCFVLDcTyBbWaeup/yGA4cgg= =
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=b0kmUD 7aOVkvvhdRACgzRx+PVAgynPB7rp78/VfjSww=; b=PyoJc1uUHTo8zbZ57RMHWw XPKjyLfQRoRgl7PpdQeKPR3IhtsNIS6w7yxBwebTZx6B3/pqA73EuHJnP7zybL/e ft57649++CqFStNmRw53pxPj0IiJwddiFRY7/S2h5g5sZYV0a+vog4BRQh2aNcbB zTgwRTtv4BsydIk9uyBchKLIJ5sei8J5Sl+dKcGyPH2EUXTo+Ynf6fOV1DwXH1aw JvmqJVnt6GrGeHSFL46TnRCu0pehqBj/VqUuw2IGIEGKOUdel87ChwZHGce0lO/b mi3Ji7URhtFaMY9xUd9XxOiEj124HrRlBmROpTF10ciuKFkAClPaIm3gf9/gyKJw ==
X-ME-Sender: <xms:ANT1XliTC5g5UqvvkaXJLdJqXSPY58gnjcSGK8z8jSMTP9s5OObq_g>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrudeluddgfeeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsegrtd erreerredtnecuhfhrohhmpedfuehrohhnucfiohhnugifrghnrgdfuceosghrohhnghes fhgrshhtmhgrihhlthgvrghmrdgtohhmqeenucggtffrrghtthgvrhhnpedvudeuieehgf dvheeuueejjeeuudfgiefgveetfeelteeffffgtdejjefgueduvdenucevlhhushhtvghr ufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegsrhhonhhgsehfrghsthhmrg hilhhtvggrmhdrtghomh
X-ME-Proxy: <xmx:ANT1XqDoZJA8YItwSozYlKW5rCgjOTRnPbN30RIF3q1wA7nCTFw5tQ> <xmx:ANT1XlEk1050nsVrvEzlOsVaOX4oeLwNKDxVeFJ0l1zUhUkzNuaOUg> <xmx:ANT1XqSoytm_4rFrfQG0HHUTqjfSZPjdG0kxfH7i6Xmge0HgTWgfrw> <xmx:ANT1Xqi5Zu6HIXEFxN1t7OK5mgoYU2oCArPkQkQlECCJd5icT_ZlRw>
Received: by mailuser.nyi.internal (Postfix, from userid 501) id 4AE1A24009E; Fri, 26 Jun 2020 06:54:56 -0400 (EDT)
X-Mailer: MessagingEngine.com Webmail Interface
User-Agent: Cyrus-JMAP/3.3.0-dev0-565-g8923300-fm-idx2020-20200622.001-g89233000
Mime-Version: 1.0
Message-Id: <ce31ca1d-77e1-45eb-8a83-3a12ef6e9aae@dogfood.fastmail.com>
In-Reply-To: <28189879-e304-49f4-8876-4f46b2af7351@beta.fastmail.com>
References: <20200625090336.GA32628@dcvr> <28189879-e304-49f4-8876-4f46b2af7351@beta.fastmail.com>
Date: Fri, 26 Jun 2020 20:54:35 +1000
From: Bron Gondwana <brong@fastmailteam.com>
To: jmap@ietf.org
Content-Type: multipart/alternative; boundary="28043fd996774bccb7728ada23a59f14"
Archived-At: <https://mailarchive.ietf.org/arch/msg/jmap/YIJZtGQCNddEfp55wnqHOYkE5JA>
Subject: Re: [Jmap] recommendations for gigantic mailboxes
X-BeenThere: jmap@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: JSON Message Access Protocol <jmap.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/jmap>, <mailto:jmap-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/jmap/>
List-Post: <mailto:jmap@ietf.org>
List-Help: <mailto:jmap-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/jmap>, <mailto:jmap-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 26 Jun 2020 10:54:59 -0000

On Fri, Jun 26, 2020, at 15:53, Neil Jenkins wrote:
> On Thu, 25 Jun 2020, at 19:03, Eric Wong wrote:
>> TL;DR: would it be reasonable to expose 10 million messages in a
>> single JMAP mailbox?
> 
> Yes, I don't see why not.
> 
>> So, JMAP is fresh ground; so I wonder if having a single mailbox
>> on the order of 10 million messages is reasonable for JMAP clients.
> 
> I mean, this is really down to implementation. There's nothing in the protocol that makes this particularly problematic, but if it's trying to save every message locally then it could run into trouble depending on its choice of on-disk format. But JMAP makes it easy to page in just the portion of the mailbox you are currently interested in viewing, so as long as the client is working in this mode it should be fine.

Yeah, as Neil says - the protocol itself is going to be totally fine. The big issue for you is going to be your data model, particularly if you're trying to do Email/query and Email/queryChanges efficiently.

Overall, it's going to be nicer than IMAP because you don't have to do as much unsolicited work - if the client wants something it will ask for it, and if it doesn't ask, you don't need to create it.

Cheers,

Bron.

--
 Bron Gondwana, CEO, Fastmail Pty Ltd
 brong@fastmailteam.com

v2.23.0 | Report a Bug | By Email