IETF Logo Mail Archive

Re: [Jmap] recommendations for gigantic mailboxes

Neil Jenkins <neilj@fastmailteam.com> Fri, 26 June 2020 05:54 UTC

Return-Path: <neilj@fastmailteam.com>
X-Original-To: jmap@ietfa.amsl.com
Delivered-To: jmap@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9D7073A0033 for <jmap@ietfa.amsl.com>; Thu, 25 Jun 2020 22:54:04 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Level:
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=fastmailteam.com header.b=KSHq//xD; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=uMJvbouR
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7dwR-erBx7ZA for <jmap@ietfa.amsl.com>; Thu, 25 Jun 2020 22:54:02 -0700 (PDT)
Received: from wout4-smtp.messagingengine.com (wout4-smtp.messagingengine.com [64.147.123.20]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 98B713A0029 for <jmap@ietf.org>; Thu, 25 Jun 2020 22:54:02 -0700 (PDT)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id A356894A for <jmap@ietf.org>; Fri, 26 Jun 2020 01:54:01 -0400 (EDT)
Received: from imap28 ([10.202.2.78]) by compute4.internal (MEProxy); Fri, 26 Jun 2020 01:54:01 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= fastmailteam.com; h=mime-version:message-id:in-reply-to :references:date:from:to:subject:content-type; s=fm3; bh=cTVMx4h Vg38UdKd2K9uIyw4mcWfh3TQpo6CXo2D/7WY=; b=KSHq//xDg7/qkz0Ae0z2fnk RlAX+m9pvUC9P6tTJkmLPrpaGVK8CLMPl1ZnhgO8db/NwfaQZ4KxPy1qur0NxHG+ P9ew6KGKB35+pFTBiBGiBZT0cpQShi9HZqurDBLOLTY+WzQHz4ZFXCGXLZ4HjQGc TEjeCj0VlyWrVDvpTEZ6LvC2jKECKkt1VLlyXQWzg5cnPHyU2nkXxZkF+79YgEKH adTot6+ZPXLkwW9i9cjHpigrWupEseZpFPsOyvRWuebE0r/sIIoTUZNObnJtWDDv dJdaGzaaykWWlaIvBWW9nWhRmvj6tw8S+iPVymJb6FEPGoHtMcor2S4//DXsgNQ= =
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=cTVMx4 hVg38UdKd2K9uIyw4mcWfh3TQpo6CXo2D/7WY=; b=uMJvbouR9oH9/0V2YVXejK GcqeroXIIHFZqQEda8Jnm+RXdcB1HE7JR4jCuRogrY56kqVwIpNsGu0sgYbnJMc5 u7BEX9jDSPds+SMh2Hsha+8XBMPsFFg7rmGZ1Gr243uULwORMLxizRIP9oaH3p81 JOpl8Vv6bKkfLn/8GgJa4oG8zkUWgzj4BuAovL+7TeecUKODPBDZoj3puMbUQqOi 18OSuo0eHrQhS71fJK3I24lH2i1jh/km1OEhHxdrei1Jn7YGR1i9NAhyb43LTkts pSiyIGX90QQFAq2L3cx6tiBEP3+uQ7TGFfN5BQV4TlZZwE+lRVE1fRrD7Qjioj9g ==
X-ME-Sender: <xms:eY31XkKtAFjMkvMdk1xMfD9sEqPr6RllTQZYFNaA5mwLTzI9dmWzdg>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrudeltddgleelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepofgfggfkjghffffhvffutgesrgdtreerreertdenucfhrhhomhepfdfpvghi lhculfgvnhhkihhnshdfuceonhgvihhljhesfhgrshhtmhgrihhlthgvrghmrdgtohhmqe enucggtffrrghtthgvrhhnpeffgfejtdeuvdevtdetkeejjefhieekueejleehgfdutddu heektdffffevlefgvdenucffohhmrghinhepjhhmrghprdhiohenucevlhhushhtvghruf hiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehnvghilhhjsehfrghsthhmrghi lhhtvggrmhdrtghomh
X-ME-Proxy: <xmx:eY31XkItwVAWCAgRoIsG3zmCwr71GvYMKqZvaA8Y0MPA60IF6u_Z_Q> <xmx:eY31XkvpId1Yi1fnuyb_LTW3i92nJqW4f0z7OyPeUgTZ7V4rZ9r0oA> <xmx:eY31XhYjEf9zTRxeRr4mx6b8gZyuXpk_J-LULZRKe0_vTvDMVnZ4ZA> <xmx:eY31XjopHZmGbH76O_hU073GHHyUJt8-c2WB8I0ezzTeneEj-QOjoA>
Received: by mailuser.nyi.internal (Postfix, from userid 501) id D2A7624009E; Fri, 26 Jun 2020 01:54:00 -0400 (EDT)
X-Mailer: MessagingEngine.com Webmail Interface
User-Agent: Cyrus-JMAP/3.3.0-dev0-565-g8923300-fm-idx2020-20200622.001-g89233000
Mime-Version: 1.0
Message-Id: <28189879-e304-49f4-8876-4f46b2af7351@beta.fastmail.com>
In-Reply-To: <20200625090336.GA32628@dcvr>
References: <20200625090336.GA32628@dcvr>
Date: Fri, 26 Jun 2020 15:53:53 +1000
From: Neil Jenkins <neilj@fastmailteam.com>
To: IETF JMAP Mailing List <jmap@ietf.org>
Content-Type: multipart/alternative; boundary="f77c14735b6c4eb9a84f251bdf83396a"
Archived-At: <https://mailarchive.ietf.org/arch/msg/jmap/fmDt8LeY0rbgNM8kRSBcelJfbW4>
Subject: Re: [Jmap] recommendations for gigantic mailboxes
X-BeenThere: jmap@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: JSON Message Access Protocol <jmap.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/jmap>, <mailto:jmap-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/jmap/>
List-Post: <mailto:jmap@ietf.org>
List-Help: <mailto:jmap-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/jmap>, <mailto:jmap-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 26 Jun 2020 05:54:05 -0000

On Thu, 25 Jun 2020, at 19:03, Eric Wong wrote:
> TL;DR: would it be reasonable to expose 10 million messages in a
> single JMAP mailbox?

Yes, I don't see why not.

> So, JMAP is fresh ground; so I wonder if having a single mailbox
> on the order of 10 million messages is reasonable for JMAP clients.

I mean, this is really down to implementation. There's nothing in the protocol that makes this particularly problematic, but if it's trying to save every message locally then it could run into trouble depending on its choice of on-disk format. But JMAP makes it easy to page in just the portion of the mailbox you are currently interested in viewing, so as long as the client is working in this mode it should be fine.

As JMAP is very new, there are not many clients to test against yet to see how they work with such a large volume. You can point the demo webmail <https://jmap.io/jmap-demo-webmail/> client at a server to test with; I've tried it with the order of ~1 million messages but not with ~10 million. This is not caching anything locally other than temporarily for performance reasons, which is a good model for accessing this kind of data.

Neil.

v2.23.0 | Report a Bug | By Email