Re: [imap5] Designing a new replacement protocol for IMAP

Bron Gondwana <brong@fastmail.fm> Thu, 16 February 2012 12:41 UTC

Return-Path: <brong@fastmail.fm>
X-Original-To: imap5@ietfa.amsl.com
Delivered-To: imap5@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 24C2121F87E0 for <imap5@ietfa.amsl.com>; Thu, 16 Feb 2012 04:41:53 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.399
X-Spam-Level:
X-Spam-Status: No, score=-3.399 tagged_above=-999 required=5 tests=[AWL=0.200, BAYES_00=-2.599, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tXFgKbLShrXk for <imap5@ietfa.amsl.com>; Thu, 16 Feb 2012 04:41:47 -0800 (PST)
Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by ietfa.amsl.com (Postfix) with ESMTP id BC27B21F87D4 for <imap5@ietf.org>; Thu, 16 Feb 2012 04:41:44 -0800 (PST)
Received: from compute3.internal (compute3.nyi.mail.srv.osa [10.202.2.43]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 9C30D210AF for <imap5@ietf.org>; Thu, 16 Feb 2012 07:41:43 -0500 (EST)
Received: from web1.nyi.mail.srv.osa ([10.202.2.211]) by compute3.internal (MEProxy); Thu, 16 Feb 2012 07:41:43 -0500
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h= message-id:from:to:cc:mime-version:content-transfer-encoding :content-type:in-reply-to:references:subject:date; s=mesmtp; bh= qaK5/8rqlPUaByUwD6JP8lqkU2I=; b=mtouIRfLrP/6nJL+3BV+ren8JoxM/qEw n/1RsvNCesCQnenGcKK0gGwyFjpH8VW/feNX2ncb0dNzvRFQhJfQTJnvM1nCxleN KfY5t+tqn6jywyhIHMOi1EKcS75iAwymm5UMoul4ovDSZQNizDxjYmRAfJoifhEv 3CqFgJJ8/NY=
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=message-id:from:to:cc:mime-version :content-transfer-encoding:content-type:in-reply-to:references :subject:date; s=smtpout; bh=qaK5/8rqlPUaByUwD6JP8lqkU2I=; b=MkB EIUSg3NAvwNp2OHT9NeVDnB3sJpxGUnH+KSPDNKnTpYH2UTs5CbV4El8KLLRIjnS j1m3B2q4WqxK5EKn+Ro8v8pQHqOuAt09R2fj+SA9J9Rw1hoazW5rCZBQsU/Q/2zD 4zpUXXxvg0tKFUD+QMaj7A5zYbij2nccYoHgNNYY=
Received: by web1.nyi.mail.srv.osa (Postfix, from userid 99) id 7439BA0009E; Thu, 16 Feb 2012 07:41:43 -0500 (EST)
Message-Id: <1329396103.8954.140661037328961@webmail.messagingengine.com>
X-Sasl-Enc: /Hg3pi1Ekz9lqtdOKw9/cc6jR1fRTGX3miGstDmztbow 1329396103
From: Bron Gondwana <brong@fastmail.fm>
To: Tony Finch <dot@dotat.at>
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
X-Mailer: MessagingEngine.com Webmail Interface
In-Reply-To: <alpine.LSU.2.00.1202161126410.31357@hermes-2.csi.cam.ac.uk>
References: <833EE8EEE88E4ADE5CDDDADB@caldav.corp.apple.com> <4F3835A1.7060804@qbik.com> <B764BD8C8B6047E659EABBE2@caldav.corp.apple.com> <4F397212.1030107@qbik.com> <20120213210805.GB13029@launde.brong.net> <alpine.LSU.2.00.1202151405550.30682@hermes-2.csi.cam.ac.uk><1329315552.1444.140661036879893@webmail.messagingengine.com> <4F3BBFA4.8010107@isode.com> <1329316981.8310.140661036883625@webmail.messagingengine.com> <66F68487BF0EED4BA7D767E2410F30B3EFF259456A@FRSPX100.fr01.awl.atosorigin.net><20120215211301.GA16253@launde.brong.net> <alpine.LSU.2.00.1202161126410.31357@hermes-2.csi.cam.ac.uk>
Date: Thu, 16 Feb 2012 13:41:43 +0100
Cc: "Discussion on drastically slimming-down IMAP." <imap5@ietf.org>
Subject: Re: [imap5] Designing a new replacement protocol for IMAP
X-BeenThere: imap5@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "Discussion on drastically slimming-down IMAP." <imap5.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/imap5>, <mailto:imap5-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/imap5>
List-Post: <mailto:imap5@ietf.org>
List-Help: <mailto:imap5-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/imap5>, <mailto:imap5-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 16 Feb 2012 12:41:53 -0000

On Thu, Feb 16, 2012, at 11:30 AM, Tony Finch wrote:
> Bron Gondwana <brong@fastmail.fm> wrote:
> >
> > This is not a problem that's unique to email.  There's nothing really
> > special about email here unless you make it special.  Sure there's a
> > bunch of indexed and optimised ways of viewing that data - sort by
> > trimmed subject, encodings, etc.  All of which could be expressed as
> > generic queries against the data model with a query optimiser on the far
> > end rather than needing a custom syntax for everything...
> 
> This sounds to me a bit more radical than I was expecting. In particular I
> thought you wanted to keep the data model so that existing servers could
> support the new protocol reasonably easily. Which implies that mailboxes
> remain separate silos rather than the store being a soup of messages
> GMail style.

Not really.  Existing servers would be a lot more efficient with a query
which limited to a single "folder", for sure - because they could optimise
it.  But it's no different than an SQL query across a partitioned table.
It means your in-memory-state needs to be big enough to accommodate all
the mailboxes that might be in the regular searches, of course.

But anyone accessing with an IMAP-model client would still access single
folders.  The only difference is that the model would accommodate
cross-folder requests as a first-order object, so you could sort them by
something other than "folder name first", which the existing cross-folder
solutions do.  If you're always sorting by folder first, you lose all the
benefits of having the server do it rather than the client synthesize the
results (the FastMail interface's cross-folder-search is a client
synthasize results thing - and it splits it up by folder... exactly
because the alternative means client-side sorting.  I wrote it many years
ago, and it does a "GUID" alternative which is "f${folderid}u${uid}" - kind
of ugly, and requires the "folder name to folderid" mapping in the database)

> > BEEP has also been mentioned.
> 
> I think BEEP is insane. Masses of complexity just to avoid using multiple
> concurrent TCP connections.

Yeah, I went and actually read the RFCs.  Inclined to agree.  It's
enterprisy in the extreme.

Now SCTP, that would be cool.  But support isn't there.

Bron.
-- 
  Bron Gondwana
  brong@fastmail.fm