[httpmail] Modelling E-mail access in a protocol-independent

[Nicolas Williams <[e-mail]>]

[Pardon the re-send.  I forgot to use my new e-mail address.]

I stumbled onto the BikINI (IMAP replacement proposal) and httpmail this
weekend as a side-effect of migrating mailboxes as part of my old employer's
acquisition by my new employer :)

I have a number of comments on both:

a) Must support for folders as "tags", with messages being able to be in
   more than one folder at once.  Folders should be seen as collections.

b) Must support HUGE mail folders -- folders with hundreds of thousands,
   or even millions of messages.  Folders' contents typically change in
   these ways:

    - new messages are appended;
    - messages are flagged a new/seen/read/... or for deletion;
    - messages flagged for deletion are expunged.

   Changes to folders can be asynchronous to the clients that have them
   "open": new e-mail can be delivered externally, multiple clients can
   access a folder concurrently and change message flags, add messages
   and even expunge messages.

   This implies that a diff/delta log model of keeping clients up to
   date with folder contents would be beneficial.

c) Server-side search facilities would be a big plus.  Particularly when
   we're talking about large folders in large mailboxes.

   Clients can always maintain local indexes, or even entire copies of
   mailboxes (many do) to speed up searches.  But server-side search
   facilities help produce light-weight clients, such as for mobile
   devices.

   I'm particularly annoyed at how many clients fail to follow threads
   across folders.  All that requires is a search by In-reply-to message
   IDs (Ok, I'm simplifying a bit...).

d) Asynchronous or cancellable operations are a big plus (who has not
   experienced an MUA freezing while downloading a huge message?).

Looking at httpmail I think the model behind it applies equally well to
BikINI:

   Everything's a resource with a URL or an action.  Resources:

    - messages;
    - mailboxes;
    - folders in mailboxes;
    - filters;
    - feeds/logs;

   Verbs:

    - get a message
    - put a message
    - modify message metadata (flags, tags/folders)
    - list a mailbox's folders
    - add/rename/delete folders
    - list a folder's messages
    - get a feed/log of changes to folders' contents

The actions map well to HTTP verbs; a RESTful protocol is feasible.

As to differences between an HTTP-based protocol and a more traditional
protocol, the main one that I can think relates to authentication
options: HTTP has fewer options than a non-HTTP protocol (which could
use SASL, for example, in addition to TLS).  (But this depends in part
on the status of HTTP/Negotiate updates, about which I'm not up to
date.)  There are other differences, such as how much chattier an
HTTP/Atom-based protocol would be than BikINI.

It's possible that there is really only room for one new protocol, and
that's likely to be httpmail in this day and age.  But the model behind
it sure seems to be generalizable.

A comment specific to httpmail: Atom is nice, but for light-weight
clients it'd be nicer if there was a simple, non-XML-based format for
folder listings and the like.  Sure, such a format would not be as
extensible as XML schemas are, but that seems like a fair price to pay
for mobile apps.

Nico
--