re: Rich, hierarchical mail object storage in IMAP

[Mark Crispin <[e-mail]>]

Hi William -

     Thank you for your comments.  I'm sorry to have taken so long to reply;
this past week has been quite busy...

     To begin, note that at UW ``mailbox'' and ``folder'' are synonyms, with
me (and the IMAP documents) using ``mailbox'' and everyone else using
``folder''.   [My co-workers claim that a mailbox is an address that you send
mail to, not an entity that you read.]  I think that you mean ``folder'' as
referring to the entity I call ``a collection of mailboxes'' (and my co-
workers call ``a collection of folders'').  Please keep this in mind when
reading the rest of this message.........

     There is no presumption in IMAP of ``a UNIX style mail storage area''.
In fact, IMAP was originally designed and implemented on a completely
different operating system.  Nor does IMAP presume that ``mailboxes are stored
in a flat fashion with no way to accomodate something like a hierarchial
folder tree.''  To my knowledge, IMAP has never been implemented on an
operating system with a flat filesystem.

     IMAP lacks any presumption of the nature of the mail store, beyond a
fundamental assumption that a character string can adequately identify a
single folder object in the mail store.

     This is quite intentional.  Any greater presumption necessarily limits
IMAP to a particular mail store scheme; possibly even to a particular
operating system.  It was a fundamental design goal of IMAP that there be no
such limitations.

     Recall that IMAP defines only the special name ``INBOX''.  All other
names are completely up to the particular server implementation.  It may well
be that certain implementations have chosen to use file path names, but there
is no requirement that they do so.

     What this implies is that any mechanism for folder hierarchy must be
external to IMAP.  Indeed, there have been any number of individuals who have
wished that IMAP solved the folder hierarchy management problem, including in
our group at UW.  It would make a lot of problems much simpler.  However, like
most simple solutions, it's *too* simplistic.  Gradually, after seeing the
extent of the interoperability problem, people have been won over towards the
external solution viewpoint.

     Consider, if you would, how much less attractive IMAP would be to you if
a ``UNIX style mail folder hierarchy'' were imposed upon you.  Not only
wouldn't your preferred model be supported, you'd have to deal with working
around a different (perhaps totally hostile) model imposed upon you.

     It can also be claimed that ``folder structure (hierarchy) is in the mind
of the beholder''.  There is nothing that prevents a client from imposing its
own hierarchical view (context management in the forthcoming new release of
Pine is somewhat like this).  Similarly, a client can flatten out heirarchy
into extinction.  Similarly, the management of a server may wish to impose (or
eliminate) physical hierarchy of the bits on their disk to suit their
operational needs.  For example, one can imagine a mailer which recognizes
that users Bob, Ted, Carol, and Alice all received a copy of the same message;
and instead of giving them each a private copy gave them a pointer to a single
shared copy.  Such details may be very relevant to the management of a server,
but are of no interest to a client.

     Some work on external hierarchy exists in IMSP (the IMAP support
protocol) and in the context management code in the forthoming Pine.

     I would like to refer you to the IMSP work at CMU for answers to your
questions about non-mailbox objects.  IMSP is designed to be a layer on top of
IMAP that does the tasks that IMAP does not do.  The idea is to split the
labor and have two smaller tools rather than one monolithic tool.

     Recently, our friends at CMU have posted some comments about address book
implementation in IMSP.  Like you, we feel that a distributed address book is
very important and certainly have this as a requirement.  We just don't have
the requirement that IMAP satisfy *all* of the requirements by itself...

     The reason for this was that I (and others) have become quite wary of
monolithic solutions that purport to solve all problems.  Such monolithic
solutions tend to be unwieldy kludge towers (X.400 comes immediately to mind)
that few people understand and fewer implement to any degree.  Worse, they
tend to ``solve all problems'' by defining out of existance areas of the
problem space.  [For example, it is easy to put folder hierarchy into IMAP if
you define that all file stores now and forever are BSD UNIX.]

     The combination of IMAP and IMSP addresses a considerable portion of the
overall distributed mail problem; although it is conceivable that at some
point in the future additional protocols may be necessary (we've already
talked about a ``netbiff'' for PC's).

     I hope that this has satisfactorily answered your concerns.  I am quite
aware that these issues are complex and the seeming absence of such important
functionality in IMAP can be difficult to fathom.  Please do not hesitate to
contact me if you have any further questions.

Regards,

-- Mark --