[lemonade] SUBMITTED keyword, Outbox and Sent folders, and CATENATE/BURL

[Peter Coates <[e-mail]>]

Within IMAP, the "proper" way to mark a message as draft is to use the
\draft flag.  Similarly the way tentatively to delete a message is to use
the \deleted flag.

Many clients instead use "Drafts" and "Deleted" folders for this purpose.

Other special purpose folders that are commonly used are "Outbox" and
"Sent".  And perhaps "Spam", but I do not want to go there for the moment.
There are no "proper" IMAP ways to mark a message as "Sent" analogous to the
\draft or \deleted flags, still less one meaning queued for submission, or
being submitted.

When I said the "Drafts", "Deleted", "Outbox", and "Sent" are commonly used
folders, I overstate.  Special folders are used for these four functions,
but the names vary.  This can and does cause both interoperability issues
between clients, and confusion on the part of users. 

Actually, the "Outbox" folder has historically not caused issues for IMAP
servers as this has almost always been a folder that was local to the
client, and it is obvious why.  With CATENATE and BURL the outbox function,
or perhaps I should say the outbox state moves to the server.

Let me go through the life cycle of a message being sent from a couple of
putative lemonade compliant client which have chosen two different models.
The first model is to mark the messages to show state, the other uses
different folders for this purpose.  I will then start a discussion about
what a client should do to achieve the best possible interoperability, and
what needs defining for interoperability.

Client model 1 (messages marked with state)
Define three flags (keywords) for the state of outbound messages: 
       QUEUED (to be submitted as soon as possible)
       SUBMITTED (successfully submitted to the submit server)
       
1. The client constructs a message to be sent, eg
C: 1 APPEND INBOX (QUEUED) CATENATE (.....)
S: 1 OK [APPENDUID 1192256 321] done

2. construct a URL for this message, eg
C: 2 GENURLAUTH
IMAP://[e-mail]/INBOX;UIDVALIDITY=1192256/;UID=321;UTLAUTH=SUB
MIT+userid INTERNAL
S: *
IMAP://[e-mail]/INBOX;UIDVALIDITY=1192256/;UID=321;URLAUTH=SUB
MIT+userid:internal:12345678
S: 2 OK done

3. submit the message
c: mail from: <[e-mail]>
s: 250 2.5.0 OK
c: rcpt to: <[e-mail]>
s: 250 2.5.0 OK
c: burl
IMAP://[e-mail]/INBOX;UIDVALIDITY=1192256/;UID=321;URLAUTH=SUB
MIT+userid:internal:12345678 last
s: 250 2.5.0 OK

4. mark the message as having been submitted
C: 4 UID STORE 321 flags SUBMITTED

<done>

Next, the equivalent using Outbox and Sent folders:

1. The client constructs a message to be sent, eg
C: 1 APPEND Outbox CATENATE (.....)
S: 1 OK [APPENDUID 1192256 321] done

2. construct a URL for this message, eg
C: 2 GENURLAUTH
IMAP://[e-mail]/Outbox;UIDVALIDITY=1192256/;UID=321;URLAUTH=SU
BMIT+userid INTERNAL
S: *
IMAP://[e-mail]/Outbox;UIDVALIDITY=1192256/;UID=321;URLAUTH=SU
BMIT+userid:internal:12345678
S: 2 OK done

3. submit the message
c: mail from: <[e-mail]>
s: 250 2.5.0 OK
c: rcpt to: <[e-mail]>
s: 250 2.5.0 OK
c: burl
IMAP://[e-mail]/Outbox;UIDVALIDITY=1192256/;UID=321;URLAUTH=SU
BMIT+userid:internal:12345678 last
s: 250 2.5.0 OK

4. Move the message to the Sent folder
C: 4A SELECT Outbox
C: 4B UID COPY 321 Sent
S: 4B OK [APPENDUID 1418141 198] done
C: 4C UID STORE 321 +flags \deleted
C: 4D UID EXPUNGE 321

<done>

I believe that users will be more comfortable with the second approach than
the former, but that the former is more in the spirit of IMAP than the
latter.  There is nothing to stop a client using a hybrid approach:

1. The client constructs a message to be sent, eg
C: 1 APPEND Outbox (QUEUED) CATENATE (.....)
S: 1 OK [APPENDUID 1192256 321] done

2. construct a URL for this message, eg
C: 2 GENURLAUTH
IMAP://[e-mail]/Outbox;UIDVALIDITY=1192256/;UID=321;URLAUTH=SU
BMIT+userid INTERNAL
S: *
IMAP://[e-mail]/Outbox;UIDVALIDITY=1192256/;UID=321;URLAUTH=SU
BMIT+userid:internal:12345678
S: 2 OK done

3. submit the message
c: mail from: <[e-mail]>
s: 250 2.5.0 OK
c: rcpt to: <[e-mail]>
s: 250 2.5.0 OK
c: burl
IMAP://[e-mail]/Outbox;UIDVALIDITY=1192256/;UID=321;URLAUTH=SU
BMIT+userid:internal:12345678 last
s: 250 2.5.0 OK

4. Move the message to the Sent folder
C: 4A SELECT Outbox
C: 4B UID COPY 321 Sent
S: 4B OK [APPENDUID 1418141 198] done
C: 4C UID STORE 321 +flags \deleted
C: 4D UID EXPUNGE 321
C: 4E SELECT Sent
C: 4F UID STORE 198 flags SUBMITTED

<done>

The Obvious down side to both the approaches using multiple folders is that
it requires extra selects which are relatively expensive commands and which
lose any CONTEXTS that might be active for the user, and which change the
meaning of any NOTIFY command that might be outstanding.  One solution to
that might be to use a second IMAP session for manipulating the message that
is being submitted.

These has been some discussion as to whether a third state, a SUBMITTING
keyword, is desirable.  This all hinges on recovery after a crash, and
avoiding having multiple clients all submitting queued messages at the same
time.  It is asserted that a combination of the SUBMITTING keyword and the
CONDSTORE semantics can be used to avoid that scenario.

I am not convinced by having a shared Outbox folder being used by multiple
clients.  I think this might lead to surprises for the user.  It might be
better for each client to construct a unique name for its outbox.  On the
other hand, having several Send folders is a known problem.  It would be
nice if we could suggest that if a client is going to use a folder for sent
messages that it use the folder "Sent".  That does not preclude the client
displaying it as whatever it wants, much as INBOX is displayed as various
things in languages other than English.

Peter

_______________________________________________
lemonade mailing list
[e-mail]
http://www.ietf.org/mailman/listinfo/lemonade
Supplemental Web Site:
http://www.standardstrack.com/ietf/lemonade