[lemonade] Re: Comments on draft-ietf-lemonade-reconnect-client-06

[Alexey Melnikov <[e-mail]>]

Hi Eric,
Thank you for your comments.

(Today is about the worst time for me to reply to your comments, as I am 
going on holidays tomorrow.)

Eric Rescorla wrote:

>$Id: draft-ietf-lemonade-reconnect-client-06-rev.txt,v 1.1 2007/10/04 22:25:53 ekr Exp $
>
>
>OVERALL
>This document describes an extension to IMAP to provide faster
>synchronization between client and server. As far as I can
>tell, the optimizations are:
>
>- Removing one round trip needed to discover which messages
>  have been expunged.
>  
>
Yes, if compared to the case when the client/server also implement the 
CONDSTORE extension (RFC 4551).
(Actually it removed 2 round trips per mailbox synchronization.)

When compared to RFC 3501, this extension can potentially provides huge 
bandwidth saving. If a client wants to synchronize flag changes in a 
mailbox, the client needs to fetch flags for *all* mailboxes. For a 
30,000 message mailbox that I currently have is quite painful over a 
slow link.

>- A more compact representation of the list of expunged
>  messages
>  
>
Correct.

>I have some skepticism about the importance of these optimizations.
>  
>
See above.

>The document does not come with any performance measurements,
>and 1 RTT really isn't that much. In particular, I wonder if
>VANISHED really saves that much bandwidth over EXPUNGED if
>compression is in use. I don't know what standard is being
>used to decide whether optimizations of this type are worthwhile.
>  
>
I will let Dave comment on this.

Just a couple of extra comments:
1). As I mentioned above, this extension saves 2 round trips per mailbox 
synchronization. A user might have multiple mailboxes.
2). This is mostly useful for mobile clients which can experience 
frequent connection loss.

>I found this document fairly hard to read. I understand that it's
>a delta to IMAP and requires quite a bit of knowledge of IMAP
>to understand, but I think it could have been written to be
>more clear about how it changes IMAP's behavior in each case.
>In particular, the examples would be improved by always
>having New and Old and having some sort of indicator of exactly
>which PDUs have changed and why.
>  
>
Right.

>DETAILED COMMENTS
>S 2.
>This would be improved by some overall diagram of the new and
>old behavior and some measurement, even an ad hoc one, of
>the performance improvement.
>  
>
Ok. An older version of this document had some numbers, but people 
complained about "irrelevant text".
See section 2.1 of 
<http://tools.ietf.org/id/draft-ietf-lemonade-reconnect-07.txt>
(Note that the protocol has changed substantially since, but the basic 
observation is still correct.)

>S 3.1.
>
>   Conceptually, the client provides a small sample of sequence numbers
>   for which it knows the corresponding UIDs.  The server then compares
>   each sequence number and UID pair the client provides with the
>   current state of the mailbox.  If a pair matches, then the client
>   knows of any expunges up to, and including, the message, and thus
>   will not include that range in the VANISHED response, even if the
>   "mod-sequence-value" provided by the client is too old for the server
>   to have data of when those messages were expunged.
>
>This is probably my ignorance of IMAP, but how can this happen? Why
>doesn't the client have a mod-sequence-value corresponds to these
>UIDs?
>  
>
I will let Dave explain this.

>      S: * VANISHED (EARLIER) 1:2,4:5,7:8,10:11,13:14 [...]
>         29998:29999,30001:30002,30004:30005,30007:30008
>
>This [...] hides the data you're optimizing away, right? This
>would help if it were called out more clearly.
>  
>
Yes. I will add a sentence on this.

>S 3.3, 3.4, 3.5.
>These would all benefit from a statement of how they differ from
>3501, rather than just stating new rules.
>  
>
(Actually, 3.5 updates UID EXPUNGE which was defined in RFC 4315.)

Right. I believe some people wanted to see sections replacing the old 
definitions, as opposed to just pointing out the difference from RFC 3501.

The following text in section 2 summarizes the difference:

   This document puts additional requirements on a server implementing
   the [CONDSTORE] extension.  Each mailbox that supports persistent
   storage of mod-sequences, i.e., for which the server has sent a
   HIGHESTMODSEQ untagged OK response code on a successful SELECT/
   EXAMINE, MUST increment the per-mailbox mod-sequence when one or more
   messages are expunged due to EXPUNGE, UID EXPUNGE or CLOSE; the
   server MUST associate the incremented mod-sequence with the UIDs of
   the expunged messages.

I can try to add a sentence to each one of 3.3, 3.4 and 3.5 to clarify that.

>   If the server is capable of storing modification sequences for the
>   selected mailbox, it MUST increment the per-mailbox mod-sequence if
>   at least one message was permanently removed due to the execution of
>   the EXPUNGE command.  For each permanently removed message the server
>   MUST remember the incremented mod-sequence and corresponding UID.  If
>   at least one message got expunged, the server MUST send the updated
>   per-mailbox modification sequence using the HIGHESTMODSEQ response
>   code (defined in [CONDSTORE]) in the tagged OK response.
>
>So, this is repeated in all three sections. That seems less than
>optimal.
>
>Rather than refing 3501, it would probably be good to point out
>why the message #s are as they are in these examples, due to
>auto-decrement.
>  
>
I've added a clarifying sentence to sections 3.3 and 3.5. I don't think 
section 3.4 needs to change.

>S 3.6.
>   The VANISHED response has two forms.  The first form contains the
>   EARLIER tag, which signifies that the response was caused by a UID
>   FETCH (VANISHED) or a SELECT/EXAMINE (QRESYNC) command.  This
>   response is sent if the UID set parameter to the UID FETCH (VANISHED)
>   command includes UIDs of messages that are no longer in the mailbox.
>   When the client sees a VANISHED EARLIER response it MUST NOT
>   decrement message sequence numbers for each successive message in the
>   mailbox.
>
>   The second form doesn't contain the EARLIER tag and is described
>   below.  Once a client has used "(VANISHED)" with a UID FETCH or
>   "(QRESYNC)" with SELECT/EXAMINE command, the server SHOULD use the
>   VANISHED response without the EARLIER tag instead of the EXPUNGE
>   response.  The server SHOULD continue using VANISHED in lieu of
>   EXPUNGE for the duration of the connection.  In particular this
>   affects the EXPUNGE [RFC3501] and UID EXPUNGE [UIDPLUS] commands, as
>   well as messages expunged in other connections.  Such VANISHED
>   response MUST NOT contain the EARLIER tag.
>
>This is pretty unclear to the non-IMAP expert. Could you explain
>in english what this is trying to accomplish in the document,
>not just specify the protocol mechanics.
>  
>
Basically the VANISHED response is used for 2 purposes: to report UIDs 
of messages expunged earlier and to report UIDs of messages expunged now.
The difference between the two is that in the former case the client 
doesn't need to decrement the number of messages in the mailbox, while 
in the latter case it must.
The former can be distinguished from the latter by presence of the 
"(EARLIER)" label.

>In the example, swap before and after. Also, it would be good
>to show an example of (EARLIER).
>  
>
Ok.

>S 4.1.
>   Strictly speaking, a server implementation that doesn't remember
>   modsequences associated with expunged messages can be considered
>   compliant with this specification.  Such implementations return all
>   expunged messages specified in the UID set of the UID FETCH
>   (VANISHED) command every time, without paying attention to the
>   specified CHANGEDSINCE modsequence.  Such implementations are
>   discouraged, as they can end up returning VANISHED responses bigger
>   than the result of a UID SEARCH command for the same UID set.
>
>Isn't this inconsistent with:
>
>   If the server is capable of storing modification sequences for the
>   selected mailbox, it MUST increment the per-mailbox mod-sequence if
>   at least one message was permanently removed due to the execution of
>   the EXPUNGE command.  For each permanently removed message the server
>   MUST remember the incremented mod-sequence and corresponding UID.  If
>   at least one message got expunged, the server MUST send the updated
>   per-mailbox modification sequence using the HIGHESTMODSEQ response
>   code (defined in [CONDSTORE]) in the tagged OK response.
>
>If not, why not?
>  
>
No, there is no inconsistency:

"a server implementation that doesn't remember modsequences" == "a 
server is incapable of storing modsequences".
The second paragraph you quoted is conditional on server's ability to 
store modsequences. This behaviour is optional for servers.

>S 5.
>   The client MUST also take note of any MODSEQ FETCH data items
>   received from the server.  Whenever the client receives a tagged
>   response to a command, it calculates the highest value among all
>   MODSEQ FETCH data items received since the last tagged response.  If
>   this value is bigger than the client's copy of the HIGHESTMODSEQ
>   value, then the client MUST use this value as its new HIGHESTMODSEQ
>   value.
>
>So, I probably misunderstand something, but my read of 4551 made
>it seem like you could do a MODSEQ FETCH that would not return
>all the metadata for every message.
>
Yes, if the client issues a FETCH MODSEQ for a subset of messages. 
However the previous paragraph in the same section requires the client 
to perform a full synchronization.

>In that case, wouldn't this
>procedure risk you having a modseq that is higher than some
>messages you haven't examined yet? What am I missing.
>  
>
The intent of this paragraph is to talk about unsolicited FETCH MODSEQ 
returned by the server *after* a full synchronization is complete (the 
previous paragraph in the same section). So the situation you describe 
can't happen, because the server is required to send all unsolicited 
FETCH MODSEQ responses.
The beginning of this paragraph is ambiguous, so I suggest that it 
should be clarified by changing the first quoted sentence to read:

After completing full synchronization, the client MUST also take note of 
any MODSEQ FETCH data items received from the server.

>I don't see any new security issues in this document.
>  
>
Regards,
Alexey



_______________________________________________
lemonade mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/lemonade
Supplemental Web Site:
http://www.standardstrack.com/ietf/lemonade