Re: [imapext] GMail's extensions: X-GM-THRID immutability

[Brandon Long <[e-mail]>]

On Thu, May 9, 2013 at 9:27 AM, Jan Kundrát <[e-mail]> wrote:

> Hi,
> this might be a bit OT for this mailing list, but given that GMail is a
> rather dominant player among the IMAP service providers and due to the
> usefulness of having data available to other, I'm posting here anyway.
>
> - It isn't clear to me what the immutability guarantees of the returned
> data of the X-GM-THRID [1] extension are. It would be terrific if it was
> safe to cache these values in the same manner as any other immutable
> per-message data, but I suspect this is not possible -- if the threading is
> "right", delivery of any new message could potentially join independent
> threads together, thereby changing the thread IDs. How is GMail handling
> this? How is the threading computed?
>

You should be able to consider X-GM-THRID as immutable, that should have
been called out in the documents.  We never join threads.

Gmail does not thread by references, its closest to threading by subject.
 Our user facing parlance is typically "conversation" instead of thread.  A
conversation is basically up to 100 messages with the same "clean" subject.
 IIRC, we will start a new conversation if there is sufficient time gap
between the two, but we will use references to put a reply on an old
conversation regardless of time... though the 100 message is a "hard" limit.

Having a limit was chosen for performance reasons, we don't want to load a
couple thousand message indexes from disk or have paging in the
conversation view interface.  The conversation view doesn't lend itself to
true threading either, where "new" messages may be in multiple places in
the page.... and most email communication doesn't have that complicated of
a thread structure to begin with.  It also fixes the typical use cases of
"find a message from foo and reply and change the subject" instead of using
an address book and some other corner cases.


>
> - Any plans to support the THREAD=REFS [2] in GMail?
>

So, the new search terms might be interesting.  MESSAGEID is basically how
we handle HEADER MESSAGEID already (ie, its not substring, its actually
mapped to the rfc822messageid: term in our index).  INTHREAD is more odd
because our threads can cross folder boundaries (since they're not folders,
they're labels) but nothing overly complicated for us.

THREAD=REFS is just as unlikely as THREAD=REFERENCES for us.  We're not
computing threads on the fly just for IMAP.  We had one early engineer on
the project joke about implementing THREAD=GMAIL, but the THREAD= concept
seems fundamentally flawed to me that both the client and server require
understanding of the threading algorithm.  X-GM-THRID is basically "the
server knows the threads, trust it".


> - Any plans for support of the SORT extensions?
>

We have only the single SORT order in everything we do, so no.  Again, we
can't load the entire folder into memory just to SORT it on the server, and
we're not going to pre-compute and store each possible order.  You'll note
there is only the one order in all of the Gmail interfaces.  Our marketing
would say, don't sort, search!


> - Any plans for CONTEXT=SORT and CONTEXT=SEARCH? :)
>

No.  We'd probably have to create a filter on the fly for CONTEXT=SEARCH,
get updates that the filter has hit, and then load and finish testing
against the new message to see if it actually hit (not all of the IMAP
SEARCH syntax is translatable to our own search syntax, some things require
loading parts of the message).  That just seems like an overall large
effort for supporting "online" low caching clients, and we don't really
support them well in many other ways, mostly because almost all clients are
actually offline caching clients.


>
> The reason why I'm asking is because I'd like to enable threading for
> Trojita's users who are accessing GMail. Any threading algorithm apart from
> THREAD=REFS has significant limitations without a server-side sorting
> simply because old threads with new arrivals are not listed as recent. The
> only remedy is giving up and hoping that UIDs somehow correlate with the
> age of a any message, which is rather crude heuristic.
>

What does "old thread with new arrivals are not listed as recent" mean?
 New messages are clearly new, since they have newer UIDs (well, new to
that folder at least).  We always display messages in our "threads" in
INTERNALDATE order.

Brandon


>
> With kind regards,
> Jan
>
> [1] https://developers.google.com/**google-apps/gmail/imap_**
> extensions#access_to_the_**gmail_thread_id_x-gm-thrid<https://developers.google.com/google-apps/gmail/imap_extensions#access_to_the_gmail_thread_id_x-gm-thrid>
> [2] http://tools.ietf.org/html/**draft-ietf-morg-inthread-01<http://tools.ietf.org/html/draft-ietf-morg-inthread-01>
>
> --
> Trojitá, a fast Qt IMAP e-mail client -- http://trojita.flaska.net/
> ______________________________**_________________
> imapext mailing list
> [e-mail]
> https://www.ietf.org/mailman/**listinfo/imapext<https://www.ietf.org/mailman/listinfo/imapext>
>