Re: [Extra] I-D Action: draft-ietf-extra-imap-messagelimit-03.txt

[Phillip Tao <[e-mail]>]

Hi Alexey,

Thanks for the response. More comments inline.

- Phillip

> On Jun 9, 2023, at 10:39 AM, Alexey Melnikov <[e-mail]> wrote:
> 
> Hi Phillip,
> 
>> On 25 May 2023, at 23:10, Phillip Tao <ptao=[e-mail]> <mailto:ptao=[e-mail]> wrote:
>> 
>> What is the expected outcome for clients which do not support this extension? The Introduction states:
>> 
>> > This extension is compatible with both IMAP4rev1 [RFC3501] and IMAP4rev2 [RFC9051].
>> 
>> But "always return NO" seems like a very loose definition of "compatible".
> You have a point, but this only relates to COPY/UID COPY command.

Yes, I could've worded this better. However, I think even this is may lead to issues. Some clients may not have implemented IMAP4 (or RFC 6851 MOVE), and may still be relying on COPY + UID STORE DELETED + EXPUNGE for move operations. Such clients, given that they haven't even implemented support for MOVE, seem unlikely to implement support for this extension. Then, this may well break the ability for them to do bulk operations (perhaps in the worst case, leading to repeated attempts to perform the same action [on the assumption that the NO is a transient server error]).

This is somewhat mitigated by the fact that they may never sync more than N messages, due to the limits for SEARCH and FETCH. However, that's not reliable, because they may have a batched syncing strategy but not a batched COPY strategy.

The above obviously requires a very specific set of client behaviors, and I'm not sure what percentage of clients would fall into that bucket. However, disregarding any quantitative measure of real world impact, this seems to depart philosophically from previous IMAP extensions. To my knowledge (which is admittedly limited), it seems like all previous extensions have strived to maintain complete backwards compatibility, such that clients/servers which do not support it would experience no change in behavior when talking to a client/server that does support it.

>> I can't think of any similar IMAP extensions in the past which radically alters server behavior without having the client explicitly opting in to it (either by using new commands/new command parameters, or by requiring an ENABLE).
> This is true. What this is trying to say is that this extension can be advertised by both IMAP4rev1 and IMAP4rev2 servers.
>> Clients may not be able to adopt this extension for some time, and some clients may never adopt it (either due to an explicit decision not to, or due to that client or client version no longer being actively maintained). Those clients will therefore be unable to communicate successfully with any server which has implemented this extension.
> I don't think what you are describing above is completely correct. I think what would happen with clients that don't specially handle this extension would encounter one of the following (let's assume that the server advertises MESSAGELIMIT=N):
> 
> 1) If a specific mailbox is smaller than N, then the client would never observe MESSAGELIMIT response code, so it would not be affected.
> 
> 2) If a specific mailbox is larger than N, then the client would be limited to operating on at most N messages at a time. For example SEARCH and FETCH will return N most recent messages in the specified range.
> 
> The client would also be unable to copy more than N messages at a time. (But a client using MOVE will be able to move at most N messages.)
> 
> Does this help?
> 
> Several notes in relation to this:
> 
> 1) If N is small, this would make life very interesting for clients. I think the document should recommend which values are reasonable for servers to pick.
> 
> 2) I think some explanation along the lines of what I explained above should be added to the document to make it clearer what the intent is for casual readers. 
> 
To me, this seems like this has a very high potential to lead to unexpected client behavior for non-adopting clients. Because the MESSAGELIMIT is applied per-command, this seems likely to lead to inconsistencies between commands. For example, searching for UNSEEN UNDELETED messages may return more results than searching for UNSEEN or UNDELETED alone. This seems likely to cause confusion for clients for the flag state (or even existence) of certain UIDs.

It doesn't seem to be explicitly stated in RFC 9051 that I can find, but I imagine most existing clients assume that an OK for a SEARCH or FETCH means that all matching results were returned.

It's not hard to imagine cases where this may lead to unexpected data loss. For example, if a user has setup a rule (in a client which supports client-side rules) to "delete all read messages". That client may do a SEARCH UNSEEN, get back N messages that are UNSEEN, and assume all remaining messages are SEEN and proceed to delete them.
> 
>> I see that version 1 of this draft did require an ENABLE, but that was subsequently removed in version 2. From the mailing list archive, it seems like there may have been some discussion in person at IETF 115 which led to this, but unfortunately the reasoning was not captured in the mailing list. Can you shed more light on why the newer versions have moved away from requiring an explicit ENABLE?
> Basically MESSAGELIMIT conveys a server limitation. There is no way for clients to opt out of it, so using ENABLE doesn't work. And the idea here is to make sure that clients using 1:* in various IMAP commands still need to be subject to the server limit.
> 
I understand the motivation for this. I also understand that this is largely for the benefit of the server, and therefore clients have little incentive to ENABLE it, so to some extent, it must be "imposed". However, I'm just concerned that it's not being imposed in a "safe" way for non-adopting clients.

Have other approaches been discussed to solve this problem? A couple ideas:

* Have this be an ENABLE-able extension. Non-ENABLEing clients face strict throttling.
* Rather than limiting per command, impose a mailbox-wide limit. Meaning that only the largest N UIDs in each given mailbox would be exposed to the client at all (whether for EXISTS, FETCH, etc.). Searches would then return the subset of those N messages which fulfill the search criteria. This would be a larger change and touching many other commands/responses not covered by this current draft, and would be more limiting for adopting clients, but would present a more consistent view and therefore be safer for non-adopting clients.
> When a server returns the MESSAGELIMIT capability, clients can at least discover the server limitation.
> 
Clients can discover the limitation if they've been updated to do so. Not all clients may be able to, and I'm still not clear how we expect those clients to behave.
> Best Regards,
> 
> Alexey
> 
>> 
>> - Phillip
>> 
>>> On May 17, 2023, at 5:45 AM, Alexey Melnikov <[e-mail]> <mailto:[e-mail]> wrote:
>>> 
>>> Hi all,
>>> 
>>> On 17/05/2023 13:42, [e-mail] <mailto:[e-mail]> wrote:
>>>> A New Internet-Draft is available from the on-line Internet-Drafts
>>>> directories. This Internet-Draft is a work item of the Email mailstore and
>>>> eXtensions To Revise or Amend (EXTRA) WG of the IETF.
>>>> 
>>>>    Title           : IMAP MESSAGELIMIT Extension
>>>>    Authors         : Alexey Melnikov
>>>>                      Arun Prakash Achuthan
>>>>                      Vikram Nagulakonda
>>>>                      Luis Alves
>>>>    Filename        : draft-ietf-extra-imap-messagelimit-03.txt
>>>>    Pages           : 9
>>>>    Date            : 2023-05-17
>>>> 
>>>> Abstract:
>>>>    The MESSAGELIMIT extension of the Internet Message Access Protocol
>>>>    (RFC 3501/RFC 9051) allows servers to announce a limit on the number
>>>>    of messages that can be processed in a single
>>>>    FETCH/SEARCH/STORE/COPY/MOVE command.  This helps servers to control
>>>>    resource usage when performing various IMAP operations.
>>> 
>>> This version addes UIDAFTER and UIDBEFORE SEARCH keys.
>>> 
>>> If people are happy with their inclusion, I think this is ready for WGLC.
>>> 
>>> Best Regards,
>>> 
>>> Alexey
>>> 
>>> _______________________________________________
>>> Extra mailing list
>>> [e-mail] <mailto:[e-mail]>
>>> https://www.ietf.org/mailman/listinfo/extra
>>