Re: [Extra] Review: draft-ietf-extra-imap4rev2-11

Timo Sirainen <timo@sirainen.com> Thu, 30 January 2020 08:26 UTC

Return-Path: <timo@sirainen.com>
X-Original-To: extra@ietfa.amsl.com
Delivered-To: extra@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 024D3120112 for <extra@ietfa.amsl.com>; Thu, 30 Jan 2020 00:26:06 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Cnsr6OiPag3e for <extra@ietfa.amsl.com>; Thu, 30 Jan 2020 00:26:01 -0800 (PST)
Received: from sirainen.com (mail.sirainen.com [94.237.26.55]) by ietfa.amsl.com (Postfix) with ESMTP id 243E512010F for <extra@ietf.org>; Thu, 30 Jan 2020 00:26:00 -0800 (PST)
Received: from [192.168.10.100] (unknown [91.155.205.240]) by sirainen.com (Postfix) with ESMTPSA id 4927A2B3C8B; Thu, 30 Jan 2020 08:25:59 +0000 (UTC)
From: Timo Sirainen <timo@sirainen.com>
Message-Id: <4056563A-AF4E-4B1D-9AD7-D079A8EAC971@sirainen.com>
Content-Type: multipart/alternative; boundary="Apple-Mail=_24C95EA1-7E3A-48E9-86C5-2EEB710DE94A"
Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3608.40.2.2.4\))
Date: Thu, 30 Jan 2020 10:25:58 +0200
In-Reply-To: <c62f07c1-cebc-48e5-a3e6-65773f5ff969@dogfood.fastmail.com>
Cc: extra@ietf.org
To: Bron Gondwana <brong@fastmailteam.com>
References: <9fdfa8e8-147d-4248-a0c1-48de171ac675@dogfood.fastmail.com> <872422BE-C8FA-4AA0-8EF1-2ECA79F64926@sirainen.com> <c62f07c1-cebc-48e5-a3e6-65773f5ff969@dogfood.fastmail.com>
X-Mailer: Apple Mail (2.3608.40.2.2.4)
Archived-At: <https://mailarchive.ietf.org/arch/msg/extra/OIc3K9gqcCCV4cjiy_kp7WTSaL8>
Subject: Re: [Extra] Review: draft-ietf-extra-imap4rev2-11
X-BeenThere: extra@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Email mailstore and eXtensions To Revise or Amend <extra.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/extra>, <mailto:extra-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/extra/>
List-Post: <mailto:extra@ietf.org>
List-Help: <mailto:extra-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/extra>, <mailto:extra-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 30 Jan 2020 08:26:06 -0000

On 29. Jan 2020, at 23.52, Bron Gondwana <brong@fastmailteam.com> wrote:

>>> 5.1 Mailbox Naming
>>> 
>>> This doesn't solve one of my naming bugbears.  The spec says things like "servers MAY accept a de-normalized UTF-8 mailbox name and conver it to Net-Unicode prior to mailbox creation.." - there is no clear way to know what the SELECTable name will be after creating a mailbox.
>>> 
>>> If I was designing this myself from scratch I would say that "SELECT of the same octets that were passed to CREATE MUST open that same mailbox" or something to that effect.  Since we're messing with how mailbox naming works somewhat for imap4rev2, I'd be keen to add this restriction on server behaviour so that clients don't fall into a "CREATE -> ALREADY EXISTS / SELECT -> DOESN'T EXIST" loop.
>>> 
>>> It still doesn't mean that you can reliably tell the correct octets for the matching mailbox from the LIST response though :(
>> 
>> Is it too late to add a new response code? Something like:
>> 
>> * OK [CREATED] TAG "a123" NAME "Denormalized name"
> 
> Presumable the name would be an astring.  If we did this, I'd want to say "if OBJECTID is supported, MAILBOXID goes into this response as well".

Yeah, would be nice.

>>> 6.3.4 and 6.3.5 CREATE and DELETE
>>> 
>>> Can we make it illegal to "DELETE" a mailbox with children rather than make it a \NoSelect mailbox pretty pretty please?  I'm just looking at some patches to deal with Outlook misbehaving with this precise situation, and it's all horrible.  Because of mailboxId requirements for JMAP we've been forced to keep these things around as "intermediate" mailboxes anyway, and it's been the source of tons of bugs.
>> 
>> \NoSelect seems to be causing confusion in general, and Dovecot nowadays has options that makes it try to avoid creating them. But I'm not sure if it needs to be made illegal in the RFC itself. I think your server implementation can just decide to reject DELETE if it has children.
>> 
>> It sounds like your issues are with \NoSelect mailboxes in general, not just this specific way of creating them with DELETE? For example CREATE a/b could create "a" se \NoSelect also.
> 
> That's indeed what happens now in our server:
> 
> . list "" a*
> * LIST (\HasNoChildren) "/" a/b/c
> . OK Completed (0.001 secs 3 calls)
> . list "" a/%
> * LIST (\Noselect \HasChildren) "/" a/b
> . OK Completed (0.001 secs 3 calls)
> . list "" a
> * LIST (\Noselect \HasChildren) "/" a
> . OK Completed (0.000 secs 3 calls)

This is also server-specific whether the "a" would be listed as part of "a*". Or in Dovecot case it's mailbox format specific - it happens with some formats but not with others.

>>> 6.3.11 STATUS
>>> 
>>>       Note: The STATUS command is intended to access the status of
>>>       mailboxes other than the currently selected mailbox.  Because the
>>>       STATUS command can cause the mailbox to be opened internally, and
>>>       because this information is available by other means on the
>>>       selected mailbox, the STATUS command SHOULD NOT be used on the
>>>       currently selected mailbox.
>>> 
>>> This advice is all well and good, but it says nothing about what the server may or may not do when it gets one of these STATUS commands.  This is particularly relevant in the context of LIST RETURN STATUS, where it's hard to construct list patterns which explicitly avoid the currently selected mailbox.
>>> 
>>> Likewise with the: MUST NOT be used as a "check for new messages in the selected mailbox" assertion.  Plenty of clients will do a STATUS run across folders and if they're doing LIST RETURN STATUS they're gonna find out about the new messages if the server returns it.  Clients can not rely on it working is a more honest representation of what happens.
>>> 
>>> I'd like to see that IMAP4REV2 servers MUST implement STATUS of the currently selected mailbox.  Yes, it's a special codepath - but if you're already testing if it's the currently open mailbox then you're at exactly the right place to do the right thing:
>> 
>> Also this is the only way to get the current UIDNEXT of the selected mailbox (since it might be higher than last message's UID).
>> 
>> The main ambiguity though is whether STATUS on the current mailbox is expected to check for new changes. Dovecot implements it in a way that it doesn't check for changes. I think there used to be even a good reason for that.
> 
> Well.. if it doesn't check for changes presumably it can't tell you the correct UIDNEXT? Either that or it will have an inconsistent response for the selected folder.

I think arguably it's even more correct that the UIDNEXT tells what it is in the current IMAP session state rather than the refreshed state. (Although I wouldn't want to require that in the RFC.) So for example:

a status inbox (uidnext)
* STATUS inbox (UIDNEXT 10)
a OK [CLIENTBUG] Status on selected mailbox completed (0.001 + 0.000 secs).

## deliver new mail

b status inbox (uidnext)
* STATUS inbox (UIDNEXT 10)
b OK [CLIENTBUG] Status on selected mailbox completed (0.001 + 0.000 secs).
c noop
* 2 EXISTS
* 2 RECENT
c OK NOOP completed (0.001 + 0.000 secs).
d status inbox (uidnext)
* STATUS inbox (UIDNEXT 11)
d OK [CLIENTBUG] Status on selected mailbox completed (0.001 + 0.000 secs).

I'll probably remove that [CLIENTBUG]..

>>> 6.4.4.1 SAVE Result
>>> 
>>> I'm not sure I understand the interaction here with closing and re-opening a mailbox.  I would expect that closing a mailbox would wipe the SAVED result,
>> 
>> I think it practically says that, since it says that SELECT/EXAMINE resets it to empty sequence. It's not really relevant whether it happens when the mailbox is closed or when the next mailbox is opened, because you can't use it in non-selected state anyway..
> 
> Right - so that whole section is pointless.  This text:
> 
>    If the server decides to send a new UIDVALIDITY value while the
>    mailbox is opened, this causes resetting of the search variable to
>    the empty sequence.
> 
> If any SELECT resets the variable.

I'm not sure I understand your point. This text is pointless? The first paragraph discussing SELECT resetting the sequence is pointless? They seem somewhat complementary to me. Actually I think it would most make sense to not allow IMAP server to change UIDVALIDITY while a mailbox is selected. I highly doubt there are many clients that can handle it correctly.

>>> 7.4.2 FETCH Response
>>> 
>>> I remember past bugs with UID in "unsolicited" FLAGS responses..  Do we need to require that including UID only happen after ENABLE IMAP4REV2 (as it is for ENABLE QRESYNC)?
>> 
>> Might be useful to require UID for any untagged FETCH reply. It can simplify client behavior.
> 
> We definitely changed from doing it always because some clients couldn't handle it - but that was 10 years ago, maybe those clients don't exist any more.  Sadly I don't remember which clients it was.

This behavior can be enabled only after ENABLE IMAP4rev2 so it wouldn't matter if some old clients couldn't handle it.