Re: [imapext] AD review of draft-ietf-imapapnd-appendlimit-extension-06

[Adrien de Croy <[e-mail]>]

> On 11/12/2015, at 2:27 PM, Stu Brandt <[e-mail]> wrote:
> 
> Wow....so do you already support 18Ebyte appends/fetches?  That's pretty impressive. You must have spent a lot of time on code to ensure streaming between network and storage.

You seem to be confusing size with bandwidth / throughput. 

And just because someone might want to transfer a file just over 4GB doesn't mean they want to transfer a 18EB file. But if someone provisions a system big enough they can do it. If they want to stop that they already have quota. 

Talk to Bill Gates about how his "nobody will ever need more than 640kB RAM" worked out. 

> 
> - Stuart
> 
> 
>> On 12/10/15 7:44 PM, Adrien de Croy wrote:
>> Understood
>>  
>> However I think you'll find some implementations are already non-compliant in that they handle larger messages already.  We         already use 64bits.
>>  
>> I understand at the time that IMAP was written the concept of a 4GB message was almost inconceivable, and probably Mark didn't want people to use 16bits, but for a protocol that sends numbers as texts with no limit on the number of decimal digits being transmitted, it makes no sense to set a max size, only a minimum upper limit.  I'm not aware of any other protocol (leastwise not in the email space) that uses text to send numbers that places a limit like this on the underlying data type or width.  In fact the whole benefit of using text is that it frees you from that, implementers can choose whatever underlying type they see fit, bearing in mind the environment they work in.
>>  
>> I'd suggest if someone wants to advertise no limit, it should use a non-numeric keyword, rather than overloading 0, or -1 or whatever which flavour of maxint is at the moment.
>>  
>> Adrien
>>  
>> ------ Original Message ------
>> From: "Stu Brandt" <[e-mail]>
>> To: "Adrien de Croy" <[e-mail]>; "[e-mail]" <[e-mail]>
>> Sent: 11/12/2015 1:20:58 p.m.
>> Subject: Re: [imapext] AD review of draft-ietf-imapapnd-appendlimit-extension-06
>>  
>>> Adrien -
>>> 
>>> I'm not looking to further entrench it, rather I'm pointing out that as long as the BNF itself defines a limit on the size of 'number', then a server's claim of having no limit is irrelevant. Such a server could express its limitless APPENDLIMIT by referring to the BNF defined limit.  I would hope that if/when 'number' as MAX_UINT32 is extended, the next logical step would be MAX_UINT64 which itself is yet another limit. 
>>> 
>>> - Stuart
>>> 
>>> 
>>>> On 12/10/15 6:51 PM, Adrien de Croy wrote:
>>>> please don't further entrench the hard MAX_INT32 limit on message size, it will have to be overturned eventually.  4GB gets smaller every day.
>>>>  
>>>> ------ Original Message ------
>>>> From: "Stu Brandt" <[e-mail]>
>>>> To: mailto:[e-mail] <[e-mail]>
>>>> Sent: 11/12/2015 12:14:11 p.m.
>>>> Subject: Re: [imapext] AD review of draft-ietf-imapapnd-appendlimit-extension-06
>>>>  
>>>>> Barry -
>>>>> 
>>>>> On the issue of "there's no limit on this mailbox", doesn't the IMAP protocol itself establish an append size limit through rfc3501 BNF alone?
>>>>> 
>>>>> literal         = "{" number "}" CRLF *CHAR8
>>>>> number          = 1*DIGIT
>>>>>                     ; Unsigned 32-bit integer
>>>>>                     ; (0 <= n < 4,294,967,296)
>>>>> This would support the notion of advertising APPENDLIMIT=4294967295 for those who *claim* to have no limit since ultimately there *is* a limit.
>>>>> 
>>>>> - Stuart
>>>>> 
>>>>>> On 12/10/15 5:39 PM, Jayantheesh S B wrote:
>>>>>> Hi Barry,
>>>>>> 
>>>>>> Please find our response inline.
>>>>>> 
>>>>>> Regards,
>>>>>> Jay
>>>>>> -----Original Message-----
>>>>>> From: imapext [mailto:[e-mail]] On Behalf Of Barry Leiba
>>>>>> Sent: Thursday, December 10, 2015 2:10 PM
>>>>>> To: Narendra Bisht; Adrien de Croy
>>>>>> Cc: [e-mail]; [e-mail]
>>>>>> Subject: Re: [imapext] AD review of draft-ietf-imapapnd-appendlimit-extension-06
>>>>>> 
>>>>>> Hi, Narendra, and thanks for the responses.  I'm eliminating all the things we're in agreement on, so we can discuss the others a bit.
>>>>>> 
>>>>>>> The substantive issue is this:  Suppose my server supports 
>>>>>>> APPENDLIMIT, and wants to advertise that to the client, but there's no 
>>>>>>> global limit.  So it says "APPENDLIMIT" in response to CAPABILITY, 
>>>>>>> without giving a number.  Good.  Now you're requiring that there be a 
>>>>>>> limit on every mailbox -- the protocol provides no way to tell the 
>>>>>>> client that there's no limit on this mailbox.  Why is that not a 
>>>>>>> problem?
>>>>>>> 
>>>>>>> [Naren] This draft is to publish the APPENDLIMIT, if there is any. If 
>>>>>>> you accept any size then there is no need to publish this policy.
>>>>>> I think you missed my point.
>>>>>> 
>>>>>> The situation I'm talking about is this:
>>>>>> 
>>>>>> - The server has no global limit, so no number is advertised on the CAPABILITY response.  The spec says, therefore, that per-mailbox limits have to be advertised on, say, STATUS commands.  Fine.
>>>>>> 
>>>>>> - There's a limit of, say, 50,000 for INBOX.  So when I do STATUS INBOX (APPENDLIMIT), the server responds with "* STATUS INBOX (APPENDLIMIT 50000)".  Fine.
>>>>>> 
>>>>>> - But there's no limit on the "Stuff-to-save" mailbox.  No limit at all.
>>>>>> 
>>>>>> How does the server respond when I do "STATUS Stuff-to-save (APPENDLIMIT)" ?  Why is it not a problem that the spec doesn't answer this question?
>>>>>> 
>>>>>> 
>>>>>> [Jay] We understood the situation. In this case we can have a big value to the APPENDLIMIT, like 2^32 - 1.
>>>>>> We can have it documented. We can have a line like:
>>>>>> "If server decides not to have any upload limit for a mailbox, then it can have a huge number set for APPENDLIMIT"
>>>>>> 
>>>>>>> -- Section 2 --
>>>>>>> 
>>>>>>> "An IMAP server that supports APPENDLIMIT extension advertises this by  
>>>>>>> including the name APPENDLIMIT in its capability list. IMAP server  
>>>>>>> MAY advertise this capability after user has logged in."
>>>>>>> 
>>>>>>> This says nothing about whether the server is allowed to advertise the 
>>>>>>> capability before the user has logged in.  I also don't think this is 
>>>>>>> an appropriate use of 2119 "MAY".  If you mean, "The IMAP server MUST 
>>>>>>> NOT advertise this capability until after the user has logged in,"
>>>>>>> then you should say it that way.  (And if that's not what you mean, 
>>>>>>> please discuss it with me and clarify.)
>>>>>>> [Naren] We purposefully marked it as a MAY, because it alerts 
>>>>>>> implementations to the possibility, but doesn't have any effect in 
>>>>>>> interoperability.
>>>>>>> 
>>>>>>> What we mean here is client should be ready to handle this no matter 
>>>>>>> when the capability is sent.
>>>>>> Great, and I saw the discussion in the other thread.  My sense here is that 2119 key words aren't appropriate for this at all: there's no choice here that needs a "MAY" -- you're just talking about how the protocol works when this option is in effect:
>>>>>> 
>>>>>> NEW
>>>>>> An IMAP server that supports APPENDLIMIT extension advertises this by including the name APPENDLIMIT in its capability list in Authenticated state.  The server may also advertise this extension before the user has logged in.
>>>>>> END
>>>>>> 
>>>>>> (With this formulation, if you really want to say "MAY" in the last sentence, I won't object further.
>>>>>> 
>>>>>> [Jay] OK
>>>>>>> -- Section 4 --
>>>>>>> 
>>>>>>> "Client can avoid use of LITERAL+ [RFC2088], when maximum upload size  
>>>>>>> supported by the IMAP server is unknown."
>>>>>>> 
>>>>>>> What?
>>>>>>> Don't you mean "The client SHOULD avoid"?  I'd even use this as an 
>>>>>>> opportunity to make it firmer, and say "The client MUST avoid".  No?
>>>>>>> If not, why not?
>>>>>>> 
>>>>>>>  [Naren] We will change it to a MUST
>>>>>> Hold off on this, because there's still discussion based on Adrian's message in another thread... which I'll bring back here:
>>>>>> 
>>>>>> On Tue, Dec 8, 2015 at 4:31 PM, Adrien de Croy mailto:[e-mail] wrote:
>>>>>>> The proposal that a client MUST avoid LITERAL+/NSLs presumes there is 
>>>>>>> a limit when in fact there may actually not be one.  Of course there 
>>>>>>> is always a finite limit, but there may be no policy limit.  In fact 
>>>>>>> we don't plan to implement the limit as we've never had a request for 
>>>>>>> it and don't see a need to deny authenticated users from appending a 
>>>>>>> mail (and see some dangers in that).
>>>>>>> 
>>>>>>> I think MAY works in that it proposes a strategy, and doesn't confuse 
>>>>>>> issues with servers that already implement LITERAL+ but not a limit.  
>>>>>>> Otherwise you may be placing a new requirement on old software to 
>>>>>>> police the new MUST, or implementing the limit places addition 
>>>>>>> requirements to alter behaviour of
>>>>>>> LITERAL+ support to enforce this which IMO over-complicates it.
>>>>>> But the point of the use of LITERAL+ with APPEND isn't just about this spec and overall limits -- it's about whether we should use LITERAL+ with APPEND *at all*.  There are other reasons that any particular APPEND might fail, and one point of using literals (and *never* allowing quoted strings, for example) is exactly to give the server a chance to say "NO" to the APPEND *before* the message data is shipped over.  Using LITERAL+ for APPEND data violates that.
>>>>>> 
>>>>>> It was always the intent of LITERAL+ that it be used as a way to eliminate the extra round trip on short strings, where the OK from the server isn't necessary -- such as for username and password at login, or for mailbox names in various places (including the mailbox name in an APPEND command.
>>>>>> 
>>>>>> My point here is that we now have an opportunity to stress this: that it's not a generally good idea to use LITERAL+ for the message data in an APPEND command, because it doesn't give the server the opportunity to say, "OK, yes, go ahead and send me the message."
>>>>>> 
>>>>>> I'm absolutely willing to accept that "MUST NOT" use LITERAL+ for that is too strong.  But I'm still going to hold out for "SHOULD NOT", and would like to continue the discussion of why you disapprove.
>>>>>> 
>>>>>> [Jay] "SHOULD NOT" looks reasonable to me, as per the above explanation.
>>>>>> 
>>>>>>> "STATUS APPENDLIMIT is considered to be fast"
>>>>>>> 
>>>>>>> What does that mean?
>>>>>>> Is this meant to be advice to the server, to make sure that it *is* 
>>>>>>> fast?  If so, then say it more that way.  If not, then please tell me what it's supposed to mean.
>>>>>>> 
>>>>>>> [Naren] It's not an advice to the server, we are trying to convey that 
>>>>>>> this approach is considered to be faster than evaluating the remaining quota for a mailbox.
>>>>>> "Is considered to be" is not useful specification language.  If you want this to affect what clients or servers do, you should word this in terms that more specifically tell them what you want them to do.
>>>>>> Please come up with a better way to say what you really mean here.
>>>>>> 
>>>>>> Let me take a stab, and you can see if this has the right sense (edit as appropriate, of course):
>>>>>> 
>>>>>> OLD
>>>>>>    STATUS APPENDLIMIT is considered to be fast and there is no need to
>>>>>>    evaluate remaining quotas (if any) when returning APPENDLIMIT values.
>>>>>>    APPEND can still fail due to ACL and quota related issues, even if the
>>>>>>    message being appended is smaller than the APPENDLIMIT.
>>>>>> 
>>>>>> NEW
>>>>>>    Computing the APPENDLIMIT should be fast, and need not take ACLs,
>>>>>>    quotas, and other such information into account.  The APPENDLIMIT
>>>>>>    specifies one part of the policy, but an APPEND command can still fail
>>>>>>    due to issues related to ACLs and quotas issues, even if the message
>>>>>>    being appended is smaller than the APPENDLIMIT.
>>>>>> 
>>>>>> END
>>>>>> [Jay] Fine
>>>>>> 
>>>>>> Also, why is this in Section 4, rather than in, say, a Section 3.3?
>>>>>> It has nothing to do with the APPEND response, which is the subject of Section 4.
>>>>>> 
>>>>>> [Jay] Alright
>>>>>>> -- Section 6 --
>>>>>>> 
>>>>>>> "The IMAP APPENDLIMIT extension described in this document can  
>>>>>>> conceivably be used to facilitate Denial-of-Service attacks.
>>>>>>>  Specifically, the information contained in the APPENDLIMIT capability  
>>>>>>> and use of the APPEND command make it somewhat quicker and easier to  
>>>>>>> devise an efficacious Denial-of-Service attack."
>>>>>>> 
>>>>>>> What?  How? If it's true (it might be, but you haven't said), then you 
>>>>>>> should explain it.
>>>>>>> 
>>>>>>> [Naren] What we meant here is that using this extension it will easy 
>>>>>>> and quicker to get the upload limit and using that limit against the 
>>>>>>> server.
>>>>>> Why?  I don't understand why that helps anyone mount an attack.  And in any case, it needs to be explained here in the document.
>>>>>> 
>>>>>> And you didn't respond to these questions in my review:
>>>>>> 
>>>>>> 
>>>>>> [Jay] If the APPENDLIMIT is known beforehand, it's easy to overwhelm server with huge data which is beyond the APPENDLIMIT.
>>>>>> This might facilitate  Denial-of-Service attacks.
>>>>>> Makes sense?
>>>>>> 
>>>>>>> "In addition, no
>>>>>>>  issues are addressed involving trusted systems and possible release  of information via the mechanisms described in this document."
>>>>>>> 
>>>>>>> What does this mean, and why is it here?
>>>>>> [Jay] We will remove this
>>>>>>> "IMAP APPENDLIMIT extension doesn't add any new security considerations  that are not already present in the base IMAP protocol [RFC3501]."
>>>>>>> 
>>>>>>> Doesn't the presence of the first paragraph belie that?
>>>>>> [Jay] Yes, we will modify accordingly
>>>>>> 
>>>>>> Barry
>>>>>> 
>>>>>> _______________________________________________
>>>>>> imapext mailing list
>>>>>> [e-mail]
>>>>>> https://www.ietf.org/mailman/listinfo/imapext
>>>>>> 
>>>>>> _______________________________________________
>>>>>> imapext mailing list
>>>>>> [e-mail]
>>>>>> https://www.ietf.org/mailman/listinfo/imapext
>