Re: #409: is parsing OBS-FOLD mandatory?

[Mark Nottingham <[e-mail]>]

Anyone? Bueller?

To make it clear -- I propose we downgrade recipient support for OBS-FOLD to a prose recommendation. 

Am happy to hear well-reasoned opinions to the contrary (but see note about interop testing below).


On 09/01/2013, at 9:54 PM, Mark Nottingham <[e-mail]> wrote:

> 
> On 19/12/2012, at 6:18 PM, Willy Tarreau <[e-mail]> wrote:
> 
>> On Wed, Dec 19, 2012 at 11:55:26AM +1100, Mark Nottingham wrote:
>>> 
>>> On 13/12/2012, at 8:18 AM, Willy Tarreau <[e-mail]> wrote:
>>> 
>>>> On Wed, Dec 12, 2012 at 10:18:28AM -0800, Roy T. Fielding wrote:
>>>> (...)
>>>>>> """
>>>>>> If a received protocol element is processed, the recipient MUST be able to parse any value that would match the ABNF rules for that protocol element, excluding only those rules not applicable to the recipient's role, and those rules whose names begin with "obs-" (e.g., obs-fold).
>>>>>> """
>>>>> 
>>>>> Do we really want to exclude non-ASCII octets (obs-text) and older
>>>>> date formats (obs-date)?  Do we demote them to SHOULD or MAY?
>>>> 
>>>> This is a good point. Line-folding causes security issues and does not
>>>> seem to be used by senders, but I think we all regularly catch some
>>>> obs-text and obs-date come from old applications or crippled devices.
>>>> 
>>>>> This change is fine with me, but it is a hard break from retaining
>>>>> compatibility and we need to be absolutely sure we want to do that.
>>>> 
>>>> I'd rather not break these ones, personally.
>>>> 
>>>> Couldn't we settle on just stating that obs-fold is normally not used,
>>>> is known to cause security issues when improperly implemented, and
>>>> should either be completely supported, or rejected, but in all cases
>>>> must be detected ?
>>> 
>>> 
>>> I'm OK with it either way, as long as we're clear about what we mean, as well
>>> as what's interoperable.
>>> 
>>> This statement is still ambiguous;
>>> 
>>> """
>>> As a convention, ABNF rule names prefixed with "obs-" denote "obsolete"
>>> grammar rules that appear for historical reasons.
>>> """
>>> 
>>> Perhaps we could start by 
>>> 
>>> 1) clarifying what "obsolete" means for senders and recipients in the error
>>> handling section (I think we've already done most of this, see other part of
>>> thread)
>>> 2) referring to that clarification in the ABNF statement quoted above (easy fix)
>>> 3) re-evaluating the use of the obs- prefix in each case (???)
>> 
>> That sounds like a reasonable plan.
>> 
>>> Personally, I think obs-date and obs-text are justified in having the prefix
>>> and resulting SHOULD, because they're not really interoperable.
>> 
>> OK. So "SHOULD" could be the default rule for obs-* and exceptions could
>> be handled specifically (eg: obs-fold).
> 
> 
> Right now, we have:
>  obs-fold - senders MUST NOT generate, recipients MUST accept
>  obs-text - new headers SHOULD NOT use, recipients SHOULD treat as opaque data 
>  obs-date - sender MUST NOT generate (implied by MUST), recipient parsers MUST accept
> 
> obs-fold was a "recipients SHOULD accept" before Roy's change. 
> 
> I suppose we could come up with a more rigid definition of what "obs-" means here, but that seems like it's sort of diminishing returns.
> 
> The issue here, I think, is whether or not we're requiring recipients to deal with line folding.
> 
> Some quick and dirty testing shows it's not interoperable. By sending requests in this form:
> 
> GET / HTTP/1.1
> Host:
>    [hostname]
> User-Agent: Foo/1.0
> 
> which our ABNF currently allows, I got errors on several sites. Given that, I wonder why requiring it to be accepted is necessary; making those sites non-conformant (as well as any clients that behave in a similar manner) isn't going to improve interop, and this feature isn't used in practice any way, precisely because it doesn't get interop. 
> 
> SHOULD makes sense; I could even see downgrading it to prose. I don't understand why this is a MUST, if we don't get good interop on it now, and it's not a feature that's in-demand, widely used, or important, AFAIK.
> 
> The same logic could be applied to obs-date, but I don't have hard data on how interoperable it is; AFAIK most implementations do a good job of covering the three possible formats.
> 
> The text around obs-text seems right, because it's such a different case.
> 
> How do others feel? Any additional data? I'm not adamant about any of this, I just want to a) make sure we all understand where we sit WRT obs-fold interop, and b) get this closed.
> 
> Cheers,
> 
> --
> Mark Nottingham   http://www.mnot.net/
> 
> 
> 
> 

--
Mark Nottingham   http://www.mnot.net/