Re: [http-state] Ticket 11: Character encoding for non-ASCII cookies values

[Mark Pauley <[e-mail]>]

On Mar 3, 2010, at 5:17 PM, Adam Barth wrote:

> See inline.
> 
> On Wed, Mar 3, 2010 at 4:51 PM, Mark Pauley <[e-mail]> wrote:
>> You're correct, we do allow those.  What I'm worried about though is that
>> valid UTF-8 sequences will contain \r\n .   In any case, writing good
>> parsers for a variable-length is really tough.
>> To give you a sense of what Safari will accept, we'll accept the following
>> for names or values in cookies:
>> CTL = [\x00-\x1F\x7F];
>> NETSCAPE_SPECIAL = ("," | ";" | "=" | SPACE | HTAB);
>> NONETSCAPE_VALCHAR = (CTL | NETSCAPE_SPECIAL);
>> NETSCAPE_CHAR = ((CHAR \ NONETSCAPE_VALCHAR) | COMMA_NOSPACE);
>> COMMA_NOSPACE = ((",")(CHAR \ (SPACE | HTAB | ";")));
>> NETSCAPE_VAL = (CHAR \ NONETSCAPE_VALCHAR)((NETSCAPE_CHAR | SPACE | HTAB)*
>> NETSCAPE_CHAR)?;
>> 
>> And NETSCAPE_VAL is effectively our TOKEN.
> 
> This grammar is significantly more complicated than what appears to be
> necessary.  Is there some reason the paring algorithm in
> <http://tools.ietf.org/html/draft-ietf-httpstate-cookie-04#section-5.2>
> is insufficient?
> 
>> We allow for quoted values, but those still reject the CTL class of
>> character.
> 
> Safari is currently the only browser to treat quote characters in
> cookie values differently from other characters.  (Note that Firefox
> used to have this behavior but changed recently to match IE and
> Chrome.)  Safari seems to have adopted this behavior in the past two
> years.  Source inside Apple have indicated that this change was made
> to improve compliance with RFC 2109 and not because of specific web
> compatibility concerns.

That's partially correct.  We received a bug from an external source that drove this, though we didn't immediately change because of the lack of compatibility issues as you say.

> 
>> So, we certainly will choke on many UTF-8 character sequences.  In that
>> case, we'll just ditch the cookie header as a whole.
> 
> Indeed.  Safari is currently the only browser that has this behavior.
> 
>> One of the main problems so far with cookies has been that the Date spec
>> used in the Netscape implementation clashes with the standard of HTTP to
>> separate values in headers by a comma token.  That is I believe, some sites
>> wish to set multiple cookies by using a comma separator whereas the comma is
>> also used as part of an unquoted field.  If we can verify that nobody is
>> actually trying to set multiple cookies per Set-Cookies header, then we can
>> easily pull this functionality and be a bit more blind to the character set
>> of the Set-Cookie header value.
> 
> The date parsing algorithm is specified in detail in
> <http://tools.ietf.org/html/draft-ietf-httpstate-cookie-04#section-5.1.1>.
> 
> Unlike most HTTP headers, a comma cannot be used to combine multiple
> Set-Cookie headers.  Safari appears to be the only browser that
> attempts to support this behavior.
> 
> If you haven't already, I'd encourage you to read the latest draft
> available at <http://tools.ietf.org/html/draft-ietf-httpstate-cookie>.
> Hopefully that will answer some number of your questions.

Interesting.  Well, this could certainly simplify much of our internal cookie parsing mechanism.  Is there going to be a push to deprecate RFC2109?  Could you point me at a page with results on how the browsers handle different Set-Cookie forms in practice?  Changing our behavior is nearly always dictated by broken compatibility with other sites.

> 
> Adam
> 
> 
>> On Mar 3, 2010, at 12:53 PM, Daniel Stenberg wrote:
>> 
>> On Wed, 3 Mar 2010, Mark Pauley wrote:
>> 
>> In the future, we ought to treat these as opaque octets.  However, the
>> current cookie spec would lead me to believe that we should reject any
>> cookies that contain control characters, which would be most non-ascii UTF-8
>> sequences, right?
>> 
>> Isn't the RFC2616 'token' a bit too strict for cookie-value ? The netscape
>> spec is _very_ liberal ("a sequence of characters excluding semi-colon,
>> comma and white space") so the current wording is a great deal more
>> restrictive.
>> 
>> Don't cookie implementations already allow and use for example ()<>:@? etc?
>> 
>> --
>> 
>> / daniel.haxx.se
>> 
>>