Re: draft-ietf-httpbis-header-structure-00, unicode range

[Ilari Liusvaara <[e-mail]>]

On Tue, Dec 13, 2016 at 07:33:26PM +0200, Kari Hurtta wrote:
> 2.  Definition of HTTP Header Common Structure
> https://tools.ietf.org/html/draft-ietf-httpbis-header-structure-00#section-2
> 
> |     unicode_string = * unicode_codepoint
> |             # XXX: Is there a place to import this from ?
> |             # Unrestricted unicode, because there is no sane
> |             # way to restrict or otherwise make unicode "safe".
> 
> What is range of unicode_codepoint ?
> 
> Next section implies that it is only plane 0 ?
> ( range 0x0000 - 0xFFFF )
> 
> 3.  HTTP/1 Serialization of HTTP Header Common Structure
> https://tools.ietf.org/html/draft-ietf-httpbis-header-structure-00#section-3
> 
> |     h1_unicode_string = DQUOTE *(
> |                         ( "\" DQUOTE )
> |                         ( "\" "\" ) /
> |                         ( "\" "u" 4*HEXDIG ) /
> |                         0x20-21 /
> |                         0x23-5B /
> |                         0x5D-7E /
> |                         UTF8-2 /
> |                         UTF8-3 /
> |                         UTF8-4
> |                         ) DQUOTE
> |     # This is UTF8 with HTTP1 unfriendly codepoints
> |     # (00-1f, 7f) neutered with \uXXXX escapes.
> 
> 
> here seems unicode to limited to plane 0
> ( 0x0000 - 0xFFFF )
> 
> Or is unicode values > 0xFFFF
> encoded with surrogates  (values 0xd8000 - 0xdffff) ?
> ( UCS-2 or UTF-16 is used )

Well, that production lists UTF8-4, which is presumably 4-byte UTF-8
sequences, and all valid ones are astral plane codepoints.

Also, if one is to use native unicode (UTF-8), escape sequences are
only needed for range 0-159. Or otherwise you also need escapes for
astral planes (and I hope the escape system there would be more sane
than the one JSON has...)


-Ilari