Re: draft-ietf-httpbis-header-structure-00, unicode range
Kari Hurtta <hurtta-ietf@elmme-mailer.org> Tue, 13 December 2016 18:11 UTC
Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2EA7F12953F for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Tue, 13 Dec 2016 10:11:58 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.797
X-Spam-Level:
X-Spam-Status: No, score=-9.797 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.001, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-2.896, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1Qg23CT29reB for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Tue, 13 Dec 2016 10:11:56 -0800 (PST)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 629B8129463 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Tue, 13 Dec 2016 10:11:56 -0800 (PST)
Received: from lists by frink.w3.org with local (Exim 4.80) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1cGrWR-0000iG-ND for ietf-http-wg-dist@listhub.w3.org; Tue, 13 Dec 2016 18:09:51 +0000
Resent-Date: Tue, 13 Dec 2016 18:09:51 +0000
Resent-Message-Id: <E1cGrWR-0000iG-ND@frink.w3.org>
Received: from titan.w3.org ([128.30.52.76]) by frink.w3.org with esmtps (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from <hurtta@siilo.fmi.fi>) id 1cGrWL-0000gJ-UG for ietf-http-wg@listhub.w3.org; Tue, 13 Dec 2016 18:09:45 +0000
Received: from smtpvgate.fmi.fi ([193.166.223.36]) by titan.w3.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.84_2) (envelope-from <hurtta@siilo.fmi.fi>) id 1cGrWF-00085V-Fu for ietf-http-wg@w3.org; Tue, 13 Dec 2016 18:09:40 +0000
Received: from basaari.fmi.fi (basaari.fmi.fi [193.166.211.14]) (envelope-from hurtta@siilo.fmi.fi) by smtpVgate.fmi.fi (8.13.8/8.13.8/smtpgate-20161014/smtpVgate) with ESMTP id uBDI90kt021280 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 13 Dec 2016 20:09:00 +0200
Received: from shell.siilo.fmi.fi by basaari.fmi.fi with ESMTP id uBDI90cx019275 ; Tue, 13 Dec 2016 20:09:00 +0200
Received: from shell.siilo.fmi.fi ([127.0.0.1]) by shell.siilo.fmi.fi with ESMTP id uBDI90a4005600 ; Tue, 13 Dec 2016 20:09:00 +0200
Received: by shell.siilo.fmi.fi id uBDI8xLV005599; Tue, 13 Dec 2016 20:08:59 +0200
Message-Id: <201612131808.uBDI8xLV005599@shell.siilo.fmi.fi>
In-Reply-To: <20161213175419.GA7943@LK-Perkele-V2.elisa-laajakaista.fi>
References: <20161213173327.C1F7D1714B@welho-filter2.welho.com> <20161213175419.GA7943@LK-Perkele-V2.elisa-laajakaista.fi>
To: Ilari Liusvaara <ilariliusvaara@welho.com>
Date: Tue, 13 Dec 2016 20:08:59 +0200
Sender: hurtta@siilo.fmi.fi
From: Kari Hurtta <hurtta-ietf@elmme-mailer.org>
CC: Kari Hurtta <hurtta-ietf@elmme-mailer.org>, HTTP working group mailing list <ietf-http-wg@w3.org>, Poul-Henning Kamp <phk@varnish-cache.org>
X-Mailer: ELM [version ME+ 2.5 PLalpha44]
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="US-ASCII"
X-Filter: smtpVgate.fmi.fi: 3 received headers rewritten with id 20161213/07316/01
X-Filter: smtpVgate.fmi.fi: ID 7316/01, 1 parts scanned for known viruses
X-Filter: basaari.fmi.fi: ID 296681/01, 1 parts scanned for known viruses
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (smtpVgate.fmi.fi [193.166.223.36]); Tue, 13 Dec 2016 20:09:01 +0200 (EET)
Received-SPF: none client-ip=193.166.223.36; envelope-from=hurtta@siilo.fmi.fi; helo=smtpVgate.fmi.fi
X-W3C-Hub-Spam-Status: No, score=-6.5
X-W3C-Hub-Spam-Report: AWL=0.513, BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-3.099, W3C_AA=-1, W3C_WL=-1
X-W3C-Scan-Sig: titan.w3.org 1cGrWF-00085V-Fu 1fb60cd52b9ef660071e7156ecdfe2db
X-Original-To: ietf-http-wg@w3.org
Subject: Re: draft-ietf-httpbis-header-structure-00, unicode range
Archived-At: <http://www.w3.org/mid/201612131808.uBDI8xLV005599@shell.siilo.fmi.fi>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/33167
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>
Ilari Liusvaara <ilariliusvaara@welho.com>: (Tue Dec 13 19:54:19 2016) > On Tue, Dec 13, 2016 at 07:33:26PM +0200, Kari Hurtta wrote: > > 2. Definition of HTTP Header Common Structure > > https://tools.ietf.org/html/draft-ietf-httpbis-header-structure-00#section-2 > > > > | unicode_string = * unicode_codepoint > > | # XXX: Is there a place to import this from ? > > | # Unrestricted unicode, because there is no sane > > | # way to restrict or otherwise make unicode "safe". > > > > What is range of unicode_codepoint ? > > > > Next section implies that it is only plane 0 ? > > ( range 0x0000 - 0xFFFF ) > > > > 3. HTTP/1 Serialization of HTTP Header Common Structure > > https://tools.ietf.org/html/draft-ietf-httpbis-header-structure-00#section-3 > > > > | h1_unicode_string = DQUOTE *( > > | ( "\" DQUOTE ) > > | ( "\" "\" ) / > > | ( "\" "u" 4*HEXDIG ) / > > | 0x20-21 / > > | 0x23-5B / > > | 0x5D-7E / > > | UTF8-2 / > > | UTF8-3 / > > | UTF8-4 > > | ) DQUOTE > > | # This is UTF8 with HTTP1 unfriendly codepoints > > | # (00-1f, 7f) neutered with \uXXXX escapes. > > > > > > here seems unicode to limited to plane 0 > > ( 0x0000 - 0xFFFF ) > > > > Or is unicode values > 0xFFFF > > encoded with surrogates (values 0xd8000 - 0xdffff) ? > > ( UCS-2 or UTF-16 is used ) > > Well, that production lists UTF8-4, which is presumably 4-byte UTF-8 > sequences, and all valid ones are astral plane codepoints. OK. > Also, if one is to use native unicode (UTF-8), escape sequences are > only needed for range 0-159. Then that ( "\" "u" 4*HEXDIG ) / is overkill. > Or otherwise you also need escapes for > astral planes (and I hope the escape system there would be more sane > than the one JSON has...) That ( "\" "u" 4*HEXDIG ) produces same than JSON have when UTF-16 is used: https://tools.ietf.org/html/rfc4627#section-2.5 To escape an extended character that is not in the Basic Multilingual Plane, the character is represented as a twelve-character sequence, encoding the UTF-16 surrogate pair. So, for example, a string containing only the G clef character (U+1D11E) may be represented as "\uD834\uDD1E". > > -Ilari / Kari Hurtta
- draft-ietf-httpbis-header-structure-00, unicode r… Kari Hurtta
- Re: draft-ietf-httpbis-header-structure-00, unico… Ilari Liusvaara
- Re: draft-ietf-httpbis-header-structure-00, unico… Kari Hurtta
- Re: draft-ietf-httpbis-header-structure-00, unico… Poul-Henning Kamp
- Re: draft-ietf-httpbis-header-structure-00, unico… Poul-Henning Kamp
- Re: draft-ietf-httpbis-header-structure-00, unico… Ilari Liusvaara
- Re: draft-ietf-httpbis-header-structure-00, unico… Poul-Henning Kamp
- Unicode escape sequence | Re: draft-ietf-httpbis-… Kari Hurtta
- Re: draft-ietf-httpbis-header-structure-00, unico… Julian Reschke
- Re: Unicode escape sequence | Re: draft-ietf-http… Poul-Henning Kamp
- Re: draft-ietf-httpbis-header-structure-00, unico… Martin J. Dürst
- Re: Unicode escape sequence | Re: draft-ietf-http… Matthew Kerwin
- Re: Unicode escape sequence | Re: draft-ietf-http… Poul-Henning Kamp
- Re: Unicode escape sequence | Re: draft-ietf-http… Martin Thomson
- Re: Unicode escape sequence | Re: draft-ietf-http… Alexey Melnikov
- Re: draft-ietf-httpbis-header-structure-00, unico… Julian Reschke
- Re: Unicode escape sequence | Re: draft-ietf-http… Julian Reschke
- Re: Unicode escape sequence | Re: draft-ietf-http… Julian Reschke
- Re: Unicode escape sequence | Re: draft-ietf-http… Poul-Henning Kamp
- Re: Unicode escape sequence | Re: draft-ietf-http… Julian Reschke
- Re: Unicode escape sequence | Re: draft-ietf-http… Alexey Melnikov
- Re: Unicode escape sequence | Re: draft-ietf-http… Julian Reschke
- Re: Unicode escape sequence | Re: draft-ietf-http… Poul-Henning Kamp
- Re: Unicode escape sequence | Re: draft-ietf-http… Martin Thomson
- Re: Unicode escape sequence | Re: draft-ietf-http… Matthew Kerwin
- Re: Unicode escape sequence | Re: draft-ietf-http… Julian Reschke
- Re: Unicode escape sequence | Re: draft-ietf-http… Matthew Kerwin
- Re: Unicode escape sequence | Re: draft-ietf-http… Poul-Henning Kamp
- Re: Unicode escape sequence | Re: draft-ietf-http… Poul-Henning Kamp
- Re: Unicode escape sequence | Re: draft-ietf-http… Kari Hurtta
- Re: Unicode escape sequence | Re: draft-ietf-http… Matthew Kerwin
- Re: Unicode escape sequence | Re: draft-ietf-http… Kazuho Oku
- Re: Unicode escape sequence | Re: draft-ietf-http… Daurnimator
- Re: Unicode escape sequence | Re: draft-ietf-http… Mark Nottingham
- Re: Unicode escape sequence | Re: draft-ietf-http… Poul-Henning Kamp
- Re: Unicode escape sequence | Re: draft-ietf-http… Mark Nottingham
- Re: Unicode escape sequence | Re: draft-ietf-http… Julian Reschke
- Re: Unicode escape sequence | Re: draft-ietf-http… Poul-Henning Kamp
- Re: draft-ietf-httpbis-header-structure-00, unico… Mark Nottingham
- Re: draft-ietf-httpbis-header-structure-00, unico… Poul-Henning Kamp
- Re: Unicode escape sequence | Re: draft-ietf-http… Martin J. Dürst