Re: [Json] Call for Consensus: Proposed Text for "8.1 Character Encoding"
"Matthew A. Miller" <linuxwolf+ietf@outer-planes.net> Mon, 20 March 2017 16:26 UTC
Return-Path: <linuxwolf+ietf@outer-planes.net>
X-Original-To: json@ietfa.amsl.com
Delivered-To: json@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A14011294D4 for <json@ietfa.amsl.com>; Mon, 20 Mar 2017 09:26:24 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.235
X-Spam-Level:
X-Spam-Status: No, score=-1.235 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_SOFTFAIL=0.665] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=outer-planes-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id acvMwBi0Z4zU for <json@ietfa.amsl.com>; Mon, 20 Mar 2017 09:26:23 -0700 (PDT)
Received: from mail-ot0-x241.google.com (mail-ot0-x241.google.com [IPv6:2607:f8b0:4003:c0f::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D76881286AB for <json@ietf.org>; Mon, 20 Mar 2017 09:26:22 -0700 (PDT)
Received: by mail-ot0-x241.google.com with SMTP id x37so19821046ota.1 for <json@ietf.org>; Mon, 20 Mar 2017 09:26:22 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=outer-planes-net.20150623.gappssmtp.com; s=20150623; h=sender:subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to; bh=UgQyLafM06MkwyOQVd08E8fdO3EPurKgh1XUOOn6uxo=; b=OrznMIy6VEq8CRhkn0YkhtGUfX67pZ771ns2ORvM6iQ2fj1zkqQ7G0OtMBZHUcxKNd 4nh1FdNPU20IiA5U98SyHmuR1TNMGBkpWXEQ/NbZAH5/L313TqsJAZ1OMjCUG8wirU3Q ovOiqqHpiHFZycZnDsLJauKESYBKZltOQ/BAcGuehRw22DaAW8+jpveyTprhFJdgUlKD BG4prszrg1ErNFdAEQJzGqg0bhHyj/vAnSxoVmNfnAdp7lHvSKwXxUdiZAZtOQ1tsUjm xwatg9B5xqkJdv0ToedWR3ACby192sywROCJbNygxO4EtOMGrADbRg2mQKStY4lTBnn4 6AEw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:message-id :date:user-agent:mime-version:in-reply-to; bh=UgQyLafM06MkwyOQVd08E8fdO3EPurKgh1XUOOn6uxo=; b=Dps1PnCS7k/WM2+tWjlNyAk/rfqdEvc5wkIM2me77e9NNhg2TS25U4TARYl5w8ZX6h 9moHjA5KOdBu5U1FYPnTf4TYbMdHGi9NaCs7N814uAzveIr38NojRiUrVRlKLby3Gw0p zepXvTwJoS5qsAEgv0ACN6cPJTb7bBThYpk1kowpgCY1jvDQd5kE9JoZ+9BInqkY/iAI TNI3i1MF77leH0Q69/zZmeAzFuNB+xOpMNN427Yxosxr9w6J/PlBxNXJ2xiiyg5R/LKW o/2qXFQtmdZZIycnGDoBHVUqp5SH+WlN5u6vW8wKehfD3XtNMpWFNemEZhHiLQNbnpks vGBg==
X-Gm-Message-State: AFeK/H0KjFFhrRsHXqi4Wwmx+Mo734ur5VOCUaq2Hwq+zxOXCk3NmywwOKxy5oW2SHpnDg==
X-Received: by 10.157.89.143 with SMTP id u15mr14942469oth.176.1490027182226; Mon, 20 Mar 2017 09:26:22 -0700 (PDT)
Received: from [10.6.23.170] ([128.177.113.102]) by smtp.gmail.com with ESMTPSA id q31sm2917179ota.6.2017.03.20.09.26.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 20 Mar 2017 09:26:21 -0700 (PDT)
Sender: Matthew Miller <linuxwolf@outer-planes.net>
To: Julian Reschke <julian.reschke@gmx.de>, "json@ietf.org" <json@ietf.org>
Cc: The IESG <iesg@ietf.org>
References: <1fb5849e-8dbf-835d-65b7-2403686248f9@outer-planes.net> <0E32A94D-CE12-4F52-9ED6-8743C49751B4@vpnc.org> <4d2f0fb3-a729-0c17-2394-bc1e005dd612@gmx.de> <d09f9a59-2411-45a0-470c-ea95072fe4fd@outer-planes.net> <dad91b19-e774-e239-36d2-9d086cca8e0d@gmx.de> <ac432615-ee84-3cdf-6b37-480626bd18c1@gmx.de>
From: "Matthew A. Miller" <linuxwolf+ietf@outer-planes.net>
Message-ID: <804f9930-26a5-a565-0607-452b386cfeb5@outer-planes.net>
Date: Mon, 20 Mar 2017 10:26:20 -0600
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.0
MIME-Version: 1.0
In-Reply-To: <ac432615-ee84-3cdf-6b37-480626bd18c1@gmx.de>
Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="HtUvPglAb1StOSRo7i7X66880bAHT0a5I"
Archived-At: <https://mailarchive.ietf.org/arch/msg/json/Voi2X-4LHj8WjQHm_8uCC4fT2L8>
Subject: Re: [Json] Call for Consensus: Proposed Text for "8.1 Character Encoding"
X-BeenThere: json@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: "JavaScript Object Notation \(JSON\) WG mailing list" <json.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/json>, <mailto:json-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/json/>
List-Post: <mailto:json@ietf.org>
List-Help: <mailto:json-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/json>, <mailto:json-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 20 Mar 2017 16:26:25 -0000
Thank you for the suggested changes, Julian. To consolidate the changes, I believe the following is your suggested text for all of Section 8.1: """ JSON text MUST be encoded in UTF-8, UTF-16, or UTF-32 Section 3 of [UNICODE]. The default encoding is UTF-8, and JSON texts that are encoded in UTF-8 are interoperable in the sense that they will be read successfully by the maximum number of implementations; there are many implementations that cannot successfully read texts in other encodings (such as UTF-16 and UTF-32). Text encoded in character encodings other than UTF-8, UTF-16, or UTF-32 cannot be used with the media tye "application/json". Implementations MUST NOT add a byte order mark (U+FEFF) to the beginning of a JSON text. In the interests of interoperability, implementations that parse JSON texts MAY ignore the presence of a byte order mark rather than treating it as an error. Recipients that wish to support Unicode encodings other than UTF-8 can do this using a detection mechanism that is based on the fact that the first character will always have a Unicode code point less or equal than 127, thus the UTF-16/32 variants can be detected by inspecting the first octets for nulls. """ Does the working group object to this change? - m&m Matthew A. Miller JSONbis Chair On 17/03/19 04:29, Julian Reschke wrote: > ...and here is a concrete proposal: > > Original text: > >> 8.1. Character Encoding >> >> JSON text SHALL be encoded in UTF-8, UTF-16, or UTF-32 [UNICODE] >> (Section 3). The default encoding is UTF-8, and JSON texts that are > > Change 1: > > Say "MUST" instead of "SHALL", as it's the more common form of > expressing this requirement. > > Change 2: > > Replace "[UNICODE] (Section 3)" by "Section 3 of [UNICODE]". > > That said, this citation isn't as stable as it should, as [UNICODE] > refers to <http://www.unicode.org/versions/latest/> and unless I'm > missing something, there's no guarantee that future versions will have > the relevant bits in Section 3. > >> encoded in UTF-8 are interoperable in the sense that they will be >> read successfully by the maximum number of implementations; there are >> many implementations that cannot successfully read texts in other >> encodings (such as UTF-16 and UTF-32). > > Change 3: > > Add "Text encoded in character encodings other than UTF-8, UTF-16, or > UTF-32 can not be used with the media type "application/json". > > (this explains the implications of the SHALL/MUST) > > >> Implementations MUST NOT add a byte order mark (U+FEFF) to the >> beginning of a JSON text. In the interests of interoperability, >> implementations that parse JSON texts MAY ignore the presence of a >> byte order mark rather than treating it as an error. > > > Finally, change 4: > > Add a new paragraph: > > "Recipients that wish to support Unicode encodings other than UTF-8 can > do this using a detection mechanism that is based on the fact that the > first character will always have a Unicode code point less or equal than > 127, thus the UTF-16/32 variants can be detected by inspecting the first > octets for nulls." > > > I believe none of these changes affects anything normative, but that > they absolutely clarify the spec. In particular, having them in the spec > would have avoided this whole discussion we just had. > > Best regards, Julian
- Re: [Json] Call for Consensus: Proposed Text for … HANSEN, TONY L
- Re: [Json] Call for Consensus: Proposed Text for … Nico Williams
- Re: [Json] Call for Consensus: Proposed Text for … Matthew A. Miller
- Re: [Json] Call for Consensus: Proposed Text for … Carsten Bormann
- Re: [Json] Call for Consensus: Proposed Text for … Martin J. Dürst
- Re: [Json] Call for Consensus: Proposed Text for … Tim Bray
- Re: [Json] Call for Consensus: Proposed Text for … John Cowan
- Re: [Json] Call for Consensus: Proposed Text for … HANSEN, TONY L
- Re: [Json] Call for Consensus: Proposed Text for … Pete Cordell
- Re: [Json] Call for Consensus: Proposed Text for … Carsten Bormann
- Re: [Json] Call for Consensus: Proposed Text for … Pete Cordell
- Re: [Json] Call for Consensus: Proposed Text for … Martin J. Dürst
- Re: [Json] Call for Consensus: Proposed Text for … Pete Cordell
- Re: [Json] Call for Consensus: Proposed Text for … Matthew A. Miller
- [Json] Call for Consensus: Proposed Text for "8.1… Matthew Miller
- Re: [Json] Call for Consensus: Proposed Text for … Carsten Bormann
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … Nico Williams
- Re: [Json] Call for Consensus: Proposed Text for … Nico Williams
- Re: [Json] Call for Consensus: Proposed Text for … Carsten Bormann
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … Carsten Bormann
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … Peter Cordell
- Re: [Json] Call for Consensus: Proposed Text for … Carsten Bormann
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … Tim Bray
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … Carsten Bormann
- Re: [Json] Call for Consensus: Proposed Text for … Peter Cordell
- Re: [Json] Call for Consensus: Proposed Text for … Carsten Bormann
- Re: [Json] Call for Consensus: Proposed Text for … Martin J. Dürst
- Re: [Json] Call for Consensus: Proposed Text for … Peter Cordell
- Re: [Json] Call for Consensus: Proposed Text for … Paul Hoffman
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … Matthew A. Miller
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … Carsten Bormann
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … Matthew A. Miller
- Re: [Json] Call for Consensus: Proposed Text for … John Cowan
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … Peter Cordell
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … HANSEN, TONY L
- Re: [Json] Call for Consensus: Proposed Text for … Pete Cordell
- Re: [Json] Call for Consensus: Proposed Text for … Matthew A. Miller
- Re: [Json] Call for Consensus: Proposed Text for … HANSEN, TONY L
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … HANSEN, TONY L
- Re: [Json] Call for Consensus: Proposed Text for … John Cowan
- Re: [Json] Call for Consensus: Proposed Text for … Matthew A. Miller
- Re: [Json] Call for Consensus: Proposed Text for … Pete Cordell
- Re: [Json] Call for Consensus: Proposed Text for … Nico Williams
- Re: [Json] Call for Consensus: Proposed Text for … John Cowan
- [Json] FW: Call for Consensus: Proposed Text for … Manger, James
- Re: [Json] FW: Call for Consensus: Proposed Text … John Cowan
- Re: [Json] FW: Call for Consensus: Proposed Text … Manger, James
- Re: [Json] Call for Consensus: Proposed Text for … Martin J. Dürst
- Re: [Json] Call for Consensus: Proposed Text for … Martin J. Dürst
- Re: [Json] Call for Consensus: Proposed Text for … Matthew A. Miller
- Re: [Json] Call for Consensus: Proposed Text for … Martin J. Dürst
- Re: [Json] Call for Consensus: Proposed Text for … Tim Bray
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … Pete Cordell
- Re: [Json] Call for Consensus: Proposed Text for … Matthew A. Miller
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … Pete Cordell
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … Carsten Bormann
- Re: [Json] Call for Consensus: Proposed Text for … Pete Cordell
- Re: [Json] Call for Consensus: Proposed Text for … Matthew A. Miller
- Re: [Json] Call for Consensus: Proposed Text for … Joe Hildebrand
- Re: [Json] Call for Consensus: Proposed Text for … John Cowan
- Re: [Json] Call for Consensus: Proposed Text for … Pete Cordell
- Re: [Json] Call for Consensus: Proposed Text for … Tim Bray
- Re: [Json] Call for Consensus: Proposed Text for … Matthew A. Miller
- Re: [Json] Call for Consensus: Proposed Text for … Martin J. Dürst
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … Alexey Melnikov
- Re: [Json] Call for Consensus: Proposed Text for … Pete Cordell
- Re: [Json] Call for Consensus: Proposed Text for … Allen Wirfs-Brock
- Re: [Json] Call for Consensus: Proposed Text for … Tim Bray
- Re: [Json] Call for Consensus: Proposed Text for … Julian Reschke
- Re: [Json] Call for Consensus: Proposed Text for … Alexey Melnikov
- Re: [Json] Call for Consensus: Proposed Text for … Pete Cordell