Re: [Json] BOMs (Was: Re: JSON: remove gap between Ecma-404 and IETF draft)
Tim Bray <tbray@textuality.com> Mon, 18 November 2013 15:58 UTC
Return-Path: <tbray@textuality.com>
X-Original-To: json@ietfa.amsl.com
Delivered-To: json@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CBEE411E80ED for <json@ietfa.amsl.com>; Mon, 18 Nov 2013 07:58:40 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.643
X-Spam-Level:
X-Spam-Status: No, score=-4.643 tagged_above=-999 required=5 tests=[AWL=-3.333, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-1, SARE_HTML_USL_OBFU=1.666]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lunIu4HfgE3K for <json@ietfa.amsl.com>; Mon, 18 Nov 2013 07:58:28 -0800 (PST)
Received: from mail-ve0-f182.google.com (mail-ve0-f182.google.com [209.85.128.182]) by ietfa.amsl.com (Postfix) with ESMTP id 6953011E8109 for <json@ietf.org>; Mon, 18 Nov 2013 07:55:02 -0800 (PST)
Received: by mail-ve0-f182.google.com with SMTP id pa12so5231356veb.13 for <json@ietf.org>; Mon, 18 Nov 2013 07:54:59 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=1ly1wnTOnxefXI8BFGqM88pymm3Kub9eoGLXhU29Qxs=; b=QbOHB8Gf8oSWG3HrIxmq9P9QDgSl8VyHayppeackIMUWr8/PqlwS7kI4eIld7GynMB jXIN/Regp2LXuBtN4uAF2kFuk/i6EKIiWH1fkGeIOlQNCVQQc5n59AqQqkkJ4fizZpKq gaoMs3+nmeMtNFvEpwCk4bs1PQ1on5BW46X+QObb36sDWtgTEYB1mHdqpQr6j0rBBJxl XQF46lfT+buOU0jkQYWrWkoo+/xQ3U3374NroW7vVWUYDMhg0rL0m7741bZA2nu9T1qe vdea/GPAaGmlXz4Ug6jEKxZPrJ/nKyp8wGCAbpa7nuX9I9WTQtvWgMYON1wzbUGHg7ka Tcbg==
X-Gm-Message-State: ALoCoQm0ojkQ/gi4TTtgbNJ/bBKn10OF6GzVMq+bDta2d9vCTFZE2teUQ6TSRSuI5jpFO1M4E6V3
MIME-Version: 1.0
X-Received: by 10.58.208.130 with SMTP id me2mr16190581vec.13.1384790099070; Mon, 18 Nov 2013 07:54:59 -0800 (PST)
Received: by 10.220.198.199 with HTTP; Mon, 18 Nov 2013 07:54:58 -0800 (PST)
X-Originating-IP: [24.84.235.32]
In-Reply-To: <C37B2FE59C164DBCA982AC81A56A09AA@codalogic>
References: <AA45B3C6-1DC5-4B1E-8045-C9FE76022584@vpnc.org> <CEA92854.2CC53%jhildebr@cisco.com> <20131113224737.GI31823@mercury.ccil.org> <f5bob5n71y7.fsf@troutbeck.inf.ed.ac.uk> <5284B095.4070004@it.aoyama.ac.jp> <C37B2FE59C164DBCA982AC81A56A09AA@codalogic>
Date: Mon, 18 Nov 2013 07:54:58 -0800
Message-ID: <CAHBU6ivieGAmNF=ZyMNoBCLO3q17E-J_g=pMN1jkfd1J_PW9iA@mail.gmail.com>
From: Tim Bray <tbray@textuality.com>
To: Pete Cordell <petejson@codalogic.com>
Content-Type: multipart/alternative; boundary="047d7bdc192cf1a68704eb7591fb"
Cc: John Cowan <cowan@mercury.ccil.org>, "Henry S. Thompson" <ht@inf.ed.ac.uk>, JSON WG <json@ietf.org>, Anne van Kesteren <annevk@annevk.nl>, es-discuss <es-discuss@mozilla.org>, "Martin J. Dürst" <duerst@it.aoyama.ac.jp>, "www-tag@w3.org" <www-tag@w3.org>, IETF Discussion <ietf@ietf.org>
Subject: Re: [Json] BOMs (Was: Re: JSON: remove gap between Ecma-404 and IETF draft)
X-BeenThere: json@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "JavaScript Object Notation \(JSON\) WG mailing list" <json.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/json>, <mailto:json-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/json>
List-Post: <mailto:json@ietf.org>
List-Help: <mailto:json-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/json>, <mailto:json-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Nov 2013 15:59:00 -0000
This feels backward, because BOMs are actually useful for UTF-16 and UTF-32, but essentially useless for UTF-8. On Mon, Nov 18, 2013 at 2:05 AM, Pete Cordell <petejson@codalogic.com>wrote: > Given the history below, would it be sensible to accept BOMs for UTF-8 > encoding, but not for UTF-16 and UTF-32? In other words, are BOMs needed > and/or used in the wild for UTF-16 and UTF-32? > > Maybe the text can say something like "SHOULD accept BOMs for UTF-8, and > MAY accept BOMs for UTF-16 and / or UTF-32"? > > Thanks, > > Pete Cordell > Codalogic Ltd > C++ tools for C++ programmers, http://codalogic.com > Read & write XML in C++, http://www.xml2cpp.com > ----- Original Message ----- From: ""Martin J. Dürst"" < > duerst@it.aoyama.ac.jp> > To: "Henry S. Thompson" <ht@inf.ed.ac.uk> > Cc: "John Cowan" <cowan@mercury.ccil.org>; "IETF Discussion" > <ietf@ietf.org>; "Paul Hoffman" <paul.hoffman@vpnc.org>; "JSON WG" > <json@ietf.org>; "Joe Hildebrand (jhildebr)" <jhildebr@cisco.com>; "Anne > van > Kesteren" <annevk@annevk.nl>; <www-tag@w3.org>; "es-discuss" > <es-discuss@mozilla.org> > Sent: Thursday, November 14, 2013 11:14 AM > Subject: Re: [Json] JSON: remove gap between Ecma-404 and IETF draft > > > Hello Henry, others, >> >> On 2013/11/14 18:44, Henry S. Thompson wrote: >> >>> John Cowan writes: >>> >>> Joe Hildebrand (jhildebr) scripsit: >>>> >>>> If 404 doesn't allow [a BOM], I don't see a strong need to add it. >>>>> Parsers can always be more forgiving of what they will parse than what >>>>> the spec says, particularly since section 9 says "A JSON parser MAY >>>>> accept non-JSON forms or extensions". >>>>> >>>> >>>> It's not clear that 404 disallows it, since 404 is defined in terms of >>>> characters, and a BOM is not a character but an out-of-band signal. >>>> >>> >>> I think this is a crucial observation. >>> >> >> Yes, and I think it's based on the experience with XML. But while this >> experience may be applicable to JSON, Anne's original comment about the >> BOM and XMLHttpRequest suggests that 404 actually currently does not >> tolerate a BOM, and that implementations (except for XMLHttpRequest) also >> don't. >> >> To give some historic background, the BOM for UTF-8 wasn't in the first >> edition of XML (http://www.w3.org/TR/1998/REC-xml-19980210#sec-guessing). >> It only later came in because Microsoft used it for notepad to be able to >> quickly distinguish between UTF-8 and the legacy system encoding. Because >> many people were writing some XML by hand, and some of them were using >> notepad, the pressure on XML to accept a BOM at the start of an UTF-8 file >> mounted, and it was included in the second edition of the XML >> Recommendation (http://www.w3.org/TR/2000/REC-xml-20001006#sec-guessing). >> >> Compared to XML, JSON may be much less edited by hand, or much less edited >> on notepad, or otherwise just have a different history from XML, but we >> have to make sure. >> >> Regards, Martin. >> >> >> I note that XML approaches >>> this problem in what might be a useful way. The XML ABNF makes no >>> mention of BOM, it's not part of any XML document as such. But it >>> _is_ allowed. The relevant wording [1] is: >>> >>> Entities ... may begin with the Byte Order Mark described by Annex H >>> of [ISO/IEC 10646:2000], section 16.8 of [Unicode] (the ZERO WIDTH >>> NO-BREAK SPACE character, #xFEFF). _This is an encoding signature,_ >>> _not part of either the markup or the character data of the XML_ >>> _document._ XML processors must be able to use this character to >>> differentiate between UTF-8 and UTF-16 encoded documents. [emphasis >>> added] >>> >>> ht >>> >>> [1] http://www.w3.org/TR/REC-xml/#charencoding >>> >> _______________________________________________ >> json mailing list >> json@ietf.org >> https://www.ietf.org/mailman/listinfo/json >> > > _______________________________________________ > json mailing list > json@ietf.org > https://www.ietf.org/mailman/listinfo/json >
- Re: [Json] JSON: remove gap between Ecma-404 and … Paul Hoffman
- Re: [Json] JSON: remove gap between Ecma-404 and … Joe Hildebrand (jhildebr)
- Re: [Json] JSON: remove gap between Ecma-404 and … Carsten Bormann
- Re: [Json] JSON: remove gap between Ecma-404 and … Joe Hildebrand (jhildebr)
- Re: [Json] JSON: remove gap between Ecma-404 and … Tony Hansen
- Re: [Json] JSON: remove gap between Ecma-404 and … Paul Hoffman
- Re: [Json] JSON: remove gap between Ecma-404 and … Allen Wirfs-Brock
- Re: [Json] JSON: remove gap between Ecma-404 and … Carsten Bormann
- Re: [Json] JSON: remove gap between Ecma-404 and … Joe Hildebrand
- Re: [Json] JSON: remove gap between Ecma-404 and … Joe Hildebrand (jhildebr)
- Re: [Json] JSON: remove gap between Ecma-404 and … Paul Hoffman
- Re: [Json] JSON: remove gap between Ecma-404 and … John Cowan
- Re: [Json] JSON: remove gap between Ecma-404 and … Joe Hildebrand (jhildebr)
- Re: [Json] JSON: remove gap between Ecma-404 and … Mark Davis ☕
- Re: [Json] JSON: remove gap between Ecma-404 and … John Cowan
- Re: [Json] JSON: remove gap between Ecma-404 and … Allen Wirfs-Brock
- Re: [Json] JSON: remove gap between Ecma-404 and … Paul Hoffman
- Re: [Json] Encoding detection John Cowan
- Re: [Json] JSON: remove gap between Ecma-404 and … Martin J. Dürst
- [Json] Encoding detection (Was: Re: JSON: remove … Pete Cordell
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Joe Hildebrand (jhildebr)
- Re: [Json] JSON: remove gap between Ecma-404 and … John Cowan
- Re: [Json] JSON: remove gap between Ecma-404 and … Henry S. Thompson
- Re: [Json] JSON: remove gap between Ecma-404 and … Allen Wirfs-Brock
- Re: [Json] JSON: remove gap between Ecma-404 and … Allen Wirfs-Brock
- Re: [Json] Encoding detection Henry S. Thompson
- Re: [Json] Encoding detection John Cowan
- Re: [Json] Encoding detection David Booth
- Re: [Json] Encoding detection Martin J. Dürst
- Re: [Json] Encoding detection Henry S. Thompson
- Re: [Json] Encoding detection Pete Cordell
- Re: [Json] Encoding detection Paul Hoffman
- Re: [Json] Encoding detection Bjoern Hoehrmann
- Re: [Json] Encoding detection Paul Hoffman
- [Json] BOMs (Was: Re: JSON: remove gap between Ec… Pete Cordell
- Re: [Json] Encoding detection Pete Cordell
- Re: [Json] BOMs Henry S. Thompson
- Re: [Json] BOMs Martin J. Dürst
- Re: [Json] BOMs Bjoern Hoehrmann
- Re: [Json] BOMs Henry S. Thompson
- Re: [Json] BOMs Pete Cordell
- Re: [Json] BOMs Bjoern Hoehrmann
- Re: [Json] BOMs (Was: Re: JSON: remove gap betwee… Tim Bray
- Re: [Json] BOMs (Was: Re: JSON: remove gap betwee… Pete Cordell
- Re: [Json] BOMs John Cowan
- Re: [Json] BOMs (Was: Re: JSON: remove gap betwee… Julian Reschke
- Re: [Json] BOMs (Was: Re: JSON: remove gap betwee… John Cowan
- Re: [Json] BOMs (Was: Re: JSON: remove gap betwee… Pete Cordell
- Re: [Json] BOMs (Was: Re: JSON: remove gap betwee… Pete Cordell
- Re: [Json] BOMs (Was: Re: JSON: remove gap betwee… John Cowan
- Re: [Json] BOMs Tatu Saloranta
- Re: [Json] BOMs Phillip Hallam-Baker
- Re: [Json] BOMs Martin J. Dürst
- Re: [Json] BOMs Martin J. Dürst
- Re: [Json] BOMs Pete Cordell
- Re: [Json] BOMs Bjoern Hoehrmann
- Re: [Json] BOMs t.p.
- Re: [Json] BOMs Allen Wirfs-Brock
- Re: [Json] BOMs Tatu Saloranta
- Re: [Json] BOMs Henry S. Thompson
- Re: [Json] BOMs Joe Hildebrand (jhildebr)
- Re: [Json] BOMs Chris Lilley
- Re: [Json] BOMs Pete Cordell
- Re: [Json] BOMs Henry S. Thompson
- Re: [Json] BOMs Martin J. Dürst
- Re: [Json] BOMs Henry S. Thompson
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Henri Sivonen
- Re: [Json] Encoding detection (Was: Re: JSON: rem… John Cowan
- Re: [Json] BOMs Allen Wirfs-Brock
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Henri Sivonen
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Anne van Kesteren
- Re: [Json] BOMs Henri Sivonen
- Re: [Json] Encoding detection (Was: Re: JSON: rem… John Cowan
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Joe Hildebrand (jhildebr)
- Re: [Json] BOMs Allen Wirfs-Brock
- Re: [Json] BOMs Bjoern Hoehrmann
- Re: [Json] BOMs Mathias Bynens
- Re: [Json] BOMs John Cowan
- Re: [Json] BOMs Bjoern Hoehrmann
- Re: [Json] BOMs Nico Williams
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Henri Sivonen
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Pete Cordell
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Julian Reschke
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Robin Berjon
- Re: [Json] Encoding detection (Was: Re: JSON: rem… John Cowan
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Tim Bray
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Allen Wirfs-Brock
- [Json] Consensus on JSON-text (WAS: JSON: remove … Matt Miller (mamille2)
- Re: [Json] BOMs Matt Miller (mamille2)
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Matt Miller (mamille2)
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Julian Reschke
- Re: [Json] BOMs Bjoern Hoehrmann
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Bjoern Hoehrmann
- Re: [Json] Encoding detection (Was: Re: JSON: rem… John Cowan
- Re: [Json] Encoding detection (Was: Re: JSON: rem… John Cowan
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Matt Miller (mamille2)
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Pete Cordell
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Pete Cordell
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Martin J. Dürst
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Allen Wirfs-Brock
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Bjoern Hoehrmann
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Henri Sivonen
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Nico Williams
- Re: [Json] Encoding detection (Was: Re: JSON: rem… John Cowan
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Jacob Davies
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Nico Williams
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Julian Reschke
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Nico Williams
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Carsten Bormann
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Nico Williams
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Carsten Bormann
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Nico Williams
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Bjoern Hoehrmann
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Nico Williams
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Henri Sivonen
- Re: [Json] Encoding detection (Was: Re: JSON: rem… John Cowan
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Tim Bray
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Alex Russell
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Paul Hoffman
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… John Cowan
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Carsten Bormann
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Tim Bray
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Phillip Hallam-Baker
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Joe Hildebrand (jhildebr)
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Carsten Bormann
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Tim Berners-Lee
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Carsten Bormann
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Allen Wirfs-Brock
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Noah Mendelsohn
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Phillip Hallam-Baker
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Ashok Malhotra
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Nico Williams
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Phillip Hallam-Baker
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Phillip Hallam-Baker
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… John Cowan
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Alex Russell
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Nico Williams
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… John Cowan