Re: [Json] Encoding detection
"Pete Cordell" <petejson@codalogic.com> Mon, 18 November 2013 10:05 UTC
Return-Path: <petejson@codalogic.com>
X-Original-To: json@ietfa.amsl.com
Delivered-To: json@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0206711E8152 for <json@ietfa.amsl.com>; Mon, 18 Nov 2013 02:05:54 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 3.161
X-Spam-Level: ***
X-Spam-Status: No, score=3.161 tagged_above=-999 required=5 tests=[AWL=0.075, BAYES_50=0.001, FH_HOST_EQ_D_D_D_D=0.765, HELO_MISMATCH_COM=0.553, RDNS_DYNAMIC=0.1, SARE_HEAD_XUNSENT=1.666, STOX_REPLY_TYPE=0.001]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sPwp2vUGTZHn for <json@ietfa.amsl.com>; Mon, 18 Nov 2013 02:05:49 -0800 (PST)
Received: from ppsa-online.com (lvps217-199-162-192.vps.webfusion.co.uk [217.199.162.192]) by ietfa.amsl.com (Postfix) with ESMTP id 9453911E8106 for <json@ietf.org>; Mon, 18 Nov 2013 02:05:48 -0800 (PST)
Received: (qmail 22826 invoked from network); 18 Nov 2013 10:05:29 +0000
Received: from host81-129-187-193.range81-129.btcentralplus.com (HELO codalogic) (81.129.187.193) by lvps217-199-162-217.vps.webfusion.co.uk with ESMTPSA (RC4-MD5 encrypted, authenticated); 18 Nov 2013 10:05:29 +0000
Message-ID: <5756A4C4ECFF4C8DA655501B28A1B093@codalogic>
From: Pete Cordell <petejson@codalogic.com>
To: Paul Hoffman <paul.hoffman@vpnc.org>, Bjoern Hoehrmann <derhoermi@gmx.net>
References: <CEAA3067.2D132%jhildebr@cisco.com> <f5bbo1mzyvw.fsf@troutbeck.inf.ed.ac.uk> <75E35201E30C4EA4B143B520CB6DF273@codalogic> <13A5A836-A670-4412-82F8-B7BBE84A2FC8@vpnc.org> <3r7i899fler32pdhu1p2nhof0ohl9g8ufp@hive.bjoern.hoehrmann.de> <4364D345-53AD-457A-8ABC-2A559A2655B7@vpnc.org>
X-Unsent: 1
Date: Mon, 18 Nov 2013 10:02:15 -0000
MIME-Version: 1.0
x-vipre-scanned: 002A78AE005BB6002A79FB
Content-Type: text/plain; format="flowed"; charset="iso-8859-1"; reply-type="original"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2900.5931
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157
Cc: JSON WG <json@ietf.org>
Subject: Re: [Json] Encoding detection
X-BeenThere: json@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: "JavaScript Object Notation \(JSON\) WG mailing list" <json.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/json>, <mailto:json-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/json>
List-Post: <mailto:json@ietf.org>
List-Help: <mailto:json-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/json>, <mailto:json-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Nov 2013 10:05:54 -0000
----- Original Message ----- From: "Paul Hoffman" <paul.hoffman@vpnc.org> To: "Bjoern Hoehrmann" <derhoermi@gmx.net> Cc: "Pete Cordell" <petejson@codalogic.com>; "JSON WG" <json@ietf.org> Sent: Sunday, November 17, 2013 8:19 PM Subject: Re: [Json] Encoding detection > On Nov 17, 2013, at 12:00 PM, Bjoern Hoehrmann <derhoermi@gmx.net> wrote: > >> * Paul Hoffman wrote: >>> On Nov 17, 2013, at 2:02 AM, Pete Cordell <petejson@codalogic.com> >>> wrote: >>> >>>> While I'm here, Joe mentioned "implementable". From an implementer's >>>> perspective the table I presented earlier might be better presented in >>>> the >>>> following order: >>>> >>>> 00 00 -- -- UTF-32BE >>>> 00 xx -- -- UTF-16BE >>>> xx xx -- -- UTF-8 >>>> xx 00 xx -- UTF-16LE >>>> xx 00 00 xx UTF-16LE >>>> xx 00 00 00 UTF-32LE >>>> >>>> Or is that taking the fun out of it for the implementer?! >>> >>> I'm not sure that is the right order. Wouldn't that make UTF-16LE be >>> mistaken for UTF-8? It seems to me that the UTF-8 rule has to be last >>> because it is more general than any of the ones above it. >> >> No, `xx` stands for a non-zero byte. > > Sorry, I thought "xx" meant "unknown". Then I think Pete's formulation is > correct. Although my note about quick detection of BOMs (i.e.: I think BOM presence can be surmised by only looking at the first byte. If it's greater than 0x80 - specifically if it's 0xef, 0xfe, 0xff - then it's an error if you don't go on to find a BOM.) missed the case for 00 00 FE FF UTF-32 (BE) w. BOM. Argh!!! Pete Cordell Codalogic Ltd C++ tools for C++ programmers, http://codalogic.com Read & write XML in C++, http://www.xml2cpp.com
- Re: [Json] JSON: remove gap between Ecma-404 and … Paul Hoffman
- Re: [Json] JSON: remove gap between Ecma-404 and … Joe Hildebrand (jhildebr)
- Re: [Json] JSON: remove gap between Ecma-404 and … Carsten Bormann
- Re: [Json] JSON: remove gap between Ecma-404 and … Joe Hildebrand (jhildebr)
- Re: [Json] JSON: remove gap between Ecma-404 and … Tony Hansen
- Re: [Json] JSON: remove gap between Ecma-404 and … Paul Hoffman
- Re: [Json] JSON: remove gap between Ecma-404 and … Allen Wirfs-Brock
- Re: [Json] JSON: remove gap between Ecma-404 and … Carsten Bormann
- Re: [Json] JSON: remove gap between Ecma-404 and … Joe Hildebrand
- Re: [Json] JSON: remove gap between Ecma-404 and … Joe Hildebrand (jhildebr)
- Re: [Json] JSON: remove gap between Ecma-404 and … Paul Hoffman
- Re: [Json] JSON: remove gap between Ecma-404 and … John Cowan
- Re: [Json] JSON: remove gap between Ecma-404 and … Joe Hildebrand (jhildebr)
- Re: [Json] JSON: remove gap between Ecma-404 and … Mark Davis ☕
- Re: [Json] JSON: remove gap between Ecma-404 and … John Cowan
- Re: [Json] JSON: remove gap between Ecma-404 and … Allen Wirfs-Brock
- Re: [Json] JSON: remove gap between Ecma-404 and … Paul Hoffman
- Re: [Json] Encoding detection John Cowan
- Re: [Json] JSON: remove gap between Ecma-404 and … Martin J. Dürst
- [Json] Encoding detection (Was: Re: JSON: remove … Pete Cordell
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Joe Hildebrand (jhildebr)
- Re: [Json] JSON: remove gap between Ecma-404 and … John Cowan
- Re: [Json] JSON: remove gap between Ecma-404 and … Henry S. Thompson
- Re: [Json] JSON: remove gap between Ecma-404 and … Allen Wirfs-Brock
- Re: [Json] JSON: remove gap between Ecma-404 and … Allen Wirfs-Brock
- Re: [Json] Encoding detection Henry S. Thompson
- Re: [Json] Encoding detection John Cowan
- Re: [Json] Encoding detection David Booth
- Re: [Json] Encoding detection Martin J. Dürst
- Re: [Json] Encoding detection Henry S. Thompson
- Re: [Json] Encoding detection Pete Cordell
- Re: [Json] Encoding detection Paul Hoffman
- Re: [Json] Encoding detection Bjoern Hoehrmann
- Re: [Json] Encoding detection Paul Hoffman
- [Json] BOMs (Was: Re: JSON: remove gap between Ec… Pete Cordell
- Re: [Json] Encoding detection Pete Cordell
- Re: [Json] BOMs Henry S. Thompson
- Re: [Json] BOMs Martin J. Dürst
- Re: [Json] BOMs Bjoern Hoehrmann
- Re: [Json] BOMs Henry S. Thompson
- Re: [Json] BOMs Pete Cordell
- Re: [Json] BOMs Bjoern Hoehrmann
- Re: [Json] BOMs (Was: Re: JSON: remove gap betwee… Tim Bray
- Re: [Json] BOMs (Was: Re: JSON: remove gap betwee… Pete Cordell
- Re: [Json] BOMs John Cowan
- Re: [Json] BOMs (Was: Re: JSON: remove gap betwee… Julian Reschke
- Re: [Json] BOMs (Was: Re: JSON: remove gap betwee… John Cowan
- Re: [Json] BOMs (Was: Re: JSON: remove gap betwee… Pete Cordell
- Re: [Json] BOMs (Was: Re: JSON: remove gap betwee… Pete Cordell
- Re: [Json] BOMs (Was: Re: JSON: remove gap betwee… John Cowan
- Re: [Json] BOMs Tatu Saloranta
- Re: [Json] BOMs Phillip Hallam-Baker
- Re: [Json] BOMs Martin J. Dürst
- Re: [Json] BOMs Martin J. Dürst
- Re: [Json] BOMs Pete Cordell
- Re: [Json] BOMs Bjoern Hoehrmann
- Re: [Json] BOMs t.p.
- Re: [Json] BOMs Allen Wirfs-Brock
- Re: [Json] BOMs Tatu Saloranta
- Re: [Json] BOMs Henry S. Thompson
- Re: [Json] BOMs Joe Hildebrand (jhildebr)
- Re: [Json] BOMs Chris Lilley
- Re: [Json] BOMs Pete Cordell
- Re: [Json] BOMs Henry S. Thompson
- Re: [Json] BOMs Martin J. Dürst
- Re: [Json] BOMs Henry S. Thompson
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Henri Sivonen
- Re: [Json] Encoding detection (Was: Re: JSON: rem… John Cowan
- Re: [Json] BOMs Allen Wirfs-Brock
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Henri Sivonen
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Anne van Kesteren
- Re: [Json] BOMs Henri Sivonen
- Re: [Json] Encoding detection (Was: Re: JSON: rem… John Cowan
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Joe Hildebrand (jhildebr)
- Re: [Json] BOMs Allen Wirfs-Brock
- Re: [Json] BOMs Bjoern Hoehrmann
- Re: [Json] BOMs Mathias Bynens
- Re: [Json] BOMs John Cowan
- Re: [Json] BOMs Bjoern Hoehrmann
- Re: [Json] BOMs Nico Williams
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Henri Sivonen
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Pete Cordell
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Julian Reschke
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Robin Berjon
- Re: [Json] Encoding detection (Was: Re: JSON: rem… John Cowan
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Tim Bray
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Allen Wirfs-Brock
- [Json] Consensus on JSON-text (WAS: JSON: remove … Matt Miller (mamille2)
- Re: [Json] BOMs Matt Miller (mamille2)
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Matt Miller (mamille2)
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Julian Reschke
- Re: [Json] BOMs Bjoern Hoehrmann
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Bjoern Hoehrmann
- Re: [Json] Encoding detection (Was: Re: JSON: rem… John Cowan
- Re: [Json] Encoding detection (Was: Re: JSON: rem… John Cowan
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Matt Miller (mamille2)
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Pete Cordell
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Pete Cordell
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Martin J. Dürst
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Allen Wirfs-Brock
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Bjoern Hoehrmann
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Henri Sivonen
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Nico Williams
- Re: [Json] Encoding detection (Was: Re: JSON: rem… John Cowan
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Jacob Davies
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Nico Williams
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Julian Reschke
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Nico Williams
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Carsten Bormann
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Nico Williams
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Carsten Bormann
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Nico Williams
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Bjoern Hoehrmann
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Nico Williams
- Re: [Json] Encoding detection (Was: Re: JSON: rem… Henri Sivonen
- Re: [Json] Encoding detection (Was: Re: JSON: rem… John Cowan
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Tim Bray
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Alex Russell
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Paul Hoffman
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… John Cowan
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Carsten Bormann
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Tim Bray
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Phillip Hallam-Baker
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Joe Hildebrand (jhildebr)
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Carsten Bormann
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Tim Berners-Lee
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Carsten Bormann
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Allen Wirfs-Brock
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Noah Mendelsohn
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Phillip Hallam-Baker
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Ashok Malhotra
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Nico Williams
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Phillip Hallam-Baker
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Phillip Hallam-Baker
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… John Cowan
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Alex Russell
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… Nico Williams
- Re: [Json] Consensus on JSON-text (WAS: JSON: rem… John Cowan