Re: [Json] Encoding detection (Was: Re: JSON: remove gap between Ecma-404 and IETF draft)

Allen Wirfs-Brock <allen@wirfs-brock.com> Tue, 26 November 2013 00:29 UTC

Return-Path: <allen@wirfs-brock.com>
X-Original-To: json@ietfa.amsl.com
Delivered-To: json@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 303D91AE0E2 for <json@ietfa.amsl.com>; Mon, 25 Nov 2013 16:29:31 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.999
X-Spam-Level:
X-Spam-Status: No, score=-0.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, J_CHICKENPOX_14=0.6, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id d2_v_Vwgshp0 for <json@ietfa.amsl.com>; Mon, 25 Nov 2013 16:29:29 -0800 (PST)
Received: from mho-02-ewr.mailhop.org (mho-02-ewr.mailhop.org [204.13.248.72]) by ietfa.amsl.com (Postfix) with ESMTP id C3C7A1AE07D for <json@ietf.org>; Mon, 25 Nov 2013 16:29:29 -0800 (PST)
Received: from 069-064-236-244.pdx.net ([69.64.236.244] helo=[192.168.0.25]) by mho-02-ewr.mailhop.org with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.72) (envelope-from <allen@wirfs-brock.com>) id 1Vl6Wn-000LjI-M8; Tue, 26 Nov 2013 00:29:22 +0000
X-Mail-Handler: Dyn Standard SMTP by Dyn
X-Originating-IP: 69.64.236.244
X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information)
X-MHO-User: U2FsdGVkX1/KGUy44EV+jumqrIdjeXkAEB+jF6GXsvU=
Mime-Version: 1.0 (Apple Message framework v1085)
Content-Type: multipart/alternative; boundary="Apple-Mail-131--903340370"
From: Allen Wirfs-Brock <allen@wirfs-brock.com>
In-Reply-To: <52931A67.7010309@it.aoyama.ac.jp>
Date: Mon, 25 Nov 2013 16:29:12 -0800
Message-Id: <0021FE1E-FA6D-4410-9CD8-FF3C2775D2A3@wirfs-brock.com>
References: <8413609C8A86497F856897AF2AA24960@codalogic> <CEAA3067.2D132%jhildebr@cisco.com> <CANXqsRJEtBoprQFrftz80ZigmBR_NHoEXK1sR4GyBtz5B2KC8Q@mail.gmail.com> <20131120223305.GB5476@mercury.ccil.org> <CANXqsRJmNmSRXssBnw3tGUt0veViENLoS=dp+gEr2RqvNAf4JQ@mail.gmail.com> <20131121165615.GA12138@mercury.ccil.org> <CANXqsRKrcR54TzSFng0ysyTV60-uZZ7QQ-G4xJOB0gO29C7-Ag@mail.gmail.com> <54E53D571E5E4589B2E9FA17DC816002@codalogic> <CAHBU6itwNwAy__Yy3n2bGMjuZnXAhK+qNbLq36piFy666m-Cng@mail.gmail.com> <EF7034DE-EA4D-43E4-9A5A-159E4A9DAB02@wirfs-brock.com> <52931A67.7010309@it.aoyama.ac.jp>
To: "Martin J. Dürst" <duerst@it.aoyama.ac.jp>
X-Mailer: Apple Mail (2.1085)
X-Mailman-Approved-At: Mon, 25 Nov 2013 17:40:08 -0800
Cc: John Cowan <cowan@mercury.ccil.org>, Pete Cordell <petejson@codalogic.com>, Paul Hoffman <paul.hoffman@vpnc.org>, JSON WG <json@ietf.org>, Henri Sivonen <hsivonen@hsivonen.fi>, Tim Bray <tbray@textuality.com>, "Joe Hildebrand (jhildebr)" <jhildebr@cisco.com>, "www-tag@w3.org" <www-tag@w3.org>, es-discuss <es-discuss@mozilla.org>
Subject: Re: [Json] Encoding detection (Was: Re: JSON: remove gap between Ecma-404 and IETF draft)
X-BeenThere: json@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "JavaScript Object Notation \(JSON\) WG mailing list" <json.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/json>, <mailto:json-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/json/>
List-Post: <mailto:json@ietf.org>
List-Help: <mailto:json-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/json>, <mailto:json-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 26 Nov 2013 00:29:31 -0000

On Nov 25, 2013, at 1:37 AM, Martin J. Dürst wrote:

> 
> 
> On 2013/11/23 1:54, Allen Wirfs-Brock wrote:
>> 
>>> 
>> 
>> You should say it that it is not an actual issue of the JSON format whose grammar clearly defines the handling of the 0xfeff code point.  Rather it is an upstream data interchange issue that should be dealt with in exactly the same way as with any other data interchange on a similar channel.  Say whatever you think is appropriate about BOMs in the transmission of data conforming to the "application/json" MIME type.  Just be clear that whatever you decide has nothing to do with the abstract, grammar-based interpretation of the actual JSON payload.
> 
> That works for ECMA-404. It does not work for the IETF draft, because it is extremely relevant for application/json, which is part of that draft.
> 
> Regards,    Martin.

It still seems pretty clear.  Anything feed as input into an actual parser for the JSON grammar as standardized by ECMA-404 must not contain any U+feff code points other than as part of JSON string values.  If the application/json wire format chooses to to use BOMs, then they must be removed before processing by a standard JSON parser.   From a JSON parser prospect, I think that's pretty much all you need to say. 

allen