Re: [Json] BOMs

Bjoern Hoehrmann <derhoermi@gmx.net> Tue, 19 November 2013 12:31 UTC

Return-Path: <derhoermi@gmx.net>
X-Original-To: json@ietfa.amsl.com
Delivered-To: json@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 636491ADF67 for <json@ietfa.amsl.com>; Tue, 19 Nov 2013 04:31:13 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.425
X-Spam-Level:
X-Spam-Status: No, score=-2.425 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RP_MATCHES_RCVD=-0.525, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LOi4pq6ypf1R for <json@ietfa.amsl.com>; Tue, 19 Nov 2013 04:31:12 -0800 (PST)
Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) by ietfa.amsl.com (Postfix) with ESMTP id 2923C1ADF76 for <json@ietf.org>; Tue, 19 Nov 2013 04:31:11 -0800 (PST)
Received: from netb.Speedport_W_700V ([91.35.62.159]) by mail.gmx.com (mrgmx103) with ESMTPA (Nemesis) id 0LiTrM-1V8aIF1sa4-00cica for <json@ietf.org>; Tue, 19 Nov 2013 13:31:04 +0100
From: Bjoern Hoehrmann <derhoermi@gmx.net>
To: Tatu Saloranta <tsaloranta@gmail.com>
Date: Tue, 19 Nov 2013 13:31:02 +0100
Message-ID: <8ulm89lpscd531fb7rbukksn6lar3jjf3g@hive.bjoern.hoehrmann.de>
References: <AA45B3C6-1DC5-4B1E-8045-C9FE76022584@vpnc.org> <CEA92854.2CC53%jhildebr@cisco.com> <20131113224737.GI31823@mercury.ccil.org> <f5bob5n71y7.fsf@troutbeck.inf.ed.ac.uk> <5284B095.4070004@it.aoyama.ac.jp> <C37B2FE59C164DBCA982AC81A56A09AA@codalogic> <f5bk3g6ufqy.fsf@troutbeck.inf.ed.ac.uk> <5289F974.9020709@it.aoyama.ac.jp> <2tuj89hcus182t4f4rqqgi1dpabt11qak7@hive.bjoern.hoehrmann.de> <CAGrxA27HM6P=C4GvGgzpBNVOrFg0Vgfn02e5n3-irQUCfZZJPg@mail.gmail.com>
In-Reply-To: <CAGrxA27HM6P=C4GvGgzpBNVOrFg0Vgfn02e5n3-irQUCfZZJPg@mail.gmail.com>
X-Mailer: Forte Agent 3.3/32.846
MIME-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-Provags-ID: V03:K0:f99n/X7eT6JNlB4q+kprT0nOSGxs5gfTy7xzHNrPyLD4AjbMbZJ /KoDLq8uRBHaFj84XVfmGKeXT755q5WYUq2mp7pYi8r476GRnqIS3rcAPa45RTb/x9LtLyQ BsXA1Rc6mwQpGmlUh0BDQnGTQlt0tE3zD48+OO4YPkjNRNX8TW5BOwKIgj+QcTBMxr8affV VPQOdKrioSG4ES7XZUaFg==
Cc: Anne van Kesteren <annevk@annevk.nl>, es-discuss <es-discuss@mozilla.org>, IETF Discussion <ietf@ietf.org>, "www-tag@w3.org" <www-tag@w3.org>, JSON WG <json@ietf.org>
Subject: Re: [Json] BOMs
X-BeenThere: json@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: "JavaScript Object Notation \(JSON\) WG mailing list" <json.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/json>, <mailto:json-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/json/>
List-Post: <mailto:json@ietf.org>
List-Help: <mailto:json-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/json>, <mailto:json-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 19 Nov 2013 12:31:13 -0000

* Tatu Saloranta wrote:
>Dominant Java implementations support UTF-16 with BOM; either directly or
>through Java's Reader implementations that handle BOMs.
>String concatenation case seems irrelevant, since BOMs are not included in
>in-memory representation anyway, as opposed to byte stream serialization.

HTTP implementations cannot correctly determine whether an entity body
is text in a single character encoding and if so what that encoding is,
accordingly the dominant API deals in byte[] arrays, not text Strings;
furthermore, many programming languages default to byte[] arrays for
string literals. That often combines into forms of

  byte[] json = sprintf('{"x": %s, "y": %s}', GET(...), GET(...));

which works fine if all three byte[] arrays are UTF-8 encoded and use
no Unicode signature, which is the case 99% of the time.
-- 
Björn Höhrmann · mailto:bjoern@hoehrmann.de · http://bjoern.hoehrmann.de
Am Badedeich 7 · Telefon: +49(0)160/4415681 · http://www.bjoernsworld.de
25899 Dagebüll · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/