Re: [Json] Call for Consensus: Proposed Text for "8.1 Character Encoding"

Martin J. Dürst <duerst@it.aoyama.ac.jp> Tue, 18 April 2017 05:22 UTC

Return-Path: <duerst@it.aoyama.ac.jp>
X-Original-To: json@ietfa.amsl.com
Delivered-To: json@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 114F3126CC7 for <json@ietfa.amsl.com>; Mon, 17 Apr 2017 22:22:34 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.902
X-Spam-Level:
X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=itaoyama.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id I3zjtsc7rsTc for <json@ietfa.amsl.com>; Mon, 17 Apr 2017 22:22:31 -0700 (PDT)
Received: from JPN01-OS2-obe.outbound.protection.outlook.com (mail-os2jpn01on0116.outbound.protection.outlook.com [104.47.92.116]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0765F129415 for <json@ietf.org>; Mon, 17 Apr 2017 22:22:28 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=itaoyama.onmicrosoft.com; s=selector1-it-aoyama-ac-jp; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Z0HKLlucBDL9MpSkZFi3FRhGo6kkwX6fPIaHF2KXXw0=; b=AWs7fjSmbDWrpiITbuuzPG7Sn15vHF3THer3R6TwbOGDqsTaQs+LtKhOpZA/DlWqJdjFfAnGmSIGVUwVHYgd5qAOvw79/73fUwP4ccWa72z2P4HM72kpeFCLv/pe9/G1YjuAOKnhJZCHo/ehHEtJyaNiV++NtzV9+bDpXkOc4x8=
Authentication-Results: ietf.org; dkim=none (message not signed) header.d=none;ietf.org; dmarc=none action=none header.from=it.aoyama.ac.jp;
Received: from [133.2.210.64] (133.2.210.64) by OS2PR01MB0644.jpnprd01.prod.outlook.com (10.167.176.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1034.10; Tue, 18 Apr 2017 05:22:25 +0000
To: Carsten Bormann <cabo@tzi.org>, "Matthew A. Miller" <linuxwolf+ietf@outer-planes.net>
References: <e69d7c21-85cb-45f4-c0c2-34c624e63049@outer-planes.net> <14252631-AD76-4537-89BF-6368F4A8CDF4@att.com> <7e6af21f-16ea-a3bc-9c01-595ae8acebba@gmx.de> <05100401-88D4-4158-A3FF-3EF144D85449@att.com> <CAD2gp_T0bfpnsCA_t4BAMtEhr7p8JkZggjnY4F+m9-M2hWLfmw@mail.gmail.com> <1e94516c-9c82-8b0e-0d2d-7dbaa83b21bd@outer-planes.net> <40e3207f-e047-c898-1f0c-4422de1d597a@it.aoyama.ac.jp> <1b3ec14a-927a-8d46-e3d3-9807a9588437@outer-planes.net> <CAHBU6ivsq8+Z=MMkUH+=Q0uwc5NCtaJLYw5cp0Qg8eX2hQQ6sA@mail.gmail.com> <b74cb31b-8e04-17d0-548a-fc164ce07c05@outer-planes.net> <20170417175627.GK23461@localhost> <10B651F1-7FE0-484D-BD2E-FD146BC5FB04@tzi.org>
CC: "json@ietf.org" <json@ietf.org>
From: =?UTF-8?Q?Martin_J._D=c3=bcrst?= <duerst@it.aoyama.ac.jp>
Organization: Aoyama Gakuin University
Message-ID: <eabbccb0-8d15-d595-7cd0-37acc0621c57@it.aoyama.ac.jp>
Date: Tue, 18 Apr 2017 14:22:22 +0900
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <10B651F1-7FE0-484D-BD2E-FD146BC5FB04@tzi.org>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 8bit
X-Originating-IP: [133.2.210.64]
X-ClientProxiedBy: OS2PR01CA0040.jpnprd01.prod.outlook.com (10.164.161.150) To OS2PR01MB0644.jpnprd01.prod.outlook.com (10.167.176.142)
X-MS-Office365-Filtering-Correlation-Id: e2fa32ed-7329-401c-6c83-08d4861ae695
X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201703131423075)(201703031133081); SRVR:OS2PR01MB0644;
X-Microsoft-Exchange-Diagnostics: 1; OS2PR01MB0644; 3:pqlV3cYT9zCUvNN6/xF9QnsZL9iUtbDJLb3W2H0RGt7dDrft4w29Uup++P3dlNp255u/h5yRmy/DIvDY9lfE5SrOjEPVhXFWGp3vymunWUK4Jr2i1kpWHLeJK6gpXLb8nDZY/beMTRlHyEEeGknu4R2w5tkwd/AUPFu6dD/mmDibxyCro2iVrf/YQE3gy9TnuTYMf8RL+piIhuEUyFUBADir3Fh1y1CIrpq51yDipEgbIU+l4McCkUUn2qYeagf1IYDVQWitCfcB3TpcdRsDgMHp9BEATnkRTX5DuwvrEF+IwYd8IU4BNk/+gOVASybXOX2uRmjSldX5AYj7hYao2w==; 25:tqV3AJv65/WlDJSqHtcFkcFXAVCpWs19T4KsykNH3r+S1xc+VpAqkFrbIjWFed/1lBgeycIKoehdfVpURlI+POkhFoD7DA0iM4Axm+Q/13gCiY6QSL1ePlyaYtDPbR0Wc0fqa1OUxfee8d7psu48CZy16rsSy/HqiMTMb7C0vMbUpL9Kp/auApgl2gk1drvQ/7aeqKEAFSiNOEj0vvrIA+kW5tNGCzsCfqNSZgJ6sWcG+FkoMKNIkFmRsZUynk6bEY7GwgMjRaJ/yybHYJ59f5txS1ztLB1lh9Qz6vDvJ/NXfyYPhdSp5ZUtlLWI9tvhIajollxpoBcdlkWZJYl4f6c0MxpHI+pfxiEQByyZJsBGdEdn9WmdrctUiRZbDQSKiw/FRtg7bpZWqAa+8az1oye+DACuIUXVsf8IsceqFWx5/NPFWDsiXRjpTDo6EDfVr595EPqt4sBr2TVHJYQm3g==
X-Microsoft-Exchange-Diagnostics: 1; OS2PR01MB0644; 31:bA5R330JcHQBsg9dOiJZohVBAHst75WxR/XmDvt88RRNGADhyb6yIzrUm2D+N/DjRRA6sJJJeTJXgoovXUOg/J0clRD2zJ7BfQzCfaws93eW3J3IMvQmluH6+VVlr6gafITiWY3mHKLAbw5103T+N2wNWN3PjGK10eRg0eruRmUL6uBH3PQywRHTjgWslDXg0G7X8sOTWcLt5tVUwU25MXnyRfHS6oCZzCl1M0bDocopcNQoH4jWKNcyeRzhrtoNJjTCC5vMNxVDckyS+FqQHg==
X-Microsoft-Antispam-PRVS: <OS2PR01MB064463B5DE7965B353240C7BCA190@OS2PR01MB0644.jpnprd01.prod.outlook.com>
X-Exchange-Antispam-Report-Test: UriScan:(100405760836317);
X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(2401047)(5005006)(8121501046)(10201501046)(3002001)(93006095)(93001095)(6041248)(201703131423075)(201702281529075)(201702281528075)(201703061421075)(20161123560025)(20161123564025)(20161123562025)(20161123555025)(6072148); SRVR:OS2PR01MB0644; BCL:0; PCL:0; RULEID:; SRVR:OS2PR01MB0644;
X-Microsoft-Exchange-Diagnostics: 1; OS2PR01MB0644; 4:YIidxQjfA3NMp2ReTjoOp9T0oDZW0Y50vzXcsQwUG1jMhVzzMCIZIZ251kxSwTkYCeYbh9Dtn3dqoNsqti7gdjOf5U/xreWuympur4JGRMHW/FMFKpsQn6g88czSwT8mSYv49jp9M/eDS6fSP1tU2KaP1c/q+kGVTtW/74/ZznzdDDcsHxxX4odsJG7fXfe9VAJRzQQqbn4Zuw07RnodCAPUiMiO09MxBuQVuVROALHynitjuDhoeMrTnY/gBGy+UBd6HVEr6Y4Aq1NlC25sWUzXc8vkc44E4vUk8qKpcgIGcca49hcIcuAaTSc4wqV+QT9skXPgHs1cHsEricrhYYR/A/3oSHknGu+AZJkIGoDxHl4AoBaOuDYtlwkLuprPOZXXqn5Ue2I1FQ2nj/gBJsBGEdkrRRcMUWFTvC+Eca1rY9DBsP4/j4tLb0NWI5NsSgceqGMOjJQwspvNw0sr5DyQwW8sJ0EwQUUCSuIdEvoMEjzRH9QXSJ5jXKJBPV6K4kO/mFLqKziCUPLDlHndxHX565Y+iKfRQb0owlMqm2pxREPMUEyW0KHxrbhrFwzvrBqQBAKiSAuVVVYl8bQ4zCcIYaAR3HL696WkUrevvUTW3FLwlYRGf/zpzfrZ57pvWCGezbziv8+XiF2LEsiLcjXcPMp+qChpcsD6TnZxwFJfAsN5T6rRgOE9dZ40tA+R35m/T8u3nVMzxwtgk3Vz6+2VsPakHqFbFLfjI3p0hX481OTEU6loHN3TqSgMRXeIWv8IKcrHmkr2/BGIjM7MTg==
X-Forefront-PRVS: 028166BF91
X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(4630300001)(6049001)(6009001)(39400400002)(39410400002)(39450400003)(39830400002)(24454002)(42882006)(81166006)(25786009)(65956001)(65826007)(76176999)(54356999)(66066001)(2950100002)(53546009)(47776003)(7736002)(4326008)(6246003)(189998001)(305945005)(23676002)(8676002)(50466002)(83506001)(53936002)(33646002)(42186005)(3846002)(86362001)(2906002)(74482002)(31696002)(38730400002)(6666003)(31686004)(90366009)(4001350100001)(5660300001)(229853002)(50986999)(6486002)(93886004)(2870700001)(6116002)(3940600001); DIR:OUT; SFP:1102; SCL:1; SRVR:OS2PR01MB0644; H:[133.2.210.64]; FPR:; SPF:None; MLV:sfv; LANG:en;
X-Microsoft-Exchange-Diagnostics: =?utf-8?B?MTtPUzJQUjAxTUIwNjQ0OzIzOkxjS2FVb2ZPNUZ6TytxZWVpbFJ2L3dTRTl6?= =?utf-8?B?aURlbEhPTS95OEZoSkdlZFMwUUpyaG1uVi84SXB3WEp4enVhREVJQW1NQ3BC?= =?utf-8?B?dFcrSDlHeVdaS25UUDljZHEvK0x0S0tUTHNNcU0xN3dycG5rTnVPTytrcUM3?= =?utf-8?B?SldOSFdGaVY1RlRObEZQRTZXMXhHeGhYTTdSWVdWSEhhajhOUzR1bTJlRlp4?= =?utf-8?B?eUhZbjgvRzg2Q1BuUmZCUmQyNjcvdWxnOUNCK0N4d3MxVHk2MGdFUTRJdGdC?= =?utf-8?B?c3U0Z1NCYUdwYzZXMFVjeWM2elUwOGY2OXRkZ3FFZEM5czFTbDNpL0xDZS93?= =?utf-8?B?cDN3MjAyWEZtdHU5S2tzTzVVUnVtTDZ3TUJJZXcvY2pKUkRYa3FteGU0V0lo?= =?utf-8?B?ZklNMkRZQk9GVlRUa3FoSDFtODJSSGNlMnp1VTdjdTNacUYrSi8zN244bUxp?= =?utf-8?B?Sm1BZ3laQWhhSnRWKzJzdDZOdkVOREJjY1pDcVNFdld3SXZlNzBnbVByaTdO?= =?utf-8?B?MTVJSlEvRWphVitZbzdZKzlGb1M3aUhGSE8xaytRcDkzaDRKVk11ZzBNNXFM?= =?utf-8?B?VnZSNlZMT0VLRGJnSTNwbTY4bVlxWDBUTkNBeWdwU0I2alhzMDRQd2Z6a3lK?= =?utf-8?B?VHNxSVVHcVMyUlFHc3pQZlhhanprQ1dMaDVGVnVKdnhNVUpQcVZ1SHJFakN4?= =?utf-8?B?dVVYMlJ0c3E2TXArbk5LaHA0YmV6ZVNKajEzM2dlZFh1UUFNU0R1VWttS0ZR?= =?utf-8?B?UUw1VUxxQlQ4WWVDQXF2MVoyK010TEUrdjUzWmc1TjNpbUxRd0kxMmYxWXVs?= =?utf-8?B?dGpRbkNKYUtQdi92Mkl2VUp2M2ltODRqRURvbGk4aGVYTVMzakthaFlXZGlm?= =?utf-8?B?U0dXcEN4cmJtdzZjRjdCQUtBUGZDU2k4VlpWY1piM21PSFUvcStaMXpvcEx4?= =?utf-8?B?NjlhcWkyeXRZdzNiUlcrYm0vZjRGWVdDQmxGN1hkVE15dzZiOUxUaGRjYSsx?= =?utf-8?B?bHRYenFsVmlWZUZUWTJ6WnIvbng1Sm9heDE0eTFHK2RaQktycFI0bVErSGVw?= =?utf-8?B?dE1wSXNyS2hNUUUwcm5lZ1BtQ3VYUlhXWWhMSkhudk9BRjhRZTlPeVZVRkVU?= =?utf-8?B?RGpUL1B0V2ZDTUZ6Qm9UTVJFbEJ3dG1EL3c3ZmQrRVRGcXlrdHh1bE4zdjJQ?= =?utf-8?B?UFpwbGwzem1EOE4yaHg3TWZYaFpMbzdqTVNKNFR3RUd4QmVrSDBiT2xzT3Nj?= =?utf-8?B?b0JqTmtuWnJTdjNLU255ak9tTkNjeGhkVkI3UkN6eE80QzRDbGl2SkdDeUNK?= =?utf-8?B?aWQ1NVFXR2J6NUlLaXUrVDczL2M1NDQ1RWVYMWFWOGRNV0h0Q0xjcWZqVmQ3?= =?utf-8?B?R1hpd0wxVlIrOVFtdS9xbUFWTjRhMHpBblliZXltWFZMcWhxTjVCNDhkWjc3?= =?utf-8?B?ZjRERnErYU9UVkhoYlZYZUVMU2Z0L2IxajRIVWdqWkJlbjMzM0U0UTM0c1FI?= =?utf-8?B?VTZoNGtXT3NpeGxrZFNQRjRXL0x2TVU4VHZaRDF4Mk5XZGs0bFBrRUM2b2RN?= =?utf-8?B?UXlRNVhweG1mZ0NaT25NV2FqSXM1VkhzMjkzblV6TFA4eHJPb0g5V0t5NWEv?= =?utf-8?B?bUFhWmZsQS8wcGNOOURPZVNrRXNiZm14ZjVPR0tVR1llOVRzZDkra3F3PT0=?=
X-Microsoft-Exchange-Diagnostics: 1; OS2PR01MB0644; 6:zzb1DwAOl8PQnNkagRUoCXaaGOp0m7cpvUZEyLKMwk4SRechfHYAkZH3Rp4leZ0AOJ6wYTZo5g6A8ChL75mA2ww8WFvZIdS6PMHUD50qS9jPT7yy5L3qmAuqneRxXv/46H7jzj+woXSixlwBxlEO6c53pyyCrnaahcNXRIZEA0fazsoiyNuerjjjno1CG4fyyTFmPALVrXUQqNM5RWsXSSQsfO3eTcUFvs+BHsiEekolbxugWkCG7KFO7HKczumdqcg4hhtlvroq0kOhswBB8toaepDASt4aNxQ0dpkp5Kp6MMMGwqQPx1+DvaAr0DYnAGx6KrGxrTjUDlX9rV2+p2CH2HcdWQT7hO5wbLQpcT8MyhhdWfbEigdjDKrjOo2qTTJ8hi+ra12rXqCZDpt41sC0zlG6zDklGRJ0CX2/SY3EL+YXJzClCL0DwbBmy+hHNRdNQN1BEx2V/oF5wQFNnQ==; 5:/cwjYSt1CLB4xCZ+Xb6SW70/IMAPjfA2BKO4NO6RiDNUebHmmm/W/LwVjKC+CBD0mazFozzt0kGwqWFk+fwukgpHN9Z7qs9s7L43aVU9iOhgC2/Kks4xtx8Ge7Gpsf1akEdym1Wd+UFhvR0HHBTGNA==; 24:i3kMXtk0YSiIGtrftbhtMdQmNzTEwkCM5AhmClryd5NBs+s/f81embzCezpItR/dHgOsP9z4I0P4Fum78tveLwOJTDhVH+OnIjlUPvXgGFs=
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-Microsoft-Exchange-Diagnostics: 1; OS2PR01MB0644; 7:UagRzc7oPxAof3KbDZ74Q6SJ8FRpQG/MKU2BvL+0o762opOK9cAqj1/PCO7EXSRmWBbJtb11tsoBT84wvqJEA0EB9Sf299fisunx4k6yet7du3loqlz9hEpl3g/GNheC6ygBhVUuaGHzs1f8dzKwoCK2vew32WCzsbbTxeOm+buUPpcbNXvDRoQXoiFZYxWTYTvjXlYm6dHqCgIAH5ywC2/0dF9goaF08OY1ruqE/3mNMujK0pESWltCz82dR9DFJZ41XmrIn0Q4/+VODK2OndiAmNcfI+ZtdnScxYlVKqhTCKQC+LnnrKV5jeKjVTfBZst9e8WkfK0Zzdq2nUr9Ww==
X-OriginatorOrg: it.aoyama.ac.jp
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Apr 2017 05:22:25.6782 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS2PR01MB0644
Archived-At: <https://mailarchive.ietf.org/arch/msg/json/lf_inlrVpvAGxGmz-9x6PtwEZho>
Subject: Re: [Json] Call for Consensus: Proposed Text for "8.1 Character Encoding"
X-BeenThere: json@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: "JavaScript Object Notation \(JSON\) WG mailing list" <json.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/json>, <mailto:json-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/json/>
List-Post: <mailto:json@ietf.org>
List-Help: <mailto:json-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/json>, <mailto:json-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 18 Apr 2017 05:22:34 -0000

On 2017/04/18 05:47, Carsten Bormann wrote:
> On Apr 17, 2017, at 19:56, Nico Williams <nico@cryptonector.com> wrote:
>>
>>> Thinking about this more, putting an encoding detection algorithm as an
>>> appendix seems like a reasonable compromise to me.  To start, how about
>>> removing the detection text from Section 8.1 and have an appendix that
>>> starts with that text plus the table?
>>
>> Or we could even just assert that such an algorithm is possible, and
>> that implementors MAY implement one.
>
> Indeed.
>
> Broken record mode:
>
> — writing up the algorithm sounds like encouraging implementation.
>   We *don’t* want people to implement this!
>   (The whole interminable non-UTF-8 saga probably just was a nod from the RFC 4627 authors to the remnants of UTF-16 land, which mostly have died off since.  Why resurrect?)
>
> - there have been about 15 attempts to define this algorithm on the mailing list.
>   All were wrong.
>   An Internet Standard should contain tried and true material, not errata fodder.
>
> - an implementer is in a much better position to get this right than the standard, because they can write unit tests.

I completely agree with Carsten. As far as I know, and as far as we have 
been told on this list, if some JSON isn't in UTF-8, then it simply will 
not interoperate.

In my view, the only reason to still have a MAY for UTF-16/32 is that 
this will avoid questions like: "I have a JSON parser in language FOO, 
it can take a string or an input stream as an argument. In FOO, strings 
are UTF-16, but the JSON RFC doesn't seem to allow this. What should I do."

This may look far-fetched to some of you (and I agree it is), but I 
remember having dealt with such questions when I was at W3C.

Regards,   Martin.