Re: [Json] Call for Consensus: Proposed Text for "8.1 Character Encoding"

I seem to have killed this thread off again.  Sorry about that.

Any conclusions?

Cheers,

Pete.

On 18/04/2017 14:01, Pete Cordell wrote:
> On 18/04/2017 06:22, Martin J. Dürst wrote:
>> On 2017/04/18 05:47, Carsten Bormann wrote:
>>> On Apr 17, 2017, at 19:56, Nico Williams <nico@cryptonector.com> wrote:
>>>>
>>>>> Thinking about this more, putting an encoding detection algorithm
>>>>> as an
>>>>> appendix seems like a reasonable compromise to me.  To start, how
>>>>> about
>>>>> removing the detection text from Section 8.1 and have an appendix that
>>>>> starts with that text plus the table?
>>>>
>>>> Or we could even just assert that such an algorithm is possible, and
>>>> that implementors MAY implement one.
>>>
>>> Indeed.
>>>
>>> Broken record mode:
>>>
>>> — writing up the algorithm sounds like encouraging implementation.
>>>   We *don’t* want people to implement this!
>>>   (The whole interminable non-UTF-8 saga probably just was a nod from
>>> the RFC 4627 authors to the remnants of UTF-16 land, which mostly have
>>> died off since.  Why resurrect?)
>>>
>>> - there have been about 15 attempts to define this algorithm on the
>>> mailing list.
>>>   All were wrong.
>>>   An Internet Standard should contain tried and true material, not
>>> errata fodder.
>>>
>>> - an implementer is in a much better position to get this right than
>>> the standard, because they can write unit tests.
>>
>> I completely agree with Carsten. As far as I know, and as far as we have
>> been told on this list, if some JSON isn't in UTF-8, then it simply will
>> not interoperate.
>
> +1
>
> If we do do this, I think we could add some example test messages to
> helps with the development, e.g.:
>
>     {"Example":1}
>     {}
>     "Example"
>     ""
>     "U+0100" (where U+0100 is the UTF form of the character, not ASCII)
>     1
>
>> In my view, the only reason to still have a MAY for UTF-16/32 is that
>> this will avoid questions like: "I have a JSON parser in language FOO,
>> it can take a string or an input stream as an argument. In FOO, strings
>> are UTF-16, but the JSON RFC doesn't seem to allow this. What should I
>> do."
>
> IMO the answer to that is, "that's why it says 'JSON text _SHOULD_ be
> encoded in UTF-8'".
>
> I agree with John Cowen, that use of UTF-16/32 is purely for internal
> scenarios; not on the Internet.  As such, I believe the IETF is going
> beyond its remit to say you can use UTF-16/32 for your internal purposes
> that I know nothing about and care nothing about, but you're not allowed
> to use other encodings that maybe more natural for your system.
>
> Pete Cordell
> Codalogic Ltd
> C++ tools for C++ programmers, http://codalogic.com
> Read & write XML in C++, http://www.xml2cpp.com
>
> _______________________________________________
> json mailing list
> json@ietf.org
> https://www.ietf.org/mailman/listinfo/json