Re: [Json] Call for Consensus: Proposed Text for "8.1 Character Encoding"

[Martin J. Dürst <[e-mail]>]

On 2017/04/18 05:47, Carsten Bormann wrote:
> On Apr 17, 2017, at 19:56, Nico Williams <[e-mail]> wrote:
>>
>>> Thinking about this more, putting an encoding detection algorithm as an
>>> appendix seems like a reasonable compromise to me.  To start, how about
>>> removing the detection text from Section 8.1 and have an appendix that
>>> starts with that text plus the table?
>>
>> Or we could even just assert that such an algorithm is possible, and
>> that implementors MAY implement one.
>
> Indeed.
>
> Broken record mode:
>
> — writing up the algorithm sounds like encouraging implementation.
>   We *don’t* want people to implement this!
>   (The whole interminable non-UTF-8 saga probably just was a nod from the RFC 4627 authors to the remnants of UTF-16 land, which mostly have died off since.  Why resurrect?)
>
> - there have been about 15 attempts to define this algorithm on the mailing list.
>   All were wrong.
>   An Internet Standard should contain tried and true material, not errata fodder.
>
> - an implementer is in a much better position to get this right than the standard, because they can write unit tests.

I completely agree with Carsten. As far as I know, and as far as we have 
been told on this list, if some JSON isn't in UTF-8, then it simply will 
not interoperate.

In my view, the only reason to still have a MAY for UTF-16/32 is that 
this will avoid questions like: "I have a JSON parser in language FOO, 
it can take a string or an input stream as an argument. In FOO, strings 
are UTF-16, but the JSON RFC doesn't seem to allow this. What should I do."

This may look far-fetched to some of you (and I agree it is), but I 
remember having dealt with such questions when I was at W3C.

Regards,   Martin.