Re: [Json] In "praise" of UTF-16

[Nico Williams <[e-mail]>]

On Mon, Sep 02, 2019 at 10:59:59PM +0200, Carsten Bormann wrote:
> Defining a deterministic encoding scheme (“canonicalization”) for JSON
> in 2019 that needs a detour through UTF-16-land looks like a cruel
> joke.  [...]

Is that what this is about?!

Oh no.

Every JSON implementor needs to deal with UTF-8, but not every JSON
implementor needs to deal with UTF-16.

Speaking as a maintainer and developer of one popular JSON
implementation that only supports UTF-8, I'm loather to have to add
support for transliterating to UTF-16 code for canonicalization
purposes.

I understand that for many implementors the use of UTF-16 for this would
be far, _far_ more convenient than UTF-8, and that were I one of them,
I'd probably be on the other side.  For this reason I think it would be
fair for either camp to end up on the rough side of consensus, and if it
be my side so be it.

But first we should find what the consensus is.  Has there been a
consensus call on this?

Whatever we do here will make some unhappy.  Like endianness, we can
have some notion of purity (network byte order) and yet often prefer
that which is most widely implemented in hardware (little endian byte
order).  In Internet standards we _tend_ to prefer purity.

> [...].  If you only ever care about Java and its contemporaries, it may
> actually seem practical to you.  Given that most map keys will be
> ASCII anyway, people will certainly find shortcuts (also known as
> sleeping interoperability problems) around the issue.  A protocol
> designer would not touch this with a 16-foot pole.

+1

Nico
--