[Json] In "praise" of UTF-16

[Anders Rundgren <[e-mail]>]

Hi JSON experts,
Pardon the subject line, I'm by no means an UTF-16 aficionado.

That UTF-16 has been deprecated by the industry at large for EXTERNAL representation of textual data is completely understandable.

However, an I-D dealing with canonical JSON serialization is currently in the IETF ISE queue got criticism for using UTF-16 encoding INTERNALLY for sorting properties/keys.
I don't see why since the only purpose of the sorting is creating a defined order.   That sorting on UTF-8 or UTF-32 would give another result is true but for the stated purpose that is of no importance.

In addition, JSON itself also depends on UTF-16 encoding for \uhhhh constants and AFAIK nobody have complained about that.
Example: A smiley Emoji has the Unicode value U+1F600 but would in a JSON escape sequence be represented as \ud83d\ude00

The reason for preferring UTF-16 in this particular case is simply because JavaScript, Windows and Java use UTF-16 as internal representation.  That's obviously a slight platform bias but the my Go and Python implementations show that the UTF-16 requirement in practice is a no-issue.

According to the Unicode standard UTF-16 belongs to the set of supported fully interchangeable encodings.

WDYT?

thanx,
Anders
https://tools.ietf.org/html/draft-rundgren-json-canonicalization-scheme-06