Re: [I18ndir] [art] Fwd: New Version Notification for draft-bray-unichars-06.txt

[Rob Sayre <[e-mail]>]

On Sun, Oct 8, 2023 at 8:45 PM Martin J. Dürst <[e-mail]>
wrote:

> On 2023-09-26 10:59, Rob Sayre wrote:
>
> > The aim here is to point out that the Web (and Java, and Windows) can
> > accommodate ill-formed Unicode. Is it possible to transmit any Windows
> path
> > name via conforming JSON in UTF-8? Yes. Is it a good idea to naively
> design
> > that into a protocol? No. But you might have to accept these things to be
> > sufficiently compatible with the Web.
>
> "the Web" is a big place. Among else, UTF-8 is "The Encoding" (see
> https://encoding.spec.whatwg.org/#the-encoding), and in the definition
> at https://encoding.spec.whatwg.org/#utf-8-decoder, surrogates (whether
> alone or not) produce an error.
>
> So when you speak about the above toxic stuff, please specifically say
> "in JSON" or "in JavaScript".
>

True in both, and the Web in general, because you can put the "toxic" stuff
in a script element in a web page that is well-formed UTF-8. There aren't
very many pages that don't contain at least a little bit of JS.

Maybe this page will be of interest:
<https://hsivonen.fi/chardetng/>

A recurring point in this thread is that someone says "proper UTF-8", or
something similar, and then we discover that things are so much worse if
one really looks, along almost any axis. Here, we see that the system
charset and time itself are the problem.

So, I am still in favor of describing how to do it right, without making
things seem nicer than they really are.

thanks,
Rob