Re: [I18ndir] [art] Fwd: New Version Notification for draft-bray-unichars-06.txt

[Rob Sayre <[e-mail]>]

On Mon, Sep 25, 2023 at 6:10 PM Tim Bray <[e-mail]> wrote:

> On Sep 25, 2023 at 11:06:38 AM, Rob Sayre <[e-mail]> wrote:
>
>> typo: "by by"
>>
>
> Oops. Couple more too, got sloppy this draft.
>

FWIW, I didn't notice it myself. GMail noticed it when I pasted in the
quote. I think a grammar checker of your choice (MS Word, Grammarly,
Google) etc is worth it here, because writing about language is difficult.
It would just save some back-and-forth.


> "There are reasonable options for dealing with problematic input. First,
>> an implementation
>> can reject text containing problematic input. Secondly, it's possible to
>> replace problematic
>> code points with placeholders.  Responding to that risk, [UNICODE
>> <https://www.ietf.org/archive/id/draft-bray-unichars-06.html#UNICODE>] section
>> 3.2 recommends
>> dealing with ill-formed byte sequences by signaling an error, or
>> replacing problematic code points with '�' (U+FFFD, REPLACEMENT
>> CHARACTER). Lastly, it can make sense to accept it, if the entire
>> implementation
>> is designed to accommodate ill-formed Unicode."
>>
>> Not attached to my words, just trying to get the point across.
>>
>
> Not sure what that point is. The proposed language loses the narrative
> about silently ignoring, and then the added “make sense to ignore” flies in
> the face of that narrative, which I think is important given the security
> considerations.  What do others think?
>

You're right. I didn't mean to drop that sentence. Let me try again:

"There are reasonable options for dealing with problematic input. First, an
implementation
can reject text containing problematic input. Secondly, it's possible to
replace problematic
code points with placeholders. Silently ignoring an ill-formed part of a
string is a known
security risk. Responding to that risk, [UNICODE] section 3.2 recommends
dealing with ill-formed byte sequences by signaling an error, or replacing
problematic code points
with '�' (U+FFFD, REPLACEMENT CHARACTER). Lastly, it can make sense to
accept it, if the
entire implementation is designed to accommodate ill-formed Unicode."

The aim here is to point out that the Web (and Java, and Windows) can
accommodate ill-formed Unicode. Is it possible to transmit any Windows path
name via conforming JSON in UTF-8? Yes. Is it a good idea to naively design
that into a protocol? No. But you might have to accept these things to be
sufficiently compatible with the Web.

One could call some of this input "toxic waste", but there is a flipside.
It would be something like "I'm sorry ACME Basic JSON Parser Version 1.0
can't handle web content". The other formats in the document make this
intent explicit, and I support that.

thanks,
Rob