Re: [I18ndir] [art] Just uploaded draft-bray-unichars-03

[Asmus Freytag <[e-mail]>]

It seems to me that the concerns about repertoire exist independent of 
whether ill-formed data can pass.

The proposed subset excludes noncharacters, which none of them lead to 
ill-formed encoding forms. It also excludes controls, again, those may 
be excluded in some other repertoires as well, but they don't represent 
an issue for the UTFs.

However, if you adopt a repertoire, it's necessary to say what that 
means. And that includes saying what will happen with data that is 
outside that repertoire. (Whether or not is is ill-formed on the level 
of the encoding form).

And part of that would include whether the specification is written as a 
text pipeline where the sender can stick escape sequences into the 
stream and the eventual receiver is expected to handle those as it they 
were permitted in the data stream (while all intervening partners can 
ignore them).

Or you would assert that the repertoire provides a firm restriction on 
the contents of any data and escape sequences may not exceed the 
repertoire. And conformant implementations will need to respond with 
proper warnings, exceptions etc.

All of these concerns should be addressed by a specification that uses 
the various subsets (and the draft should spell out guidance).

Icing on the cake would be if there was some terminology that 
differentiated these two approaches.


Seen in this light, the surrogates are only a special instance of a more 
general issue (except that they are also ill-formed on the level of the 
encoding form.). Therefore, it is probably not productive to deep end on 
which libraries raise exceptions properly. It would suffice to note that 
unless the specification (using a subset) itself addresses the need for 
strict response to ill-formed or out-of-repertoire input, 
implementations depending on it cannot be sure that the input is clean.

A./