Re: [art] AD review of draft-ietf-appsawg-file-scheme-13.txt

[John C Klensin <[e-mail]>]

--On Wednesday, November 23, 2016 11:47 +1000 Matthew Kerwin
<[e-mail]> wrote:

> ​Oh, yeah, I think I overlooked UTF-16 and EBCDIC, which
> contain characters that are allowed by the syntax but don't
> encode to US-ASCII-compatible octets. 

You are confusing "encoding" with "coded character set".  The
_characters_ available in Unicode encoded in UTF-16 are, by
definition, exactly the same as the _characters_ available in
Unicode encoded in UTF-8.  EBCDIC is another matter.  The
character repertoire and code point assignments depend on
selection of code pages; some have exact and unambiguous
mappings to Unicode but I wouldn't count on that being true for
all of them.

I also don't know what a "US-ASCII-compatible octet" is.  Unless
one turns that high bit on, bits are bits.

> How about this?:
 
> "
> When a file URI is produced, characters SHOULD be encoded
> using UTF-8, and any characters not allowed by the syntax in
> Section 2 MUST be percent-encoded.
> "
 
> I think that recaptures the original intent better as well,
> now I look at it.

See my previous note.   First of all, percent-encoding really
encodes octets, not characters.  That may have been a mistake
(see RFC 5137) but it is far too late to consider changing that
now.  Outside the ASCII range (and _only_ outside the ASCII
range), the thing that makes [Unicode in] UTF-8 important is
that percent-encoding of, e.g., "Ω" (U+03A9) yields "%CE%A9"
and not, e.g., %0e%A9 which is what one would get if one encoded
the UTF-16 octets instead.   Again, RFC 3986 (and hence Section
2) disallow "/" (U+002F) but it is a (by definition
single-octet) ASCII character and its percent-encoding is %2F
for reasons that have everything to do with the original
definition of percent-encoding and just about nothing to do with
the ASCII<->UTF-8 relationship.

There actually is a protocol issue here and, if you are going to
say "SHOULD" rather than "MUST", perhaps we should make you
address it.   If I have a file system whose file names and paths
are natively expressed in Unicode encoded in UTF-16, then it
actually is a big deal whether you expect that, at least outside
the ASCII repertoire and maybe including it) to be transcoded to
UTF-8 and then percent-encoded or whether, since 3986 doesn't
make specific allowances for other character sets or encodings,
you want to treat _all_ file names and paths expressed in the
native form of that file system as "not allowed by the syntax"
end percent-encode the native octets.   If you want to allow the
latter, you need to explain it.  If you intend the former, then
non-ASCII character MUST be in (or converted to) Unicode and
encoded in UTF-8 because anything else will result in ambiguity
and nasty interoperability problems.

    john