Re: [I18ndir] I-D on filesystem I18N

[Nico Williams <[e-mail]>]

On Tue, Jul 07, 2020 at 02:20:43PM -0400, John C Klensin wrote:
> --On Tuesday, July 7, 2020 10:58 -0500 Nico Williams
> <[e-mail]> wrote:
> > Hmm, unless you meant that mixing of RTL and LTR scripts
> > should be forbidden.  But that's not a realistic proposition
> > given that file extensions tend to be a) ASCII (LTR) and b)
> > required, so even within a single directory it may not be
> > possible.  And even where file extensions are not required,
> > there is the problem of "dotfile" conventions for hiding
> > files.  And even if all of those were non-issues and we could
> > ban mixing of RTL & LTR scripts in any given _filename_, we
> > could never keep script mixing from happening in absolute
> > _paths_.  So we can't effectively avoid display issues by
> > forbidding mixing of RTL/LTR scripts.
> > 
> > Display issues for mixed RTL/LTR paths affect a very large
> > variety of software, from terminal emulators to browsers
> > (because paths tend to be used as URI/IRI local parts) to
> > native GUI apps, and much more.  What more can we do about
> > them than reference Unicode?
> 
> First of all, remember that, as long as numerals are expressed
> in positional form (e.g., with Indo-Arabic digits and their
> equivalent in other scripts), there is no such thing as a
> strictly RTL script.  That is at least one reason why "bidi" is
> much more appropriate as a characterization of those scripts
> than RTL, even with no script mixing.   The problems you are

Sure.

> discussing are, however, exactly where "just referencing
> Unicode" breaks down.  The Bidi algorithm was designed for, and
> works really well for, ordinary running text, at least for
> scripts with syntactically unambiguous "words" and word
> boundaries.  As soon as something has a special syntax because
> it is not running text but a special type of object expressed in
> text form -- such as a filename-- unthinkingly applying the Bidi
> algorithm leads to trouble.  

That's fair, but again, running code that displays file paths and file
names and URIs/IRIs is common and far flung.  I agree that we could
borrow from RFC 5893 to provide advice to implementors of code that
displays any of those things.  We just need to understand that we won't
be boiling any oceans, and that advice regarding display targets a very
different set of implementors than advice regarding filesystem protocols
and advice regarding filesystems themselves.

> We've recognized that it the past.  For example, RFC 5893 does
> not say "just use Unicode Bidi".  It recognizes that domain
> names provide special challenges (with the most obvious example
> being that DNS labels are not required to follow the rules for
> "words" in relevant languages and because the fundamental syntax
> for non-TLD FQDNs is Label.Label... and so provides rules for
> dealing with domain names that are different from the Unicode
> Bidi rules.   It seems to me that precisely the argument you've
> made leads to a conclusion that file names and file paths are
> special types of objects and, if they are allowed to include
> code points drawn from nominally RtoL scripts, then they are
> going to need their own rules.  

Possibly.  Keep in mind that we don't and won't have a filesystem
equivalent of A-labels, and we don't have a notion of filesystem-aware
apps or protocol slots.

On the plus side, these are _filesystems_, not _the DNS_.  The problems
that arise have much much smaller impact.

(Unless we bring IRI local parts into scope, which we might do since
often IRI local parts are taken from filesystems.  But still, the impact
of bidi issues there is much smaller than for the DNS.  Mind you, I
don't want to blow up the I-D's scope too much, and would rather say
anything about IRI local parts in a subsequent document, if we do it at
all.)

> Now, of course, you could write your spec to say "this works
> only for filesystems that do not allow characters to which
> Unicode assigned RTL properties and it may not be appropriate
> for scripts that encounter significant issues when putting
> identifiers through Case Folding or that have compatibility
> equivalents to other characters either".   I think you would
> have trouble getting IETF consensus (or adoption in the field by
> people who already have their own systems --even if ASCII-only--
> and would need to change them) with such restrictions but, if
> you disagree, go for it.

I don't think we need to do that.  We could say that bidi can cause
problems, describe the impact, and provide implementation advice, just
as long as we understand that it's unlikely that all the relevant
software will be updated accordingly any time soon.

> Again, I'd recommend careful study of the PRECIS specs, at least
> to the extent of either being able to appropriately profile them
> for some of your needs or being able to explain why they are not
> relevant.

Noted.

Nico
--