Re: [Ietf-languages] Fwd: I-D Action: draft-msporny-d-langtag-ext-00.txt

["Doug Ewell" <[e-mail]>]

Mark Davis wrote:

> 1. So from the tag "ar-Arab", we get the script "Arab". Then use
> https://github.com/unicode-org/cldr/blob/master/common/properties/scriptMetadata.txt,
> which has a mapping from script to direction (RTL=YES). (I'm pointing
> to trunk, just so people can read the file easily; one would use the
> latest release.)

Yes, all right, if you are a text process that renders or formats text, but you don't know that the Arabic script is RTL, then fine, use CLDR to tell you that. OK.

> 2. But let's suppose that you have just "ar". Since the script is not
> explicit, the best way to get it is also CLDR. You can use
> https://github.com/unicode-org/cldr/blob/master/common/supplemental/likelySubtags.xml,
> which has a mapping from language or language+region to default
> language-script-region. So "ar" => "ar_Arab_EG", from which we get the
> script "Arab", and then use step 1. Or from "fr" you'd get "Latn" and
> map it to RTL=NO.

You don't need CLDR for this step. The BCP 47 Language Subtag Registry could have told you that:

Type: language
Subtag: ar
Description: Arabic
Added: 2005-10-16
Suppress-Script: Arab      ← THIS
Scope: macrolanguage

I mean, jeez, that's why we invented Suppress-Script in the first place. (Well, sort of. We invented it to go the other way, when you're *creating* a tag, so when you're tempted to write "ar-Arab" you can stop and just write "ar" instead. But this scenario works just as well.)

If you have a more complex matching scenario in mind, involving region and variant subtags, or crossing over from Breton to French or whatever, then go ahead and use CLDR. Or, I suppose, you can also use it here if you already didn't know Arabic was a RTL script in #1 and you had to have CLDR available for that anyway.

This is a lot like RFC 4647 matching: it isn't good enough for all scenarios, so you might need customized or more ambitious matching algorithms for those, but for many simpler cases it's just fine. But it DOES exist and there's no need to pretend it doesn't and invent another basic scheme.

> It isn't that Arabic would be displayed left to right, it is what
> establishes the paragraph ordering. The problem arises when you have
> mixed text. Look at the following example, using the convention that
> lowercase = English and uppercase=Arabic. The majority of the text and
> the first strong character are both English, but the sentence is meant
> to be used in an Arabic environment, so the default paragraph
> embedding level needs to be RTL.

It still doesn't sound as though inserting 'ltr' or 'rtl' into the language tab would solve this problem. We have Unicode bidi controls for this. (I know HTML has its own tags for this, which are preferred in that context, but the GitHub thread indicates this isn't mostly about HTML.)

>> 4. Scripts exist in other directionalities besides LTR and RTL...
>
> While this is true, for the [v]ast majority of cases, LTR and RTL are
> the important issues. Most computer systems don't really handle
> vertical natively; one needs to have more specialized text processing
> systems, and that is not, I imagine, the target for this syntax.

Today, yes. N years from now, when these specialized text processing systems become mainstream, maybe it will be a different story. You don't want to lock out that possibility by hard-coding the set of allowable values into the RFC for all time.

> I don't see that there is any reason to approve it, given that it is,
> as far as I can tell, completely unnecessary and would just complicate
> implementer's lives to no good end.

Agreed, obviously.

What we need to do is get together with the folks on the GitHub thread and explain the situation to them, how this proposed solution is neither necessary nor sufficient, and show them the right way(s) to do what they need.

--
Doug Ewell | Thornton, CO, US | ewellic.org