Re: [Ietf-languages] Fwd: I-D Action: draft-msporny-d-langtag-ext-00.txt

["Phillips, Addison" <[e-mail]>]

Some background would be probably useful in this discussion. I’m playing catch-up here due to the U.S. holiday.

This is an outgrowth of a recent review of a number of specifications at W3C (in Manu’s case, specifically the Verifiable Claims spec). Like many current specs, VC uses JSON as a file format and depends on Linked Data specifications such as RDF and JSON-LD. The W3C I18N working group identified some time ago that these formats do not provide for metadata to set the base direction in the UBA and we’ve been raising the visibility of this—we have an open issue with the TAG and comments on a variety of specs.

Here are useful links for context: [1][2]. To document the requirements and current status Richard Ishida and I have been editing a W3C note “string-meta” [3] which might make a good explainer (send comments to our github).

The problem here is there are specs that need to progress in maturity, whose authors accept the need to manage base direction, but which have no way to do so with RDF/JSON-LD compliant serializations.

From the discussion below/elsewhere, I’m obviously aware of this I-D. However, I do not agree with it at present. I think that there are other, better ways to address this issue.

I’m not going to respond to the entirety of this thread because this discussion is better off happening at [1], in the RDF space, where it is actually a problem and I would urge those interested to decamp there.

Additionally, I am hosting a call on Wednesday at 1300 UTC. Details can be found at [1].

Thanks,

Addison

Addison Phillips
Sr. Principal SDE – I18N (Amazon)
Chair (W3C I18N WG)

Internationalization is not a feature.
It is an architecture.



[1] https://github.com/w3c/rdf-dir-literal/issues/3<https://github.com/w3c/rdf-dir-literal/issues/3#issuecomment-495937506>
[2] https://github.com/w3ctag/design-reviews/issues/178
[3] https://w3c.github.io/string-meta/ (or the official copy: https://www.w3.org/TR/string-meta)

From: Mark Davis ☕️ [mailto:[e-mail]]
Sent: Monday, May 27, 2019 6:52 AM
To: Doug Ewell <[e-mail]>; Phillips, Addison <[e-mail]>
Cc: Manu Sporny <[e-mail]>; Martin J. Dürst <[e-mail]>; IETF Languages Discussion <[e-mail]>
Subject: Re: [Ietf-languages] Fwd: I-D Action: draft-msporny-d-langtag-ext-00.txt

Doug, I agree most of the points you are making, especially #1.

I think what they are trying to do is shoehorn in a parameter that lets them set the paragraph embedding level (https://unicode.org/reports/tr9/#BD4) for the Bidi Algorithm. But instead of that hack, as you point out, one can deduce the direction from the language tag. The best way to do this is to get the ordering from CLDR for an ordinary language tag like "ar" or "ar-Arab".

1. So from the tag "ar-Arab", we get the script "Arab". Then use https://github.com/unicode-org/cldr/blob/master/common/properties/scriptMetadata.txt, which has a mapping from script to direction (RTL=YES). (I'm pointing to trunk, just so people can read the file easily; one would use the latest release.)

2. But let's suppose that you have just "ar". Since the script is not explicit, the best way to get it is also CLDR. You can use https://github.com/unicode-org/cldr/blob/master/common/supplemental/likelySubtags.xml, which has a mapping from language or language+region to default language-script-region. So "ar" => "ar_Arab_EG", from which we get the script "Arab", and then use step 1. Or from "fr" you'd get "Latn" and map it to RTL=NO.

A few more comments below.

On Mon, May 27, 2019 at 2:47 AM Doug Ewell <[e-mail]<mailto:[e-mail]>> wrote:
Manu Sporny wrote:

> There is a time pressure here. Our i198n concerns have been hanging
> out there for more than 9 months and our WG charter is up in a couple
> of months. We need to wrap this up in 3 weeks. Or to put it another
> way, if we don't wrap this up in 3 weeks, we won't be addressing this
> issue, which would be a shame.

I know it is flippant to say "that's not our problem," and I apologize in advance for that, but trying to push through this extension quickly, without consulting or even notifying the language-tagging community, does not seem to me an appropriate way to compensate for this lapse. It was only by chance that Martin happened to spot this I-D and was able to bring it to our attention.

Apparently Addison did know about this effort, and is credited in the Acknowledgements section of the I-D, but it would be nice if the author(s) of an extension proposal would check in with ietf-languages as part of their effort. RFC 5646 does not require this; I wish it did. The IETF at large and W3C are not experts in this field, and probably will not be able to detect significant operational problems in such a proposal.

> In any case, if you're going to engage in this discussion, the issue
> #3 above is probably the place to do it.

I believe THIS LIST is the place to discuss this I-D. (Definitely not on some GitHub account.)

I have other questions and/or concerns, some of which overlap with Martin's:

1. In the proposal's lone example, the Arabic script is a right-to-left script. How does "ar-d-rtl" indicate right-to-left directionality in a way that "ar-Arab" does not?

2. Given #1, and given that the script subtag 'Arab' is a Suppress-Script for the language subtag 'ar' (which means "ar" is equivalent to "ar-Arab" for almost all purposes), how is "ar" not sufficient? I agree with Martin's comment here: what rendering process is likely to display Arabic left-to-right?

It isn't that Arabic would be displayed left to right, it is what establishes the paragraph ordering. The problem arises when you have mixed text. Look at the following example, using the convention that lowercase = English and uppercase=Arabic. The majority of the text and the first strong character are both English, but the sentence is meant to be used in an Arabic environment, so the default paragraph embedding level needs to be RTL.

rindfleischetikettierungsüberwachungsaufgabenübertragungsgesetz IS A LONG WORD.

3. I also agree with Martin that the definition "automatically detected" for subtag 'auto' is not adequate. How does it differ from leaving off the D extension altogether?

Agreed, not well specified. But -d- is not needed in the first place, so moot.

4. Scripts exist in other directionalities besides LTR and RTL. Chinese, Japanese, and Korean can be written top-to-bottom, right-to-left. Mongolian in Mongolian script is properly written top-to-bottom, left-to-right, but is sometimes (although incorrectly) rendered LTR as well. Some languages have been written boustrophedon, either with or without reversing the glyphs when transitioning from LTR to RTL. None of these scenarios are covered in the proposal, but some of them seem much more in need of explicit marking than the Arabic example.

While this is true, for the fast majority of cases, LTR and RTL are the important issues. Most computer systems don't really handle vertical natively; one needs to have more specialized text processing systems, and that is not, I imagine, the target for this syntax.


5. Given #4, the lack of a registry for the proposed extension, or even the mention of one, is a significant problem. The set of exactly 3 values associated with this extension ('ltr', 'rtl', and 'auto') would be fixed; adding to it would require updating the RFC, which is much more work than updating a registry.

Agreed, that would be a major drawback.  But -d- is not needed in the first place, so moot.


Without these issues being addressed in a satisfactory way, I would lobby IETF not to approve this I-D.

I don't see that there is any reason to approve it, given that it is, as far as I can tell, completely unnecessary and would just complicate implementer's lives to no good end.

--
Doug Ewell | Thornton, CO, US | ewellic.org<http://ewellic.org>


_______________________________________________
Ietf-languages mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/ietf-languages