Re: [I18ndir] Writing direction

[Asmus Freytag <[e-mail]>]

On 5/16/2022 4:40 PM, John C Klensin wrote:
>
> --On Monday, May 16, 2022 14:17 -0700 Asmus Freytag
> <[e-mail]>  wrote:
>
>> On 5/16/2022 12:25 PM, John C Klensin wrote:
>>> --On Monday, May 16, 2022 11:33 -0700 Asmus Freytag
>>> <[e-mail]>   wrote:
>>>
>>>> John,
>>>>
>>>> you are correct that any operation that's limited to the
>>>> backing store does not need to know the directionality.
>>>>
>>>> Anything you want to display, on the other hand, benefits
>>>> from knowing the display direction.
>>>>
>>>> A directionality tag on BCP47 would have the advantage that
>>>> you don't need an extra field that's needed only some times.
>>>> However, it opens the issue of what happens when a protocol
>>>> uses the existing BCP47 *and* has a separate field for
>>>> specifying directionality.
>>> I actually think that is fairly straightforward although I
>>> agree it should be spelled out.  Cases:
>>>
>>> (i) Protocol exists, specifies directionality via some
>>> separate field or markup.
>>>       --> keeps doing what it has been doing since it will not
>>> recognize the extension.
>> OK
>   
>>> (ii) New protocol comes along and specifies directionality by
>>> use of the extension.
>>>       -->  Obviously (I hope) uses the extension field.
>> OK -- that is: SHOULD use the extension field if data is
>> displayed as part of the protocol or by some process that uses
>> the protocol fields to figure out parameters for display.
> If one were designing a new protocol and decided to use
> directionality as an extension part of the language tag, why
> would one want to encourage poking around through [other]
> protocol fields or parameters to try to guess?  Or are you
> saying that there might be circumstances in which a new protocol
> might want to use other information just like the "existing"
> protocols in (i)?  I'm not sure whether or why that should be
> encouraged but don't see a good reason to try to ban it?

The question is whether it's better to get the info from a BCP47 tag or 
from some other field (in a new protocol). Thinking about it, the use of 
a dedicated field (if not optional) makes sure that it's always clear 
what the intended directionality is supposed to be. In the tag extension 
situation, it's more normal to have a way of handling a missing 
extension, so you get squishy on whether omitting the extension is 
intentional.

But it boils down to optional vs. not, not really where the info is 
presented.

Also, not all protocols deliver single strings. As in when you serve up 
an HTML file over HTTP. Then you get to handle the issue of whether the 
language and script part may be global to the data, but directionality 
may not be (or, theoretically vice versa).

A new protocol has to spell out those issues.

>
>>> (iii) Older protocol, with directionality specified as in (i),
>>> is revised.   Now it seems to me that those doing the revision
>>> have three choices:
>> Unless data is never displayed,...
>>> (a) Stick with the status quo/old way to do
>>> things,
>> OK
>>>    (b) deprecate the old method and specify use of the
>>> tag extension, or
>> OK (with your caveat)
>>> (c) allow either and specify what happens if
>>> both are provided and not consistent.  Absent plans about a
>>> flag day or sufficient changes to really create two separate
>>> protocols (the "old" and "new" ones) in the same general
>>> space, I don't know of a practical way to do (b), but it is
>>> logically possible.
>> OK: specifically, may designate one as "default" and the other
>> as "override".
> >From the standpoint of good protocol design, it seems to me a
> bad idea to tell something to look in both sets of places and
> then compare.  So this should be "look for X.  If a value is
> found/ can be determined that way, believe it without even
> figuring out whether other information might be available.  Iff
> X is not present, look for Y."  Are we saying the same thing in
> different words or do I not understand your suggestion?

Almost. For a single string you'd say "explicit direction" beats info in 
the tag extension. And you'd be fine.

But what for complex/compound data.

If each piece in the collection can have a direction attribute and 
language attribute, but the collection as a whole can also have these 
attributes.

If you override the language (locally) with a null-extension and don't 
provide an explicit direction, I posit that retaining a direction from a 
global BCP tag for a different language is a mistake.

Think document vs paragraph language in HTML. I wouldn't want every 
English paragraph in an HTML document tagged as Hebrew + RTL to show in 
RTL even if I leave out the direction extension on the language override.


>
>> Consider HTML. An obvious choice would be to allow such an
>> extension to set a default direction, so that directional
>> markup becomes redundant (only required to override the
>> default).
> Ok.  I think that is what I said above (with "X" as the
> extension) but I, personally and so far, believe the argument in
> the last part of
> https://www.w3.org/International/questions/qa-direction-from-language
> against use of an extension with HTML (and CSS, SVG, XML,
> etc.)... more or less because it would introduce confusion and
> cause a _very_ long transition.  I see the extension model as
> being far more interesting for new protocols that do not have
> separate direction metadata.
For HTML as such, the "win" would be limited of allowing extensions. But 
what about new protocols that are similar "collections" where each 
member can be tagged and displayed independently (at least what concerns 
the direction).
>
>> That would seem to increase the chances that display is
>> correct.
> It actually doesn't.   If the two ways to specify things are
> consistent with each other in a given document, then it doesn't
> increase anything, merely wastes space and/or time.    If they
> are not consistent, either one is almost always better than the
> other (little or no improvement either), or the application has
> to make an arbitrary choice between two specifications, one of
> which is wrong.
>
> That would turn into even more bad news if the markup metadata
> allowed only "LTR" or "RTL" but the extension also allowed some
> coding for "top to bottom" or "serpentine", perhaps in
> combination with one of the first two and some fallback rules.

I think you could reason yourself to a definition with reasonably clear 
precedence rules. But yes, users would need to run those in their heads 
to be sure they get what they intend. That may be a good reason to argue 
against them.

Look, I'm not actually proposing one over the other, I'm more interested 
in pointing out the need to address such issues in discussing best 
practice for protocol design.

Let me suggest one more issue: tags that are imported from some other 
part of the system.

If your model is that of an author, where the use of directionality and 
language tag can be curated by someone who can look at the text and 
decide it's correct, that's different from when you've been told some 
data is in language x (or created by a user working in a UI that's set 
to language x).

If the user profile has room for a language tag (w/ use of extensions 
not limited) then that's all you got. You could never set an explicit 
direction field based on that. if the language tag happens to cover 
direction, that's great.

You could always look for an extension, strip it off and then supply to 
the protocol in a separate field.

Anyway, not sure where I'm going with this, but I can see that usage 
scenarios will depend on your assumptions about the nature of the 
(strings in the) protocol and the source of the data used for language 
and direction tags.


>
>> Would have to address http vs. html specification of language
>> ID, does a language ID w/o extension override a later ID that
>> lacks an extension? Only if it's the same language would be my
>> guess. Otherwise you'd end up defaulting to RTL when English
>> is embedded in Arabic, but the English language tag doesn't
>> carry direction.
> Yes, I think so.  But, if anyone asked me for advice about HTTP
> or HTML, it would be to pick (iii)(a), ignore the extension if
> it appeared and the application noticed, and go merrily on one's
> way.  The existing mechanisms work well and, at least as
> important, are well understood and the size of the installed
> base is such that trying to retrofit handling of a language tag
> extension --especially if one needed to consider cases like the
> above-- would almost certainly not be worth it.

Best to understand these examples as intended for "protocols that are 
similar to" rather that suggestions to change said protocols.

But even HTML would have to state what will happen if the extension is 
used. Will there be a guarantee that it will be ignored going forward?

All good questions.

Some of them could be answered in a draft.

A./

>
> best,
>      john
>