Re: [I18ndir] Writing direction

["Martin J. Dürst" <[e-mail]>]

Hello everybody,

Sorry I'm quite late with my reply. My conclusion is that it's most 
probably not worth creating a bidi extension for language tags. See 
below for the reasons.

On 2022-05-18 14:32, Asmus Freytag wrote:
> On 5/17/2022 9:57 PM, Patrik Fältström wrote:

>> Ok, obviously without knowing the complete context here I would say 
>> that first of all the big problem is mixing protocol parameters with 
>> display. I call this "leakage". We see this in DNS where a domain name 
>> is visible to the user. We see it in other parts of a URI, an email 
>> address etc. Oh, email address is a perfect example. It does have a 
>> "free text" name, and then an address. But many people want to use 
>> their name as an email address which leads to collisions and other 
>> things. This while applications that only show the name have similar 
>> security risks like text that is a link that people click on might 
>> have a destination that is not what the end user guesses or believes.
>>
>> To the "free text".
>>
>> To me there are two issues here:
>>
>> 1. Display is very important to the end user. We have the context 
>> within which the sender of the text has, and the context of the 
>> receiver of the text. If a text is to be displayed we even without 
>> talking about general directionality (that do impact rendering) we 
>> have the issue of mixing two contexts. Even if I have some clue about 
>> i18n I have very to no knowledge about the same text, same script, 
>> same language is possible to display with different directionality. I 
>> believe some asian scripts can do this, and for example hebrew. So the 
>> first question is what problem is to be solved. I guess it is "to have 
>> the receiver understand what general directionality the sender of the 
>> text decides". The receiver can then display in whatever 
>> directionality context the sender wants.
>>
>> 2. Second question is whether general directionality is a degree of 
>> freedom that is really needed in this protocol. I think it is really 
>> really really important to agree this *is* important. And I mean that 
>> it is much more important than deciding that "the free text in this 
>> protocol has a directionality context that is R2L", or L2R for that 
>> matter. I.e. that this protocol element (because it is a protocol 
>> element after all, even if the element contains "free text"). If the 
>> string is short, I claim one can create a string with the help of 
>> directionality is like if the general directionality was the opposite 
>> of what the general directionality is.
>>
>> 3. If the answer to the second question is that one can absolutely not 
>> have a given directionality, I still think one should not give up. One 
>> can still say that "the directionality of the free text element is 
>> R2L", with the addition "if the free text element is to be a L2R 
>> context, then the first character of the element MUST be U+2066 
>> "Left-To-Right Isolate".

A side remark: Because LTR is much more widely used, I think the default 
should be LTR, and RTL should be marked with
RIGHT-TO-LEFT ISOLATE (U+2067). Also, if you put that in, then please 
also put in a POP DIRECTIONAL ISOLATE U+2069 at the end.

But in very many cases, simply requiring the use of first-strong (auto) 
isolation when including the protocol element in a bigger context may be 
okay. The use of first-strong isolation guarantees that all the text is 
kept together in one junk (possibly flowed across multiple lines 
depending on length).

<snip>

> embedding bidi controls into protocol text data is ugly, because they 
> end up, sooner or later, embedded in the plain-text backbone of an HTML 
> page. (I'm sure that's a law that's already named by someone out there).

Yes, fully agreed, it's ugly. But it's not uglier than a language tag 
which includes directionality information ending up in an HTML page. In 
addition to that, having one kind of ugliness is clearly better in my 
eyes than having two separate kinds of ugliness. Also, the W3C 
Internationalization activity already has an article discussing this 
case: https://www.w3.org/International/questions/qa-bidi-unicode-controls.

Also, please note the following: If we introduce some kind of bidi 
extension for language tags, whenever the protocol elements in question 
are put in some context, we have to remove that extension. Either the 
protocol element gets added to plain text, in which case we have to add 
bidi controls, or it gets added to (HTML) markup, in which case, we have 
to add markup (and remove the extension). It doesn't necessarily look 
like good protocol design to create something that has to be removed as 
soon as it's actually used :-(.

<snip>

> Unlike all the other presentational markup that exists to affect 
> text-layout, the bidi direction is special in that it affects things 
> like the order of first and last name or any other elements where "order 
> in the sentence" affects the meaning (and not just the appearance) of 
> the text.

I agree on this point, of course.

<snip>

> What are the types of texts that show up in IETF protocols?

That's indeed the most important question in this discussion. Things may 
range from protocol elements that occasionally have to be checked by a 
human to stuff that's essentially intended for humans only. They may 
also range from very short word-like stuff to very long texts.

Solutions range from restricting the protocol element to using a single 
script (similar to the restrictions we have on DNS labels) to requiring 
that the protocol element be included in <bdi></bdi> when added to 
markup (see the example in 
https://html.spec.whatwg.org/#the-bdi-element) to the advice "you'd 
better allow markup here anyway".

This means that the range of 'protocol elements' where a bidi extension 
to language tags might make sense is actually not very wide. Combining 
this with the fact that it has to be removed as soon as it's used 
somewhere, the idea of a bidi extension to language tags looks pretty 
much like a nonstarter to me. Sorry it took me so long to get to that 
conclusion.

Please also note that depending on the nature of the protocol element, 
language information may not be needed. As an example, there were 
discussions about adding language information to domain name labels when 
domain name internationalization was undertaken. But it was easy to 
convince people that it doesn't make sense. There is no difference 
between an English version of the label 'chat' and a French version, 
although their pronunciation and meaning differs depending on the 
language. What will happen is that each user will interpret and 
pronounce the label depending on their language abilities and the 
context it is in, and that's it. That may apply to cases where bidi 
information (or at the least, bidi isolation) is still highly desirable. 
Another reason for not adding a bidi extension to language tags.


Regards,    Martin.