Re: [Ltru] Re: extlang

["Mark Davis" <[e-mail]>]

I see it a somewhat different way. The fact that there is a macro language
(zh), should not skew the way we encode an individual language (yue). You
say: "Extlangs help this a little by avoiding the choice on the initial
subtag. Thus: "ar-EG" is related to "ar-arz-EG" and "ar-arb-EG" in distinct
and somewhat logical ways."

However, functionally, we have to see advantage in subordinating some
languages as extlangs. There isn't value if they just add complication, as
per your "The problem with lookup (and I use lookup extensively, so it
concerns me deeply) might suggest that some extra smarts related to extlangs
is going to be needed."

In order to have a good case for the extlang model, we would need to see
concrete scenarios where we can demonstrate that "zh-cmn" and "zh-yue" work
better than "cmn" and "yue" resp., and demonstration that those are more
important than the scenarios where they cause problems.

Note that ISO 639-3 does not at all force the use of the extlang model; the
extlang model is just one possible way of expressing the information in ISO
639-3. We already have macrolanguages and "subordinate" languages in BCP 47,
but we put them at the same level. For example, ISO 639-3 already
categorizes no and sh as macrolanguages, and nb, no and sr, hr, and bs as
subordinate to them. We are not going to (and cannot) be forcing users to
encode nb as no-nb, nor sr as sh-sr. We shouldn't be making Cantonese a
subordinate language either.

At one point, I did think that having the extlang structure would be better,
but the more I get into actually implementing them, the more I find that
they are just a complication for no good result. What I think we should
instead be doing is adding a field to the registry that says that X is a
macro language for Y, and adding information to 4647 that indicates how one
can make use of this information in matching. That would also extend to the
current situation with no and sh in a uniform manner. It is also a much less
fragile mechanism: one can add more such X,Y relations over time without
gumming up everything.

Mark

On 3/19/07, Addison Phillips <[e-mail]> wrote:
>
> The idea of macro-languages is that they are not, themselves, languages.
> Rather, they are groupings of languages that can be usefully referred to
> collectively. That is, strictly speaking, "Chinese" (meaning "zh") isn't
> a language---and neither is "Arabic" (by which I mean the code "ar").
>
> Unfortunately, that isn't common usage or understanding of the
> situation. To most people, "Arabic" is a language. The idea that it has
> regional or other variations seems natural enough, but not, perhaps,
> that it is a set of somewhat related languages that share a historical
> and/or written tradition.
>
> Allowing both the macro- and plain-language codes on the same level is
> recipe for confusion: does "ar-EG" == "arb-EG" == "arz" == "arz-EG"?
> What is supposed to match? Which tag should be used for a given
> document? How should one distinguish these?
>
> We have a tradition, furthermore, of not having "secret information" in
> language tags requiring extra mapping tables to make sense of or process
> the tags. This tradition is punctuated (and punctured) by an equal
> tradition of assigning "secret" meaning to language tags. Thus, for the
> longest time, "zh-TW" meant "Traditional Chinese".
>
> Extlangs help this a little by avoiding the choice on the initial
> subtag. Thus: "ar-EG" is related to "ar-arz-EG" and "ar-arb-EG" in
> distinct and somewhat logical ways.
>
> Mark's concern is that this tagging system doesn't play nicely with
> basic filtering. He's not cited the fact that we have two filtering
> schemes: extended filtering works more reasonably with extlangs.
> "zh-yue-Hans-CN", "zh-cmn-Hans", and "zh-Hans-CN" all match the range
> "zh-Hans", for example.
>
> The problem with lookup (and I use lookup extensively, so it concerns me
> deeply) might suggest that some extra smarts related to extlangs is
> going to be needed. On the other hand, some of this is going to have to
> be related to Maxim #1: "Tag Content Wisely". Choosing to avoid extlangs
> where they add no distinguishing information (not uncommon in resource
> file lookup for Arabic or Chinese, say) or *consistently* including them
> when they do (Lahnda???) will make things better.
>
> I admit to a good bit of trepidation writing the above, though.
>
> Addison
>
> Don Osborn wrote:
> > Thanks Frank for the summary list and Mark for the pointer. Actually I
> > was going to reference that page and ask if it is exactly what is in
> > question.
> >
> >
> >
> > Next (trying my luck here), is there any kind of way(s) that these can
> > be subgrouped? For instance ar ends up referring to standard Arabic, if
> > I understand correctly, and zh has been discussed already a lot; but
> > some other macrolanguages do not necessarily have a single standard
> > form. Some (macro) languages have a lot more in writing than others.
> > What I'm getting at is are there different sets of (possible)
> > complications that can be identified for the macrolanguages, languages
> > and extlang relationships, such that the list of 54 can be disaggregated
> > (perhaps in more than one way)?
> >
> >
> >
> > Not as if folks don't have enough else to think about, but it seems like
> > such an analysis, if it hasn't been done, might raise other productive
> > questions.
> >
> >
> >
> > Don
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *From:* Mark Davis [mailto:[e-mail]]
> > *Sent:* Sunday, March 18, 2007 2:37 PM
> > *To:* Frank Ellermann
> > *Cc:* [e-mail]
> > *Subject:* Re: [Ltru] Re: extlang
> >
> >
> >
> > Another way to look at it is on
> > http://www.sil.org/iso639-3/macrolanguages.asp, which provides the
> > language names and breakdowns.
> >
> > Mark
> >
> > On 3/18/07, *Frank Ellermann* <[e-mail]
> > <mailto:[e-mail]>> wrote:
> >
> > Don Osborn wrote:
> >
> >  > what is the actual number of (macro)languages where extlang issues
> arise?
> >
> > I try to determine this "manually" for Doug's latest 4645bis:
> > There are 480 "Prefix:" lines in the extlang part.
> >
> > ar      30              ay       2              az       2
> > bal      3              bik      5              bua      3
> > chm      2              cr       6              del      2
> > den      2              din      5              doi      2
> > fa       2              ff       9              gba      5
> > gn       5              gon      2              grb      5
> > hai      2              hmn     21              ik       2
> > iu       2              jrb      5              kg       3
> > kok      2              kpe      2              kr       3
> > ku       3              kv       2              lah      8
> > man      7              mg      10              mn       2
> > ms      13              mwr      6              oc       5
> > oj       7              om       4              ps       3
> > qu      44              raj      6              rom      7
> > sc       4              sgn     124             sq       4
> > sw       2              syr      2              tnh      4
> > uz       2              yi       2              za       2
> > zap     58              zh      13              zza      2
> >
> > That's 54 at the moment.
> >
> > Frank
> >
> >
> >
> > _______________________________________________
> > Ltru mailing list
> > [e-mail] <mailto:[e-mail]>
> > https://www1.ietf.org/mailman/listinfo/ltru
> >
> >
> >
> >
> > --
> > Mark
> >
> >
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Ltru mailing list
> > [e-mail]
> > https://www1.ietf.org/mailman/listinfo/ltru
>
> --
> Addison Phillips
> Globalization Architect -- Yahoo! Inc.
>
> Internationalization is an architecture.
> It is not a feature.
>



-- 
Mark

_______________________________________________
Ltru mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/ltru