[Ltru] Macrolanguage usage

Peter Constable and I were at the Unicode Technical Committee meeting, and
had a chance to talk about macrolanguages. One of the key points is to
provide implementers with enough guidance as to what they should do, while
not precluding reasonable alternatives. Here's what we were thinking about
(not in formal language, but the points we wanted to make).

===

Formally, a macrolanguage identifier could be used to tag or look up content
in any encompassed language; alternately, specific, individual-language
identifiers could be used to tag or look up content in those languages. In
consideration of these alternatives, the following provides guidance for how
to maintain backwards compatibility while giving the ability to clearly tag
and lookup content.

1. Implementations generally should tag and/or lookup content with the
specific language where possible (this is the general recommendation with
language tags). In the case of macrolanguages, this means that Cantonese
should be tagged and looked up with "yue", Hakka with "hak", Tajiki Arabic
with "abh", Plains Cree with "crk", and so on.

2. An exception to this general recommendation may apply in the case of
macrolanguages with predominant forms, listed in Table 8. For backwards
compatibility in those cases:

   - an implementation could tag and/or lookup content in the predominant
   language either with the macrolanguage or the encompassed language. (eg
   either "ar" or "arb" for Standard Arabic).
   - an implementation could make a distinction between these in lookup, or
   could return the same content. That is, lookup for "zh-SG" and "cmn-SG" may
   return the same content, or may return different content if the
   implementation needs to make a distinction for some purpose.
   - where content written in an encompassed language is also understandable
   in the predominant language (that being a distinct language encompassed by
   the same macrolanguage), the content could also be tagged with the
   macrolanguage identifier. Thus if a Cantonese passage is understandable if
   read as Mandarin, it could also be tagged with "zh", or where a Tajiki
   Arabic passage is also understandable in Standard Arabic it could be tagged
   with "ar".

3. Another exception to this general recommendation applies in the case of
applications that have limitations that exclude the identifiers for
encompassed, individual languages of a macrolanguage. For example, some
content cataloguing systems limit language identifiers to those in ISO
639-2; as a result, they may support a macrolanguage identifier but not the
identifiers for the encompassed languages of that macrolanguage.

Mark and Peter

--
Mark