Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching

[John Cowan <[e-mail]>]

It was only done for macrolanguages that already had compound tags like
zh-yue, plus a few others that seemed likely to be treated similiarly,
often because they only a single written form between them.  Aymara is a
macrolanguage with two individual languages, Central and Southern, but only
Central has a written form and is an official language (of Bolivia).  If
you don't care about the diffference, you can use ay, or you can use ayc or
ayr to distinguish, but nobody has to worry about abstracting over them.



On Thu, Aug 29, 2019 at 11:55 PM Florian Rivoal <[e-mail]> wrote:

>
>
> On Aug 30, 2019, at 11:17, John Cowan <[e-mail]> wrote:
>
> Got it. Yes, with extended filtering, you need to canonicalize toward
> extlangs.  I thought you were talking about simple equality matching after
> canonicalization.
>
> Note that almost all macrolanguages do not have extlang tags; the
> languages can be identified by the macrolanguage or by the individual
> language tag, but not both.  Indeed, only ar, kok, lv, ms, sw, uz, zh have
> extlang tags subordinated to them;
>
>
> Ah, indeed, I had failed to notice that. That doesn't make what I am
> proposing to do wrong, but it does limit its usefulness to a subset of the
> macrolanguage families. So my Chinese example would work, but my Norwegian
> one wouldn't.
>
> Is there a reason for the lack of extlang on these macrolanguages that
> don't have it? inferring the equivalent information by inverting the
> "Prefered value" information doesn't sound hard, but if it wasn't done,
> presumably there's a reason.
>
> sgn does too but is not a macrolanguage.
>
>
> It doesn't sound terribly useful to me to do the extlang canonicalization
> for the sake of sgn, but neither does it sound harmful.
>
> —Florian
>
>
> On Thu, Aug 29, 2019 at 9:14 PM Florian Rivoal <[e-mail]> wrote:
>
>>
>>
>> > On Aug 30, 2019, at 10:08, Florian Rivoal <[e-mail]> wrote:
>> >
>> >
>> >
>> >> On Aug 29, 2019, at 22:44, John Cowan <[e-mail]> wrote:
>> >>
>> >>
>> >>
>> >> On Thu, Aug 29, 2019 at 5:00 AM Florian Rivoal <[e-mail]>
>> wrote:
>> >>
>> >> In case that influences what you have to say, note that what I intend
>> to do is not to store the canonicalized-to-extlang form anywhere. It would
>> only be for internal processing: when performing an extended filtering
>> operation, where it is unknown whether the ranges and tags are in extlang
>> form or not, canonicalize both to extlang form do the extended filtering
>> operation on that.
>> >>
>> >> In that case you can equally canonicalize away from the extlang form
>> as toward it.  I recommend that.
>> >
>> > Can you?
>> >
>> > Let's say you want to match (using extended filtering) the zh range
>> against documents that may contain the zh-yue or yue tags (and possibly
>> other zh-cmn, zh-hakka, zh, zh-HK…). This could be something a typesetter
>> wants to do to use a particular font and set of line breaking rules for any
>> chunk of Chinese (in the broad sense) text.
>> >
>> > If we canonicalize to extlang form:
>> >  zh -> zh
>> >  zh-yue -> zh-yue
>> >  yue -> zh-yue
>> > Therefore, the zh range will match both the documents that contained
>> zh-yue or yue. This is what I want.
>> >
>> > If we canonicalize away from extlang form:
>> >  zh -> zh
>> >  zh-yue -> yue
>> >  yue -> yue
>> > Therefore, the zh range will match neither documents that contained
>> zh-yue nor yue. This is not what I want, and is worse than not
>> canonicalizing at all.
>> >
>> > So it seems to me that no, we cannot canonicalize away from the extlang
>> form and get the same results.
>> >
>> > If the extended filtering operation did something smart with
>> macrolanguages, then I wouldn't need canonicalization at all, but it
>> doesn't, so I feel I need to canonicalize, and as described above, only
>> canonicalization to extlang actually seems to help.
>> >
>> > Am I missing something?
>> >
>> > —Florian
>>
>> Sorry for not including that in the previous message, but to give another
>> example, if I want to use the no range to match any of: no, no-bok, no-nyn,
>> nb, or nn, canonicalization to extlang form works, and canonicalization
>> away from it doesn't.
>>
>> —Florian
>
>
>