Re: [Ltru] Mail regarding draft-ietf-ltru-4646bis and draft-ietf-ltru-matching

[Florian Rivoal <[e-mail]>]

> On Aug 29, 2019, at 22:44, John Cowan <[e-mail]> wrote:
> 
> 
> 
> On Thu, Aug 29, 2019 at 5:00 AM Florian Rivoal <[e-mail]> wrote:
> 
> In case that influences what you have to say, note that what I intend to do is not to store the canonicalized-to-extlang form anywhere. It would only be for internal processing: when performing an extended filtering operation, where it is unknown whether the ranges and tags are in extlang form or not, canonicalize both to extlang form do the extended filtering operation on that.
> 
> In that case you can equally canonicalize away from the extlang form as toward it.  I recommend that.

Can you?

Let's say you want to match (using extended filtering) the zh range against documents that may contain the zh-yue or yue tags (and possibly other zh-cmn, zh-hakka, zh, zh-HK…). This could be something a typesetter wants to do to use a particular font and set of line breaking rules for any chunk of Chinese (in the broad sense) text.

If we canonicalize to extlang form: 
  zh -> zh
  zh-yue -> zh-yue
  yue -> zh-yue
Therefore, the zh range will match both the documents that contained zh-yue or yue. This is what I want.

If we canonicalize away from extlang form: 
  zh -> zh
  zh-yue -> yue
  yue -> yue
Therefore, the zh range will match neither documents that contained zh-yue nor yue. This is not what I want, and is worse than not canonicalizing at all.

So it seems to me that no, we cannot canonicalize away from the extlang form and get the same results.

If the extended filtering operation did something smart with macrolanguages, then I wouldn't need canonicalization at all, but it doesn't, so I feel I need to canonicalize, and as described above, only canonicalization to extlang actually seems to help.

Am I missing something?

—Florian