[EAI] General issues and strategy (was: Re: Content Issues [ was: Internationalized Email Internet Draft])

[John C Klensin <[e-mail]>]

Nalini,

Given your decision to go for separate threads, which I think is
a good one, some general comments:

--On Friday, October 14, 2016 13:30 +0000
[e-mail] wrote:

> John / Tony,
> I am going to split your comments into separate threads so
> that I can keep track of each.   
>...
> Yes.  I see your point.   Let me say first the basic thing
> that we are trying to do is to discuss the holistic user
> experience of internationalized emails from an operational
> point of view.   In so doing, the co-mingling happened.  We
> could do a second draft for content issues or change the
> abstract of this one to better state what our real goal is.
> Secondly, as you guys know well, there are lots of other
> issues with IDN, browser support, etc.   What we were
> actually hoping is that we could have a forum (perhaps like
> DNSOps or v6Ops) where we could come together to define and
> discuss such problems, move towards best practices (or work
> arounds! Not that I like that, but it happens.)   Because we
> have not even started on problems that we see such as search
> algorithm ranking of IDNs and so on.   We were hoping that
> others would step up to author such other drafts.

A few very general comments about context.  This might
anticipate some future note from you, so I hope I can get it off
quickly...

(1) I trust you are aware of the ICANN=-sponsored "Universal
Acceptance" effort, which is covering some of this same ground.
Personally, I believe they are seriously out in the weeds, doing
a good deal of work on assumptions about the DNS and the EAI
work that are just not valid for a number of technical reasons,
but it would be unfortunate to waste whatever cycles are
associated with duplicative efforts.  As far as I know, Don
Holland is still lead; I think he is on the EAI list but am not
sure.

(2) I trust you are also aware that there is an IAB I18N
program.  It has been, again IMO, fairly unhealthy in the last
couple of years, largely because the available cycles of the
most active and expert people have disappeared into the IANA
Transition activity.  If you want a forum or workshop, they
might be a good place to start.  Ted Hardie is lead; I have no
idea whether he is on the EAI list.

(3) Despite the obvious interest and importance of this issue, I
have observed very little actual energy in the IETF for dealing
with it or i18n issues more generally.  By the time the EAI WG
got its documents out, there was little interest in specific
work beyond "we just want this to work".  By the time the PRECIS
WG got its RFCs out, there was little general interest in
careful review of documents (resulting in a need to start work
on revisions almost immediately thereafter).  AFAICT, the
strongest and most general sentiment in the WG was "just tell me
what to do so I don't need to understand or think about this".
That is understandable, but not a good foundation for getting
work done.  The community has been completely unable to engage
with the "non-decomposing character" issue that has paralyzed
some important IDNA applications for a couple of years now (IMO,
the LUCID BOF, supposedly addressed to that issue, did a good
job of illustrating the more fundamental problems with IETF
engagement, including a lengthy discussion of just how to spell
Zürich in English (or, for that matter, in various versions of
German).  

(4) Even among those who were active in the EAI work, there has
been, AFAICT, silence so far except from Tony and myself.  I
know that some are off chasing general equivalence of DNS names.
I believe that effort shows a fundamental lack of understanding
of the properties of the relevant data structures but it seems
to be getting agenda time in Seoul (see "DNSBUNDLED" on Jari's
"New Work in Seoul" BOF list).

(5) You should also be sure you understand that many of the
issues with non-ASCII identifiers, especially identifiers 

(5) If you want rapid deployment of fully-internationalized
email messages, it is important that you understand that there
were alternate hypotheses for how to get both the IDN and that
work done.   For IDNs, we could have changed the DNS and server
matching algorithms (which might have permitted approximate
matching and made a number of issues with comparison and
normalization much easier) or we could have adopted an "above
DNS" model which would have also helped with the matching and
search issues. However, there was a great deal of pressure, some
of it emotional and paralleling the comments about "English" in
your draft, for "fast" and "in the DNS", and that got us the
mappings, restrictions, and trick encoding of IDNA.   Every bit
of new evidence or pressure for synonymous labels and matching
that is predictable given local language culture is an argument
that we got it wrong and should have gone for one of the other
approaches, probably a member of the "Above DNS" family.   For
EAI, there was a similar choice.  We could have stuck with
all-ASCII address local parts and focused on better use of
encoded names (e.g., by making it clear that not copying a name
phrase into a reply or to or from an address book was a very bad
practice, encouraging address book lookups by the name phrase or
equivalent as well as the address, etc., thereby saving a good
many transition and interoperability problems.  I note that was
the solution adopted by the ITU, an organization with far more
representation and influence from countries and people who speak
English as a second language if at all, for a broad range of
I19n issues.   Instead, there was a strong commitment in the WG
and those who pushed for it to have all-non-ASCII mailbox
addresses and to phase out the encoded word approach.  At least
a significant fraction of the WG understood the likely (or
certain) costs of that decision including slow deployment and
interoperability issues that would likely limit EAI use to
"within community" communications.  If, at any point, you want
to argue for discarding SMTPUTF8 and returning to an encoded
word model, there are less difficult (but also less culturally
and aesthetically attractive) ways to move forward.

best,
   john


(BTW, as a further illustration of the "too few people with
spare cycles" problem, I'm writing this note rather than working
on the current round of PRECIS revision drafts (and they are, in
turn, holding up work in URNBIS) and am copying Peter, who I
don't think is on the EAI list either, so he knows what is
happening.)