Re: [EAI] General issues and strategy (was: Re: Content Issues [ was: Internationalized Email Internet Draft])

[<[e-mail]>]

John,

--On Friday, October 14, 2016 13:30 +0000
[e-mail] wrote:

>> John / Tony,
>> I am going to split your comments into separate threads so
>> that I can keep track of each.   
>>...
>> Yes.  I see your point.   Let me say first the basic thing
>> that we are trying to do is to discuss the holistic user
>> experience of internationalized emails from an operational
>> point of view.   In so doing, the co-mingling happened.  We
>> could do a second draft for content issues or change the
>> abstract of this one to better state what our real goal is.
>> Secondly, as you guys know well, there are lots of other
>> issues with IDN, browser support, etc.   What we were
>> actually hoping is that we could have a forum (perhaps like
>> DNSOps or v6Ops) where we could come together to define and
>> discuss such problems, move towards best practices (or work
> arounds! Not that I like that, but it happens.)   Because we
>> have not even started on problems that we see such as search
>> algorithm ranking of IDNs and so on.   We were hoping that
>> others would step up to author such other drafts.

>A few very general comments about context.  This might
>anticipate some future note from you, so I hope I can get it off
>quickly...

>(1) I trust you are aware of the ICANN=-sponsored "Universal
>Acceptance" effort, which is covering some of this same ground.
>Personally, I believe they are seriously out in the weeds, doing
>a good deal of work on assumptions about the DNS and the EAI
>work that are just not valid for a number of technical reasons,
>but it would be unfortunate to waste whatever cycles are
>associated with duplicative efforts.  As far as I know, Don
>Holland is still lead; I think he is on the EAI list but am not
>sure.

Yes.  I am aware of this & want to get Don Holland involved.  I somehow had the idea he would be in Seoul but maybe I am wrong.  I think I will try to have a private conversation with him and Harish ahead of time.

>(2) I trust you are also aware that there is an IAB I18N>program.  It has been, again IMO, fairly unhealthy in the last
>couple of years, largely because the available cycles of the
>most active and expert people have disappeared into the IANA
>Transition activity.  If you want a forum or workshop, they
>might be a good place to start.  Ted Hardie is lead; I have no
>idea whether he is on the EAI list.
I had a very brief conversation with Ted Hardie over email.  I will contact him again.

>(3) Despite the obvious interest and importance of this issue, I
>have observed very little actual energy in the IETF for dealing
>with it or i18n issues more generally.  By the time the EAI WG
>got its documents out, there was little interest in specific
>work beyond "we just want this to work".  By the time the PRECIS
>WG got its RFCs out, there was little general interest in
>careful review of documents (resulting in a need to start work
>on revisions almost immediately thereafter).  AFAICT, the
>strongest and most general sentiment in the WG was "just tell me
>what to do so I don't need to understand or think about this".
>That is understandable, but not a good foundation for getting
>work done.  The community has been completely unable to engage
>with the "non-decomposing character" issue that has paralyzed
>some important IDNA applications for a couple of years now (IMO,
>the LUCID BOF, supposedly addressed to that issue, did a good
>job of illustrating the more fundamental problems with IETF
>engagement, including a lengthy discussion of just how to spell
>Zürich in English (or, for that matter, in various versions of
>German).  

>(4) Even among those who were active in the EAI work, there has
>been, AFAICT, silence so far except from Tony and myself.  I
>know that some are off chasing general equivalence of DNS names.
>I believe that effort shows a fundamental lack of understanding
>of the properties of the relevant data structures but it seems
>to be getting agenda time in Seoul (see "DNSBUNDLED" on Jari's
>"New Work in Seoul" BOF list).


Sure.  I understand your frustration.  
Harish and I have been in touch with the community in Africa and Latin America as well, as of course, India.   If you will, since this is more of a problem for the non-English speaking world, then those of us who actually speak languages other than English should step up to the plate and be responsible for doing work.  One of the representatives for one of the communities has told me that they may fund someone to come to Seoul just to work on this project.
We have also started a conversation with some of the OS / browser manufacturers - the Mozillas and Microsofts of the world.
I want to get a core group of maybe 5 - 10 people from various communities and then people from the OS / browser world.  It will take some time to get critical mass but as you say, the issue is very critical.  That is definitely being seen at least in the non-Western parts of the world.  (Sorry, I don't know if I am being somehow politically incorrect in phrasing this!)
I hope that you and Tony can guide us.  It will take us some time to come up to speed.  It has definitely helped to do a proof of concept implementation.  Harish and I conference call every week sometimes for two hours to discuss these issues.   We have learned a lot in the past two months.  Mostly based on problems that I have had!
We appreciate very much your and Tony's efforts to get the WG and documents to this stage.  But, now we need to work on implementation and migration issues.   I hope you will be available to guide us.  I am not thinking that we will expect you to be responsible for doing all the work.
I was born in India.  Many, many of the people in India do not (and never will) speak English.   Two of the happiest years of my life were living at the edge of the bush in French speaking West Africa.  Again, English was not widely spoken. 
But, English is the de-facto language of the Internet.  The next billion people on the Internet need access in local language.   I think that at the IETF one thing that we can do is quietly and thoughtfully concentrate and work out the technical issues.   That is our strength.   It will take us some time to form a core group.   My hope is that by IETF99, we will have enough people to do something interesting.

>(5) You should also be sure you understand that many of the
>issues with non-ASCII identifiers, especially identifiers 

Yes.   Are there any RFCs or papers that you might point me to?

>(5) If you want rapid deployment of fully-internationalized
>email messages, it is important that you understand that there
>were alternate hypotheses for how to get both the IDN and that
>work done.  For IDNs, we could have changed the DNS and server
>matching algorithms (which might have permitted approximate
>matching and made a number of issues with comparison and
>normalization much easier) or we could have adopted an "above
>DNS" model which would have also helped with the matching and
>search issues. However, there was a great deal of pressure, some
>of it emotional and paralleling the comments about "English" in
>your draft, for "fast" and "in the DNS", and that got us the
>mappings, restrictions, and trick encoding of IDNA.  Every bit
>of new evidence or pressure for synonymous labels and matching
>that is predictable given local language culture is an argument
>that we got it wrong and should have gone for one of the other
>approaches, probably a member of the "Above DNS" family.  For
>EAI, there was a similar choice.  We could have stuck with
>all-ASCII address local parts and focused on better use of
>encoded names (e.g., by making it clear that not copying a name
>phrase into a reply or to or from an address book was a very bad
>practice, encouraging address book lookups by the name phrase or
>equivalent as well as the address, etc., thereby saving a good
>many transition and interoperability problems.  I note that was
>the solution adopted by the ITU, an organization with far more
>representation and influence from countries and people who speak
>English as a second language if at all, for a broad range of
>I19n issues.  Instead, there was a strong commitment in the WG
>and those who pushed for it to have all-non-ASCII mailbox
>addresses and to phase out the encoded word approach.  At least
>a significant fraction of the WG understood the likely (or
>certain) costs of that decision including slow deployment and
>interoperability issues that would likely limit EAI use to
>"within community" communications.  If, at any point, you want
>to argue for discarding SMTPUTF8 and returning to an encoded
>word model, there are less difficult (but also less culturally
>and aesthetically attractive) ways to move forward.


This is very interesting.  Let me think it over and we may want to discuss in our core group.   We also need the people from DNS involved.  Let me talk to a few people over there & pick their brains.

best,
  john


>(BTW, as a further illustration of the "too few people with
>spare cycles" problem, I'm writing this note rather than working
>on the current round of PRECIS revision drafts (and they are, in
>turn, holding up work in URNBIS) and am copying Peter, who I
>don't think is on the EAI list either, so he knows what is
>happening.)
John, thank you for everything you have done & for all your help with this.
Nalini