Re: [I18nrp] Additional input needed for i18nRP BOF

[John C Klensin <[e-mail]>]

--On Wednesday, June 13, 2018 14:59 -0700 Adam Roach
<[e-mail]> wrote:

> On 6/6/18 21:31, John C Klensin wrote:
>...
>> The other is an actual
>> i18n spec.  So, as an exercise for those designing solutions,
>> consider draft-klensin-idna-5892upd-unicode70
>>     and
>>     draft-klensin-idna-rfc5891bis
>> 
>> and how the proposed solutions would work for them.
> 
> John --
> 
> After thinking through this issue a bit, I suspect the answer
> would look something like this:
> 
> Given the assertions that have been made about the likelihood
> of successfully creating a working group that would adopt and
> progress these documents, and given their nature, it seems
> that they would be reasonable candidates for AD sponsorship.
> In the context of the thumbnail sketch that has been building
> up on this list, I would expect that the sponsoring AD would
> work closely with the proposed i18n directorate to ensure that
> the proper level of review has taken place.

Adam,

Maybe.   Let me see if I can explain my concern.  Over the years
and depending on who is on the IESG, I think I've seen a general
pattern in the difference between individual submissions and WG
documents, at least for documents that end up being treated as
the slightest bit controversial, i.e., where the consensus among
all of those who have opinions is not clear and obvious.

I need to temporarily take a few steps back from i18n and look
at patterns of document review.

WG documents come into IETF Last Call with claims of WG
consensus, often better than just "rough".   For many of these
controversial documents, here is an uproar during IETF LC.
Again, there are exceptions, perhaps many of them, but there are
often strong assertions that the WG contained most or all of the
people in the IETF who were both expert and interested
(effectively by definition), that anyone who was an expert but
did not participate actively in the WG had given up their right
to comment, etc., etc.  Typically, the IESG does not fine
tuning, but the document ultimately goes through.  Maybe I
wasn't paying attention when it happened, but I cannot remember
a single instance in which a WG has submitted a document for
standards track approval and, after IETF Last Call and IESG
review of the document and results, the IESG goes back to the WG
and says "no to the document and it is clear that the views and
expert conclusions of your active participants are out of line
with those of the community and we are shutting you (and this
work) down until there is a plan to create well-worked-out and
better aligned documents".   Now perhaps that never occurs
because WGs always have adequate representation of all of the
positions in the community, are never dominated (even if
accidentally) by a particular narrow cluster of interests, and
so on, but I hope you agree that there are a few other
hypotheses that may be plausible in at least some cases.

For an independent submission, the pattern has sometimes been
quite different.  If there is evidence that a document is likely
to be controversial and/or viewed by the community as
problematic technically, it typically will not be AD-sponsored
and hence won't even see IETF LC.   That is probably how it
should be: I assume that, if ADs were asked how many individual
submissions they had sponsored and put into IETF LC that they
knew were defective for substantive technical reasons, the
answer would be few or zero.  There also appears to be a bias in
the system with documents generated by people with close
relationships to particular ADs and/or strong track records in
the IETF having much better success getting AD sponsorship than,
e.g., newcomers.   However, if one of those individual
submission documents gets into Last Call and there is a
significant (in volume) uproar, the document is far more likely
to be dropped or withdrawn than one that was generated and
submitted by a WG.  Again, that may very well be as it should be
in the overwhelming majority of cases -- for individual
submissions, IETF LC is the only public, group, review of the
specification and that review is useless if it doesn't filter
out bad (or insufficient) work.

It is implicit, but the above depends on the assumption that the
community and/or internal expertise in the IESG will, in
reviewing comments, be able to rather easily determine the
differences among:

(i) Legitimate and informed comment (whether one agrees with
it/them or not).
(ii) Well-intentioned comment from someone who is thoroughly
confused or that isn't relevant to the particular specification
(e.g., is about something else entirely or makes assumptions
that the IETF has already considered and reach consensus to
reject.
(iii) Trolls

Occasionally, it is hard to distinguish between (ii) and (iii)
but, while the difference may be important with regard to the
noise level and the question of whether it is worth investing
time and energy to help the person making the commenter
understand, probably neither should influence decisions made
about the specification and they rarely do.  

As an extreme example, assume there is a new spec for a TCP
Congestion Control variation.  I'd expect many comments from
people who at least mostly understand the issues, some in favor,
some opposed, and some raising questions about fine points,
relationships to other approaches, and details of documentation.
If we are ordinarily unlucky, a few comments will also show up
about how, if we only adopted TCPv7 and IPv17, we wouldn't have
any of those pesky congestion control problems.   It seems clear
that those comments would not be taken seriously enough to
increase the odds that the specification would be rejected or
returned to the author for drastic revision.

[Disclaimer: some of the comments that follow echo ones I made
in more detail and a more specific context in my note to Nico
yesterday.  Neither that note nor this one are an attack on him;
I have found some of the recent discussions with him very
helpful in improving my own understanding and appreciate his
patience with me as we tried to sort out how and where we
disagreed.  Yes, there are positions and behaviors that bear a
partial and superficial resemblance to some comments he has made
and for which I have bad words, but I don't believe those are
his positions or behaviors.]

However, when we apply whatever those patterns predict to
individual submission i18n documents, we find what I think is a
different problem.  We've heard comments made with great
conviction in BOFs, WGs, and the IETF list that reflect the
assumption that what people have learned about the writing
systems for one or two languages (possibly even using the same
script) is enough to generalize to everything else or that, if
one understands a couple of very different languages and writing
systems, everything else is enough like one or both of those
that a solution for them will take care of everything else.
Similarly, we've seen people who believe the whole problem (or
all of its interesting aspects) are about how one codes
characters without understand the many issues there much less
all of the issues coding doesn't cover) are about confusability,
typically confusability at the code point level.  The latter has
been much more common around ICANN and Internet Governance
circles than around the IETF, but that has not helped with the
noise level).  In addition, we've had arguments made against
particular aspects of, or changes to, details of, e.g., IDNA or
PRECIS, that are based on rejection  And so on.

I've save time and space by not trying to explain why many of
those complaints, proposals, and statements are incorrect even
if they have enough elements of truth in them for which examples
can be given, but the questions are about detection and
strategy.   Those types of comments are sometimes difficult, but
possible, to handle in WGs where there is a clear focus on a
chartered task and, usually, enough concentration of knowledge
to at least recognize the problem, just as a TCP-related WG
would have no trouble telling the difference between a proposal
to improve on our current congestion control techniques and
someone who claims to be able to eliminate the need for it
entirely or who method merely requires operating a backchannel
that runs above the speed of light.

I'm less sanguine about BOFs (we've seen some bad experiences)
or a directorate because the mission statements and control
policies have generally been less clear (usually in the interest
of desirable flexibility).  And I'm concerned about whether the
IESG (all of the IESG) would actually pay attention to advice
from a directorate in the presence of rather loud contradictory
advice during a Last Call.   I don't think those potential
problems are insurmountable, but I think that being aware of the
potential problems and factoring them in, would provide a much
more likely basis for success (rather than a lot of frustration
and needing to have this discussion again in a few years) then
just deciding to create a directorate and a few extra rules and
hoping everything works out.

As a closing comment, I do not consider myself an expert in this
area, except in the sub-area of having learned a lot about what
I don't know.  The latter is an important store of knowledge and
seems to be ever-increasing.   As a specific example, almost
every time I've consulted an expert on some specific aspect of
our i18n work, I come away with three things and sometimes a
fourth:

 * An answer, which is usually more complex and ambiguous
	than I had hoped for or anticipated
 * An in-depth understanding that the question I asked
	was incomplete, showed ignorance of the issues, or was
	just the wrong one and why.
 * A considerable increase in the size of  yet another
	area in which I didn't know nearly enough and still
	don't.  
 * More support for the hypothesis that, in this area,
	almost any simplistic solution is either not going to
	work or is going to need a good deal of explanation
	about what problems it isn't going to solve.

I think that, if more of us could adopt that sort of stance
toward this material, we be making big steps toward resolving
issues (and identifying those 5hat cannot be resolved) rather
than going around and around in the same circles.  Noting that
we don't need that attitude toward things that are simply
engineering problems (even hard ones with tradeoffs like
congestion control), that is just my personal opinion and as of
this week.

Thanks for listening.  And, Adam, thanks for signing off on the
BOF.

best,
    john