[Icar] independence of reviews; variability

[Dave Crocker <[e-mail]>]

Folks,

At the icar meeting, last week, Harald and I had an exchange about the
independence of review effort. I was being channeled through jabber, so
I'm relying on the jabber log for the details. I think the exchange
highlights some very basic issues, both about reviewing and about IETF
culture and organization. It would be extremely helpful to pursue this
further.

Here are the relevant parts of the jabber logs. I'm hoping I have
retained all the relevant text, which was interspersed with other
threads.  I've also included some of the log from Harald's presentation,
which touches on variability of reviews.

This message is just the relevant text.  I'll post comments separately.

d/


--- (edited) JABBER LOG ---

dcrocker says: I strongly suggest that the review function be
independent of the IETF administrative management team. The reviewing
stuff is not about helping an "area" or dealing with any aspect of
working group process. It is strictly for technical commentary.

hta: every time dave says something, I have to say the opposite. if you
want to create a body with power and responsibility outside the IETF
management structure with power and responsibility, be careful what you
ask for

dcrocker says: what "power" does a review group have and what "benefit"
is there in making such a purely technical effort subject to the varied
goals of ietf management?

hta: if review is to make any sense, it has to influence the quality,
relevance, timeliness of ourput. the managemt exists to influence the
quality, relevance, timeliness of ietf output

<hartmans> Why. Of course I don't see the value of independence in the
review other than to get people who will see clarity problems.

[JJJ]: Are you saying we can already ignore reviewers?

hta: If you have reviewers who make a difference, they have power

dcrocker says: working group / iesg conflict is a fact of life. an
independent review process provides _independent_ input to such
conflict.

Next: harald alvestrand on draft-ietf-alvestrand-twolevel-00.txt (if I
got this right)

... note document proposes giving review groups ability to approve
documents [did I get this right]; that is out of scope for here

[Dave Thaler]: how is it determined which team a document is reviewed by?

hta: throw dice, except that when a group of docs have strong internal
refrerences, they should be treated by the same review team

[Dave Thaler]: concern I have with this is that not all review teams will be
equal, not just in terms of quality, but in terms of what direction
one team may say "this is a good idea" while another wants to go another
direction who has ultimate responsibility? central management important
can't "shop around" for a team that is friendly to what you want to
do.  need some way to counter that

hta: if you're going to have more than one group, you need to expect and
deal with variations and judgement. need to have someplace in the system
that can take a review team and say "you're out of line" i believe in
central authoritty (we have no kings, but we have central authority)

dcrocker says: shopping around is fine, if the reviews must be credible
on their own right, rather than by virtue of some organizational
affiliation. If a wg gets only friendly reviews, they have not done
their job.

[Dave Thaler]: if the team is chosen randomly, and there are differences between
teams, then WG may not know how to construct a document that will
pass. f it is not random, there may be forum shopping\

hta: if we have multiple teams, we may have more guidelines

greg daily(?): when doc goes to ietf last call, there can be a dynamic
selection of reviewers

<dcrocker>: differences between sets of reviews - this is goodness,
not badness. the job of a wg is to resolve differences and
considerations. if reviews do not differ, they probably are not adding
much benefit.



--
 Dave Crocker <dcrocker-at-brandenburg-dot-com>
 Brandenburg InternetWorking <www.brandenburg.com>
 Sunnyvale, CA  USA <tel:[phone]>


_______________________________________________
Icar mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/icar