Late review management (Re: [Icar] independence of reviews; variability)

[[[NOTE: This note is ONLY about LATE review. Early review is a different 
kettle of fish.....]]]

Spencer:

there are two ways to view reviews, I think:

- As input to further work ("early review")
- As a check that this is good enough to be declared finished
(what I usually refer to as "late review")

In the first category, I think getting serious thought applied is more 
important than consistency - and structures need to reflect that.
In the second category, I think consistency is important - and in two 
directions, not one:

- The "floor" - that at least SOME review is carried out in ALL cases
- The "ceiling" - that product not be held to an unreasonably high 
standard, that stuff that is really nits is not declared show-stoppers, and 
that one show respect for well justified technical choices, even when one 
fundamentally disagrees with them.

(The last one is actually something most IESG members have spent time 
learning - not what issues to find, but what issues are NOT worth raising, 
because the time lost is more costly than the value add of the improvement)

Adding more structured late reviews is, in a sense, "raising the floor".
(In -twolevel, it is also intended to get away from the idea that a single 
small group has to do *all* the reviews for the IETF. That's nonscalable.)

As was raised in the ICAR meeting, if we have multiple groups reviewing 
documents, consistency (of both "floor" and "ceiling") isn't automatic, and 
we should make sure we have mechanisms where someone sees if we have 
consistency, and can take steps to increase consistency if the divergence 
becomes big enough to be harmful. So - someone has to manage it.

If management of consistency of late reviews is completely independent from 
other management structures, we set ourselves up for another "late 
surprise" problem - that the standards groups have been told to apply 
during development is not the standards they will be held to in the final 
review. So - I think that having *one* management structure for the IETF 
makes sense.

I don't particularly care what the shape of that management structure is, 
or what it is called - I care that it works.

                    Harald

--On 8. mars 2004 21:44 -0600 Spencer Dawkins <spencer@mcsr-labs.org> wrote:

> FWIW, my understanding of the current "IETF Last Call" mechanics is
> that reviewers today are, in the final analysis, "uncontrolled". A
> scathing review correctly pointing out technical flaws is always
> admissible at any point in the review cycle, all the way through
> publication by the RFC Editor, no matter who wrote it or why.
>
> If we don't think anyone except area directors reads documents any
> more, discussion about "reviewers inside/outside the management
> structure" might make sense. I had hoped that we would not be starting
> down that road.
>
> Having a Internet Review Board that is entirely independent of current
> management structures works for me. Having an Internet Review Board
> that "reports" to the IAB works for me. We may end up with Area Review
> Teams, but that is *not* the only reasonable answer.

_______________________________________________
Icar mailing list
Icar@ietf.org
https://www1.ietf.org/mailman/listinfo/icar