Re: [Icar] an early review experiment

On Tue, 18 May 2004, Mark Allman wrote:
> In that context, the broad questions that we'd like to discuss are:
>
>   + At what point should early review happen?  Pre-WG document?  WG -00
>     document?  Pre-WGLC?  All of the above?  Other?

Because it seems *very* difficult to kill off WG work items after they 
have been accepted, some review should happen just before accepting a 
document as a WG work item.

This should be done to kill any obvious bogosities and alert of
cross-area problems.

A second set of early reviews should happen at some point after -00 WG 
document and between/around WGLC.  Whenever the document is deemed 
suitably stable for consumption..

>   + Who is responsible for finding reviewers for a document?
>
>   + From what pool are the reviewers drawn?

These two are IMHO the most difficult questions from the practical 
point of view.

For example, as WG chair, I've been instructed to get review from 
e.g., apps area for a document.  Despite begging from the apps ADs, 
mailing numerous times on appsarea mailing list, getting a person 
signed up for review (who ultimately did nothing), I have to say it 
must be *very* clear who has the "responsibility" for producing 
reviewers/reviews for the document.

This cannot just be the WG chair or the shepherding AD in question.  
Maybe if there was an area-specific early review board, the chair(s)  
of that board would be responsible for getting the document reviewed.

As to the pool where the reviewers should come from.. that's maybe 
slightly easier.  This stuff would just need to be organized 
sufficiently well, with sufficient buy-in from the IESG, and 
appreciation by the peers.  The criteria like used for SIRS or to get 
ARTS might work here, as well as volunteers.  But the most important 
thing is that those people do actually review documents (at least 1 
per month), and there is a system to make sure the review will end up 
getting done.

>   + Precisely, what is produced?

A SIRS-like decree like "ship it, needs polish, needs a lot of work,
kill this madness" would probably be useful, but this should be 
coupled with technical comments as well.

>   + What does a reviewer do with a review?

Did you mean 'what does the reviewee (whose document one reviewed) do 
with the review?' as I don't understand why the above question is 
relevant.

Assuming so, this is again a good question.  The review should be sent
tot he document editor(s), WG chairs, and possibly the associated AD
-- and one of which should file it in an issue tracker if used.  The
reviews should also be publicly stored in a web location.

The comments should be treated with equal weight as IESG Discuss 
comments.

>   + What other things need nailed down before we can think about an
>     experiment?

It's already implicit from the above, but it would probably make sense 
to consider at more length when this kind of review should be 
initiated and by whom. I.e., whose responsibility is it to start 
the review process, and ensure that sufficient review gets done 
before going to the next step in a draft's lifetime (e.g., 
adopting it as WG item, etc.).

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

_______________________________________________
Icar mailing list
Icar@ietf.org
https://www1.ietf.org/mailman/listinfo/icar