Re: [Icar] an early review experiment

These questions are discussed in detail in
draft-carpenter-solution-sirs-01.txt
but I have done some cut and paste below.

Mark Allman wrote:

> 
>   + At what point should early review happen?  Pre-WG document?  WG -00
>     document?  Pre-WGLC?  All of the above?  Other?

      It is not expected
      that every version of an Internet Draft should be
      submitted for SIR review. However, it is advisable to
      request reviews at the very beginning (to check for
      fundamental issues), as major technical issues are
      resolved, and again just before the document is
      submitted for IESG approval (or to the RFC Editor).
      Thus three SIR review cycles per document may be
      considered the minimum. The document author should
      ensure that the SIRs reviewing a document understand
      what stage in its life cycle it has reached, so that
      they can review it appropriately.

      Both Working Groups and individual submitters should
      realise that carding should start early (to detect and
      hopefully fix fundamental problems) and be repeated as
      often as needed (to avoid submitting inadequate
      documents to the IESG).
> 
>   + Who is responsible for finding reviewers for a document?

      2.2.1.    Working Group documents

      For documents being processed by a Working Group (WG),
      the WG solicits the assistance of SIRs.

      [which to my mind means the WG Chairs or Secretary]
      ...
      2.2.2.    Individual submissions

      For individual submissions, the document author(s)
      should solicit SIR reviews, according to the same
      principles applied to Working Group documents,

> 
>   + From what pool are the reviewers drawn?

      An initial target of 100 people is suggested. It is
      recognized that this may be insufficient, so the number
      should be revised in the light of experience.

      Q.   Is this anything like the right number?
      A.   Assume we need 9 reviews per year (3 review cycles,
           3 reviewers each time) for each of 200 RFCs. That gives each
           SIR one review every 20 days, approximately. This seems
           high, and suggests that the final target should be 200
           people.

      The goal is to identify a team of people with adequate
      experience contributing to IETF technical work, who are
      likely to be trusted as a group to be both technically
      expert and unbiased.

> 
>   + Precisely, what is produced?

There is a whole section (3.3) about this in the SIRs draft,
from which this is a very small extract:

      SIRs should review for all kinds of problems, from
      basic architectural or security issues, Internet-wide
      impact, technical nits, problems of form and format
      (such as IANA Considerations or incorrect references),
      and editorial issues. As a draft progresses from its
      initial, "-00" version towards one that is ready for
      submission, successive SIR reviews should progress from
      the general architectural level to the editorial level.

> 
>   + What does a reviewer do with a review?

3.1. Reviewing in Public

      ... All reviews will be published on the web
      site, with adequate tooling (linked to
      the ID Tracker) so that SIRs can publish reviews
      without adminstrative overhead, every review can be
      readily found, and all reviews will be automatically
      archived. In fact, a WG or document author in need of
      reviews should be able to request them through the web
      site
> 
>   + What other things need nailed down before we can think about an
>     experiment?

 From the SIRs experience, do not even think about starting until
you have a good web site up & running where (a) all reviewers can be
registered (b) authors and WG Chairs can request reviews and
(c) reviewers can post, and the public can read, their reviews.

       Brian

_______________________________________________
Icar mailing list
Icar@ietf.org
https://www1.ietf.org/mailman/listinfo/icar