Re: [Icar] an early review experiment

Allison Mankin <mankin@psg.com> Wed, 19 May 2004 22:00 UTC

Received: from optimus.ietf.org (iesg.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA22697 for <icar-archive@odin.ietf.org>; Wed, 19 May 2004 18:00:07 -0400 (EDT)
Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQZ2N-00022t-GI for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:43 -0400
Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i4JLthPr007858 for icar-archive@odin.ietf.org; Wed, 19 May 2004 17:55:43 -0400
Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYnq-0005Gw-DZ for icar-web-archive@optimus.ietf.org; Wed, 19 May 2004 17:40:42 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id RAA20605 for <icar-web-archive@ietf.org>; Wed, 19 May 2004 17:40:38 -0400 (EDT)
Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQYnn-0004wW-Tp for icar-web-archive@ietf.org; Wed, 19 May 2004 17:40:40 -0400
Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQYlh-0004TE-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:38:30 -0400
Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1BQYhc-0003il-00 for icar-web-archive@ietf.org; Wed, 19 May 2004 17:34:16 -0400
Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQYbd-0007Zu-1l; Wed, 19 May 2004 17:28:05 -0400
Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1BQXje-0001su-Ee for icar@optimus.ietf.org; Wed, 19 May 2004 16:32:18 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA12400 for <icar@ietf.org>; Wed, 19 May 2004 16:32:15 -0400 (EDT)
Received: from ietf-mx.ietf.org ([132.151.6.1] helo=ietf-mx) by ietf-mx with esmtp (Exim 4.32) id 1BQXjc-0000Tx-Bw for icar@ietf.org; Wed, 19 May 2004 16:32:16 -0400
Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1BQXgC-00005i-00 for icar@ietf.org; Wed, 19 May 2004 16:28:45 -0400
Received: from psg.com ([147.28.0.62] ident=mailnull) by ietf-mx with esmtp (Exim 4.12) id 1BQXer-0007gP-00 for icar@ietf.org; Wed, 19 May 2004 16:27:21 -0400
Received: from localhost ([127.0.0.1] helo=psg.com) by psg.com with esmtp (Exim 4.30; FreeBSD) id 1BQXep-000N45-U4; Wed, 19 May 2004 20:27:19 +0000
To: Mark Allman <mallman@icir.org>
cc: icar@ietf.org
Subject: Re: [Icar] an early review experiment
In-Reply-To: Message from Mark Allman <mallman@icir.org> of "Wed, 19 May 2004 09:07:10 EDT." <20040519130710.F21BA77AA5C@guns.icir.org>
Date: Wed, 19 May 2004 13:27:19 -0700
From: Allison Mankin <mankin@psg.com>
Message-Id: <E1BQXer-0007gP-00@ietf-mx>
Sender: icar-admin@ietf.org
Errors-To: icar-admin@ietf.org
X-BeenThere: icar@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/icar>, <mailto:icar-request@ietf.org?subject=unsubscribe>
List-Id: Improved Cross-Area Review <icar.ietf.org>
List-Post: <mailto:icar@ietf.org>
List-Help: <mailto:icar-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/icar>, <mailto:icar-request@ietf.org?subject=subscribe>
X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org
X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60

Hi, Mark,

Views on both of these:

> 
>   + What does a reviewer do with a review?

I think reviews which are part of the ICAR process need
to be public.  I agree with Melinda's reasoning.  In addition, the
working group can get better at cross-area thinking, which can happen
best if the review's in the open.
> 
>   + How binding are early, cross area review comments on authors /
>     editors / WGs?
> 

>  + one end of the spectrum would be that ICAR-type comments must be
>    addressed before the document moves forward
>
>  + the other end of the space seems to me to be that the ICAR-type
>    reviews are just like any other review that comes in from a
>    community member (i.e., no special status)

I'm in the middle, for now.  You started out asking for this to be a
discussion of an experiment.  After we experiment, my position could
change.

I'd like to see all reviews result in some kind of response about how
comments are addressed.  ICAR-type comments should require* that this
response be prepared and sent to the mailing list.  But if the
document does not change at all in response to the review, that is OK:
the document should not block (that is, the Chair's, WG's and AD's
discretion with respect to the document should not change).

The ICAR review still has clout: from the enforced response, the Chair
should lead a discussion, and the result should be a serious working
group consideration of the review.

In my experience of quite a few AD reviewss (and those of AD's
directorates), there's a steep learning curve for cross-area review,
and also a difficult line between when something is actually a working
group consensus matter rather than a cross-area matter.  My experience
is that the IESG learns how best to cross-area review from reviewing
each others' reviews and from reviewing hundreds of documents in all the
areas.  There are certainlly other ways to get there, but we have no
handle on this now.

Bottom line: I think it's important to see how well the IETF can
produce consistent cross-area reviewers, which includes developing the
shared and concrete view of what the cross-area field is.  For this
experiment, I'm against blocking documents

Allison.




_______________________________________________
Icar mailing list
Icar@ietf.org
https://www1.ietf.org/mailman/listinfo/icar