Re: Late review management (Re: [Icar] independence of reviews; variability)

David Meyer <dmm@1-4-5.net> Wed, 10 March 2004 23:07 UTC

Received: from optimus.ietf.org (optimus.ietf.org [132.151.1.19]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA12255 for <icar-archive@odin.ietf.org>; Wed, 10 Mar 2004 18:07:24 -0500 (EST)
Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1Cmv-0005Bl-Tf for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 18:06:57 -0500
Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id i2AN6v3S019934 for icar-archive@odin.ietf.org; Wed, 10 Mar 2004 18:06:57 -0500
Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1Cmv-0005BP-NV for icar-web-archive@optimus.ietf.org; Wed, 10 Mar 2004 18:06:57 -0500
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA12209 for <icar-web-archive@ietf.org>; Wed, 10 Mar 2004 18:06:54 -0500 (EST)
Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1Cmt-0006de-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 18:06:55 -0500
Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1Clx-0006VT-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 18:05:58 -0500
Received: from optimus.ietf.org ([132.151.1.19]) by ietf-mx with esmtp (Exim 4.12) id 1B1Cl1-0006Nb-00 for icar-web-archive@ietf.org; Wed, 10 Mar 2004 18:04:59 -0500
Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1Cl3-0004g9-Cs; Wed, 10 Mar 2004 18:05:01 -0500
Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1B1Ck5-0004Rj-NH for icar@optimus.ietf.org; Wed, 10 Mar 2004 18:04:01 -0500
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id SAA11936 for <icar@ietf.org>; Wed, 10 Mar 2004 18:03:58 -0500 (EST)
Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1B1Ck3-0006Es-00 for icar@ietf.org; Wed, 10 Mar 2004 18:03:59 -0500
Received: from exim by ietf-mx with spam-scanned (Exim 4.12) id 1B1CjA-00066a-00 for icar@ietf.org; Wed, 10 Mar 2004 18:03:05 -0500
Received: from m106.maoz.com ([205.167.76.9]) by ietf-mx with esmtp (Exim 4.12) id 1B1CiF-0005p3-00 for icar@ietf.org; Wed, 10 Mar 2004 18:02:08 -0500
Received: from m106.maoz.com (localhost.localdomain [127.0.0.1]) by m106.maoz.com (8.12.11/8.12.11) with ESMTP id i2AN1afh018285; Wed, 10 Mar 2004 15:01:36 -0800
Received: (from dmm@localhost) by m106.maoz.com (8.12.11/8.12.10/Submit) id i2AN1apN018284; Wed, 10 Mar 2004 15:01:36 -0800
X-Authentication-Warning: m106.maoz.com: dmm set sender to dmm@1-4-5.net using -f
Date: Wed, 10 Mar 2004 15:01:36 -0800
From: David Meyer <dmm@1-4-5.net>
To: Margaret Wasserman <margaret@thingmagic.com>
Cc: Dave Crocker <dhc@dcrocker.net>, Harald Tveit Alvestrand <harald@alvestrand.no>, Spencer Dawkins <spencer@mcsr-labs.org>, icar@ietf.org
Subject: Re: Late review management (Re: [Icar] independence of reviews; variability)
Message-ID: <20040310230136.GA18259@1-4-5.net>
References: <1221060422.20040308164330@brandenburg.com> <035201c40588$c86cc840$0400a8c0@DFNJGL21> <227129254.1078828209@localhost> <161118984.20040310131612@brandenburg.com> <p0602044ebc75384bb9a3@[192.168.2.2]> <20040310225839.GA17999@1-4-5.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20040310225839.GA17999@1-4-5.net>
User-Agent: Mutt/1.4.1i
X-public-key: http://www.1-4-5.net/~dmm/public-key.asc
X-philosophy: "I just had to let it go" -- John Lennon
Sender: icar-admin@ietf.org
Errors-To: icar-admin@ietf.org
X-BeenThere: icar@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/icar>, <mailto:icar-request@ietf.org?subject=unsubscribe>
List-Id: Improved Cross-Area Review <icar.ietf.org>
List-Post: <mailto:icar@ietf.org>
List-Help: <mailto:icar-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/icar>, <mailto:icar-request@ietf.org?subject=subscribe>
X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on ietf-mx.ietf.org
X-Spam-Status: No, hits=0.0 required=5.0 tests=AWL autolearn=no version=2.60

On Wed, Mar 10, 2004 at 02:58:39PM -0800, David Meyer wrote:
>> 	Margaret,
>> 
>> >> >Once again, diversity of perspectives is our friend... that is, if we
>> >> >believe in community rough consensus, rather than hierarchical assertion
>> >> >of authority.
>> >> 
>> >> This touches on a thought I've had off-and-on for the past couple of 
>> >> years...
>> >> 
>> >> The current IETF structures, policies and processes are based on some 
>> >> fundamental assumptions two of which are (1) that making decisions by 
>> >> rough consensus, reached through an open process will achieve good 
>> >> results, and (2) we can select leaders (ADs, WG chairs, etc.) and 
>> >> trust them to fairly solicit, judge and act upon rough community 
>> >> consensus.  Sometimes, though, we seem to question those assumption, 
>> >> and we end up adding structures, policies or processes that are 
>> >> intended to protect us from the results of our own consensus-driven 
>> >> decisions.  I think that is (and has been) a mistake.
>> 
>> 	I couldn't really parse the last sentence in this
>> 	paragraph, i.e., what do you think is the mistake,
>> 	questioning the assumptions or adding structures,
>> 	policies, or processes? 
>> 	
>> 	That being said, honestly questioning one's assumptions
>> 	with an eye toward improvement is never wrong. However,
>> 	the latter (adding structure, etc) clearly can. So maybe

s/clearly can./clearly can be.

Sorry about that.

Dave

>> 	its not really so much about the questioning (that much
>> 	is healthy for our or any organization); maybe it is more
>> 	about what we do in few those cases (if any) that we
>> 	might find in which our common assumptions don't hold. In
>> 	the case of the IETF, it is pretty clear that there are
>> 	at most a few cases in which our consensus based approach
>> 	is less efficient than we might like; it clearly works
>> 	well in the vast majority of cases (nuff said on that
>> 	one; the record speaks for itself).    
>> 
>> >> While there is no decision making process (hierarchical, democratic, 
>> >> I just get to decide...) that achieves good results all of the time, 
>> >> the consensus-driven process has worked well for the IETF over the 
>> >> years.  By failing to trust it, we don't actually move to another 
>> >> effective decision making process, we just break the one that we have.
>> 
>> 	Maybe there is be something more (less?) granular here,
>> 	like trusting the "Rough consensus and running code"
>> 	isn't a binary thing? That is, it works extremely well
>> 	for us most of the time, however, we do need to be
>> 	vigilant and watch out for those few instances in which
>> 	it doesn't. So maybe it is less about trust, and possibly
>> 	more about trying to make sure those processes that have
>> 	been so successful for us continue to serve us well. That
>> 	goal would seem (minimally) to require continual
>> 	reevaluation as we, our technology, and our industry
>> 	evolve. At least that is how I see this issue.      
>> 	
>> >> I think that this same line of thinking applies to review...  We 
>> >> should put into place the mechanisms, tools, training, etc. to 
>> >> improve the community's capacity to provide quality review.  Perhaps 
>> >> we should provide mechanisms that help WGs' find and recruit 
>> >> reviewers.  We could even develop some guidelines about what type and 
>> >> quantity of review makes sense at each level.  But, ultimately, I 
>> >> think that we should trust that our WGs (and our WG chairs, document 
>> >> editors, etc.) actually _want_ to  produce good quality, 
>> >> well-reviewed work.  We can improve their ability to do that by 
>> >> giving them better tools, but we won't achieve anything by trying to 
>> >> enforce quality through "better" rules.
>> 
>> 	Completely agree. 
>> 
>> >> If we decide, as a community, that we can no longer trust the 
>> >> consensus process and/or the motivations of our ADs, WG chairs or 
>> >> document editors, we have a _much_ bigger problem than can be solved 
>> >> by changing our review processes.
>> 
>> 	Again, completely agree.
>> 
>> 	Dave

_______________________________________________
Icar mailing list
Icar@ietf.org
https://www1.ietf.org/mailman/listinfo/icar