Re: Thought experiment [Re: Quality of Directorate reviews]

Keith Moore <moore@network-heretics.com> Fri, 08 November 2019 01:54 UTC

Return-Path: <moore@network-heretics.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 27FB1120043 for <ietf@ietfa.amsl.com>; Thu, 7 Nov 2019 17:54:27 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level:
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_NONE=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=messagingengine.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ulmCKcw2pnV9 for <ietf@ietfa.amsl.com>; Thu, 7 Nov 2019 17:54:25 -0800 (PST)
Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id BCE63120018 for <ietf@ietf.org>; Thu, 7 Nov 2019 17:54:25 -0800 (PST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 4EBBF543; Thu, 7 Nov 2019 20:54:24 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Thu, 07 Nov 2019 20:54:24 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=U5BAaNJUB1Lt9qU+aHENYwf9DLcZ15WqXwtF8YkB2 A4=; b=Q4mgeNt9DgPCubzSosLp9k+NQrsPwf71/bD+48/rZ61sanrAVRYXD23UP aTEIF2RfzCkNexzrEdcEoOnz3pw2GFm8PZ66g9WjdwTLf0blqiptVIFkpmvWd3Kk UJpGJV5dOFO7719bXRzDvRATTO/c6Z+4lUtjI1IRIpjHEw7BC94npKGdtkEpEYyn Q03JR0u5J9NF82onqvoQeYWNJqvnE9UgkJCKHU7xw2HkamYfxrsSI2O5EpUW3CrV ufl3c0z/UL2VyIhqpAfxGLFTx382IktveBC2gbdsNTY0x90vF47JDlsX8j+UGusg bZMpNe+OytsjhCWhFzYnXHG9FQ3dQ==
X-ME-Sender: <xms:z8rEXVwNgzum7-I4C_LcfPWBb9LUA42ii8A44fbqBziVtviuP_1AZQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedruddvtddgfeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefuvfhfhffkffgfgggjtgfgsehtke ertddtfeejnecuhfhrohhmpefmvghithhhucfoohhorhgvuceomhhoohhrvgesnhgvthif ohhrkhdqhhgvrhgvthhitghsrdgtohhmqeenucfkphepuddtkedrvddvuddrudektddrud ehnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmohhorhgvsehnvghtfihorhhkqdhhvghr vghtihgtshdrtghomhenucevlhhushhtvghrufhiiigvpedt
X-ME-Proxy: <xmx:z8rEXS7jUkpuCFAUfhGUKBZwbIazhN6DSgMYTfxrThv2cxfH1WAIOg> <xmx:z8rEXU7URMdVWMQb1pAA4CuccLTG3v-Bsf2RlKriw2Rlw-5upr_IfA> <xmx:z8rEXbKM0cE8NudQQk0JC7jcKHmC6uzoEZ-P67sOETxCZ-X9Yn2DyQ> <xmx:z8rEXSHQ4oEBSWEb7CrkJ4RfOCmhZ_fgXiRs0gL8oLf_GPY96bJ7mA>
Received: from [192.168.1.97] (108-221-180-15.lightspeed.knvltn.sbcglobal.net [108.221.180.15]) by mail.messagingengine.com (Postfix) with ESMTPA id 508B0306005B; Thu, 7 Nov 2019 20:54:23 -0500 (EST)
Subject: Re: Thought experiment [Re: Quality of Directorate reviews]
To: ietf@ietf.org
References: <38E47448-63B4-4A5D-8A9D-3AB890EBDDDD@akamai.com> <09886edb-4302-b309-9eaa-f016c4487128@gmail.com> <26819.1572990657@localhost> <2668fa45-7667-51a6-7cb6-4b704c7fba5a@isode.com> <2C97D18E-3DA0-4A2D-8179-6D86EB835783@gmail.com> <91686B28-9583-4A8E-AF8A-E66977B1FE13@gmail.com> <012b9437-4440-915c-f1f9-b85e1b0be768@gmail.com> <20191107014849.GC12148@localhost> <57465486-71b1-a87a-fa8c-bad7157f9025@gmail.com> <3caeb4cf-b92b-99fd-77df-7b1aef3e2979@network-heretics.com> <20191107194408.GF12148@localhost> <9eab2c4e-455e-c3cb-cdf7-9c3467ada06c@gmail.com>
From: Keith Moore <moore@network-heretics.com>
Message-ID: <4e105279-9e8b-0fbe-4e96-0a0616e5fc3c@network-heretics.com>
Date: Thu, 7 Nov 2019 20:54:20 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0
MIME-Version: 1.0
In-Reply-To: <9eab2c4e-455e-c3cb-cdf7-9c3467ada06c@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/SivjpTliC4fNa3nfPKob42Y5oio>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2019 01:54:27 -0000

On 11/7/19 8:31 PM, Brian E Carpenter wrote:

> Anyway: my new experiment would be one that the IESG could decide to
> start tomorrow. It's simply that the IESG would only ever issue one form
> of DISCUSS ballot, which would look like this:
>
> Pat Areadirector has entered the following ballot position for
> draft-ietf-somewg-somedraft-99: Discuss
> ....
> ----------------------------------------------------------------------
> DISCUSS:
> ----------------------------------------------------------------------
>
> There are still open issues from the following reviews:
> <links to reviews>
>
> In other words, the IESG simply busy-waits until all review issues
> have been resolved, rather than finding and fixing the issues
> personally.
>
> (If an AD wishes to post a review for a given draft, that would be
> a personal choice, not part of the IESG workload.)
>
> This is a procedural change, and would not prevent a substantive
> DISCUSS in unusual circumstances.

To me the problems seem like: (a) getting sufficient reviews done, and 
(b) establishing review criteria well in advance, not only for the 
reviewers but also as feed-forward to the WG and document editors.  (not 
that reviewers couldn't identify issues outside of those criteria, but 
having a somewhat-shared view of those criteria would make it easier to 
compare different reviews for example)

I keep looking for ways to identify these criteria earlier in the 
process.   Not that there would never be last-minute showstoppers - 
we're always learning new things.   But if we could reduce the number of 
late surprises that would make everything easier IMO.

Keith