Re: Quality of Directorate reviews

Keith Moore <moore@network-heretics.com> Sat, 16 November 2019 14:36 UTC

Return-Path: <moore@network-heretics.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 95ED1120164 for <ietf@ietfa.amsl.com>; Sat, 16 Nov 2019 06:36:33 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level:
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_NONE=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=messagingengine.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9TUZUmteGEC0 for <ietf@ietfa.amsl.com>; Sat, 16 Nov 2019 06:36:32 -0800 (PST)
Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F2EFF12012D for <ietf@ietf.org>; Sat, 16 Nov 2019 06:36:19 -0800 (PST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 41B2221F9F; Sat, 16 Nov 2019 09:25:58 -0500 (EST)
Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Sat, 16 Nov 2019 09:25:58 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=Vzs29KMQDktPDonM4YACJkps1sTq5dXcPoNmSHsiD yg=; b=evCqmdZHnWidw1q5K77+vKoa6HzO3/Odl4drMB1r+mcaaFUtA+g/hY/cc 6ZI8c6X2yfrYdiM3iNwR8mjILmyJYQ3crBIp5S72Y+QvIXlSiHfnc7dQqRer7ZGX YpsYnOnAQhNHU6+7AA0oq5rCLK4Ni8ISuYmG1H9WQcX6XAUlWOYLeWe5eKjxRYem Bu9HbKuxw11UIiikDDN65sSYHFaaoQEDo+Yqpz/tlxOqdYhWIeNtXyRTUcMavLzq kmzDyzrtMrCQQr165RMzxLgdZNRdx2P4wwPpQMAnTm3e79RQFCPHRLrXGqt118wU nFXTTnudX1w0fYcy0UYpQkbrMXwlw==
X-ME-Sender: <xms:9QbQXYYNp_oMrM3R6lO5zSuFGudcYbbblQB127NzX0mgzbyvlhrK_g>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedrudefledggeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhepuffvfhfhkffffgggjggtgfesthekredttdefjeenucfhrhhomhepmfgvihht hhcuofhoohhrvgcuoehmohhorhgvsehnvghtfihorhhkqdhhvghrvghtihgtshdrtghomh eqnecukfhppedutdekrddvvddurddukedtrdduheenucfrrghrrghmpehmrghilhhfrhho mhepmhhoohhrvgesnhgvthifohhrkhdqhhgvrhgvthhitghsrdgtohhmnecuvehluhhsth gvrhfuihiivgeptd
X-ME-Proxy: <xmx:9QbQXfkTymn8pLsWdVcZNm4yXoz0chA9k49m9mLE6HLLThi_SZo64Q> <xmx:9QbQXV3tJmvbgeuePzARlDCJvfyeRDfLQdR2q1y_TDH9KEKnrG54Ig> <xmx:9QbQXRShlOwn0w55uaSWDTy5qdciw9pOXcJBsjCDKYZ56kRgF_6Crw> <xmx:9gbQXYLoZkOyenImV08_Sf6erTBqrq_sJff0E00PxniyxulS0rtFLg>
Received: from [192.168.1.97] (108-221-180-15.lightspeed.knvltn.sbcglobal.net [108.221.180.15]) by mail.messagingengine.com (Postfix) with ESMTPA id E8A14306005C; Sat, 16 Nov 2019 09:25:56 -0500 (EST)
Subject: Re: Quality of Directorate reviews
To: Benjamin Kaduk <kaduk@mit.edu>, Michael Richardson <mcr+ietf@sandelman.ca>
Cc: ietf@ietf.org
References: <MN2PR11MB43669E4CEF13CDA51A764F9AB5790@MN2PR11MB4366.namprd11.prod.outlook.com> <20191.1573054128@localhost> <15BCDF05-FB13-45D2-A5DF-70618EBA1A5A@gmail.com> <9182.1573147520@localhost> <A3493C65-7F8A-407D-A9F4-FF36296C0920@gmail.com> <CAMm+LwiP4Ypuyh2xsd8qBjUfwuNzOYOfp3OrDnPmU-YwMH2pMw@mail.gmail.com> <02eb79d1-1830-5830-ed95-b743f601a8de@network-heretics.com> <f60f410e-1cab-368b-b981-4e85c0f6a816@sandelman.ca> <84ee7053-1dbb-bfcc-c576-c2cf115a743e@network-heretics.com> <31471.1573886540@dooku.sandelman.ca> <20191116070802.GN32847@kduck.mit.edu>
From: Keith Moore <moore@network-heretics.com>
Message-ID: <cb56955b-062f-28b1-77b6-67f80cece27c@network-heretics.com>
Date: Sat, 16 Nov 2019 09:25:55 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0
MIME-Version: 1.0
In-Reply-To: <20191116070802.GN32847@kduck.mit.edu>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/Rjxiu-aUM6nqezkE-sapdOZrJa0>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 16 Nov 2019 14:36:34 -0000

On 11/16/19 2:08 AM, Benjamin Kaduk wrote:

> My understanding is that most directorates have a secretary that does the
> assignments (secdir does, at least).  By the time an AD is looking at the
> review next to the document it might only be a few days before the telechat
> where the document is up for approval, which is not really enough time to
> get another review in without deferring the document.  Maybe we should go
> get that extra review and try to remove the stigma against deferring
> documents; I don't have a sense for how the community would feel about
> that.

IMO there's always going to be some conflict between an AD doing his/her 
job and "what the community wants", because nobody thinks his/her own 
document is lowering the quality of IETF standards. So the pressure to 
approve documents despite serious flaws, and/or to approve documents 
without delay (even though some documents really do require more time to 
review and more eyeballs to look at them), will always be there.   There 
should not be a "stigma" associated with deferring documents if there's 
any reason at all to subject them to more scrutiny.

Neither IESG nor the RSE is the source of most delays, after all; most 
delays are in the working groups.   And yeah, people compile stats on 
the IESG and the RSE to see how well they "perform" but those stats 
don't measure the most relevant things, like the quality of IETF's 
output, or the extent to which IETF standards are utilized, or the 
extent to which they're followed (this is a different thing), or 
(especially) the degree to which IETF standards meet the Internet's 
needs.   There's an unfortunate tendency these days to pay more 
attention to quantities that are easily measured, than to those that 
actually matter.

> And yes, the AD should look at the directorate review when it arrives, but
> looking only at the review and not the document being reviewed is not
> always enough to tell whether additional review would be valuable.

Entirely agree.   I don't think IESG should presume that directorate 
reviews will always be sufficient, and to do so IMO is tantamount to 
IESG abandoning its duty.

Keith