Re: Quality of Directorate reviews

Keith Moore <moore@network-heretics.com> Fri, 15 November 2019 14:08 UTC

Return-Path: <moore@network-heretics.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6965212086B for <ietf@ietfa.amsl.com>; Fri, 15 Nov 2019 06:08:59 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level:
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_NONE=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=messagingengine.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HmUcJU2pMKzn for <ietf@ietfa.amsl.com>; Fri, 15 Nov 2019 06:08:57 -0800 (PST)
Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 728BE120013 for <ietf@ietf.org>; Fri, 15 Nov 2019 06:08:57 -0800 (PST)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id A42F822089; Fri, 15 Nov 2019 09:08:55 -0500 (EST)
Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Fri, 15 Nov 2019 09:08:55 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=z9jF8gzbCAOlBujEDKVVygDvr1z3d4NSvpLmlxUoR c0=; b=uTVem0fipT8yoScZyuWeyKaoKrgNSK4UWNa21K5Y1ZYjI6EotFRwbfLRY JF452WcrxYlZMgLeriVrpbWbvihX0ivWfAq70+RT70SdD7MB/3IlRKPqdbywfPcF e1Nx5Czr8BXMemr/g4cZd0QMsSN15HvlBNHi8CAHVL+tzAW9ZOSkYfV4jen2KAVy 8rYIENwfU3RDmjxRxqg5J8Psbh0QuF1bvR33e+CWlzT3xUcQU4E+Tt+DL/Jbnxtf e9chlIcjPAUgM3+cOasC4bWcllxPNUlXk6ILtEyV6LQLjZT2UXQrkSh11UBT7iD9 Vubtm2P6YLFDrAtvXEDC+fTrBApYg==
X-ME-Sender: <xms:d7HOXSf7ADCQOPbEEDmmM85B5Fs3h8IF5lIDlXNp8DM8PLrMXOG23w>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedrudefhedgiedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefuvfhfhffkffgfgggjtgfgsehtke ertddtfeejnecuhfhrohhmpefmvghithhhucfoohhorhgvuceomhhoohhrvgesnhgvthif ohhrkhdqhhgvrhgvthhitghsrdgtohhmqeenucfkphepuddtkedrvddvuddrudektddrud ehnecurfgrrhgrmhepmhgrihhlfhhrohhmpehmohhorhgvsehnvghtfihorhhkqdhhvghr vghtihgtshdrtghomhenucevlhhushhtvghrufhiiigvpedt
X-ME-Proxy: <xmx:d7HOXZ20o_mRktlsJ4ereM9Ah0nxFtzGovBU6_p5m8bgZQ-hZLeX9g> <xmx:d7HOXa6G7dR0HdxnZAbLky_8v0LuX1755fTA6XSLaqLlLXGfTVkLNQ> <xmx:d7HOXYJ4su7JTZ-wHKWJVUiUlQuVE84FGy9oTA76MzcNa-FwbHsSzA> <xmx:d7HOXfIEpxbW65UUAIFVY1qycSt6nKzp1F2qlfpVlcYc5Jcd1f4Bkg>
Received: from [192.168.1.97] (108-221-180-15.lightspeed.knvltn.sbcglobal.net [108.221.180.15]) by mail.messagingengine.com (Postfix) with ESMTPA id 7D0E68005C; Fri, 15 Nov 2019 09:08:54 -0500 (EST)
Subject: Re: Quality of Directorate reviews
To: ietf@ietf.org
References: <157279399807.13506.13363770981495597049.idtracker@ietfa.amsl.com> <0EF64763-BA25-468A-B387-91445A61D318@gmail.com> <CAJU8_nUovmFmgNiYx0ez_1f+GPdU9xGViDYWfowEEomrn0pyDw@mail.gmail.com> <alpine.LRH.2.21.1911040841160.27600@bofh.nohats.ca> <CE06CC6D-E37F-4C90-B782-D14B1D715D4B@cable.comcast.com> <38E47448-63B4-4A5D-8A9D-3AB890EBDDDD@akamai.com> <09886edb-4302-b309-9eaa-f016c4487128@gmail.com> <26819.1572990657@localhost> <2668fa45-7667-51a6-7cb6-4b704c7fba5a@isode.com> <2C97D18E-3DA0-4A2D-8179-6D86EB835783@gmail.com> <MN2PR11MB43669E4CEF13CDA51A764F9AB5790@MN2PR11MB4366.namprd11.prod.outlook.com> <20191.1573054128@localhost> <15BCDF05-FB13-45D2-A5DF-70618EBA1A5A@gmail.com> <9182.1573147520@localhost> <A3493C65-7F8A-407D-A9F4-FF36296C0920@gmail.com> <CAMm+LwiP4Ypuyh2xsd8qBjUfwuNzOYOfp3OrDnPmU-YwMH2pMw@mail.gmail.com> <02eb79d1-1830-5830-ed95-b743f601a8de@network-heretics.com> <f60f410e-1cab-368b-b981-4e85c0f6a816@sandelman.ca>
From: Keith Moore <moore@network-heretics.com>
Message-ID: <84ee7053-1dbb-bfcc-c576-c2cf115a743e@network-heretics.com>
Date: Fri, 15 Nov 2019 09:08:50 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0
MIME-Version: 1.0
In-Reply-To: <f60f410e-1cab-368b-b981-4e85c0f6a816@sandelman.ca>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/0YPJBlEn9j-KrP5R3YGV55CRHYY>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 15 Nov 2019 14:08:59 -0000

On 11/15/19 2:56 AM, Michael Richardson wrote:

> On 2019-11-13 11:25 p.m., Keith Moore wrote:
>> On 11/13/19 10:07 AM, Phillip Hallam-Baker wrote:
>>
>>> Maybe what we need is a structure that assigns multiple reviewers for
>>> some projects and rubber stamps others.
>> Seems like ADs already have a fair amount of discretion to ask for
>> multiple in-depth reviewers vs. getting minimal review.   If having a
>> human make such decisions isn't your idea of an appropriate
>> "structure", I'd be curious to know what is.
>>
> The issue is that is only so much senior security clue to go around.
> There is a non-trivial amount of effort for an-out-area reviewer to spin
> up enough understanding about what a WG is doing.  There are a lot of
> documents that simply allocate a new attribute from an existing registry
> and then use it for something.  Determining if this has a trivial or
> non-trivial security impact can be difficult.  If it turns out to be
> trivial, then we've wasted the reviewers time (opportunity cost).  If it
> turns out not to be trivial (and the reviewer missed that), then if we
> are lucky, we catch it at IESG time, and then it might be a year later.

I don't disagree with any of the above.  And yet, I don't see how it's 
responding to either of the above replies.

Keith