Re: Quality of Directorate reviews

Michael Richardson <mcr+ietf@sandelman.ca> Fri, 15 November 2019 07:56 UTC

Return-Path: <mcr+ietf@sandelman.ca>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ABB7D120115 for <ietf@ietfa.amsl.com>; Thu, 14 Nov 2019 23:56:24 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.2
X-Spam-Level:
X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8XD69-dDehUY for <ietf@ietfa.amsl.com>; Thu, 14 Nov 2019 23:56:22 -0800 (PST)
Received: from tuna.sandelman.ca (tuna.sandelman.ca [IPv6:2607:f0b0:f:3:216:3eff:fe7c:d1f3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2457712011D for <ietf@ietf.org>; Thu, 14 Nov 2019 23:56:21 -0800 (PST)
Received: from [192.168.41.2] (unknown [49.77.183.152]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by tuna.sandelman.ca (Postfix) with ESMTPSA id 29A6E3897C for <ietf@ietf.org>; Fri, 15 Nov 2019 02:53:10 -0500 (EST)
Subject: Re: Quality of Directorate reviews
To: ietf@ietf.org
References: <157279399807.13506.13363770981495597049.idtracker@ietfa.amsl.com> <0EF64763-BA25-468A-B387-91445A61D318@gmail.com> <CAJU8_nUovmFmgNiYx0ez_1f+GPdU9xGViDYWfowEEomrn0pyDw@mail.gmail.com> <alpine.LRH.2.21.1911040841160.27600@bofh.nohats.ca> <CE06CC6D-E37F-4C90-B782-D14B1D715D4B@cable.comcast.com> <38E47448-63B4-4A5D-8A9D-3AB890EBDDDD@akamai.com> <09886edb-4302-b309-9eaa-f016c4487128@gmail.com> <26819.1572990657@localhost> <2668fa45-7667-51a6-7cb6-4b704c7fba5a@isode.com> <2C97D18E-3DA0-4A2D-8179-6D86EB835783@gmail.com> <MN2PR11MB43669E4CEF13CDA51A764F9AB5790@MN2PR11MB4366.namprd11.prod.outlook.com> <20191.1573054128@localhost> <15BCDF05-FB13-45D2-A5DF-70618EBA1A5A@gmail.com> <9182.1573147520@localhost> <A3493C65-7F8A-407D-A9F4-FF36296C0920@gmail.com> <CAMm+LwiP4Ypuyh2xsd8qBjUfwuNzOYOfp3OrDnPmU-YwMH2pMw@mail.gmail.com> <02eb79d1-1830-5830-ed95-b743f601a8de@network-heretics.com>
From: Michael Richardson <mcr+ietf@sandelman.ca>
Message-ID: <f60f410e-1cab-368b-b981-4e85c0f6a816@sandelman.ca>
Date: Fri, 15 Nov 2019 15:56:12 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0
MIME-Version: 1.0
In-Reply-To: <02eb79d1-1830-5830-ed95-b743f601a8de@network-heretics.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/M-V50isrSkIRsGbIpKH1vJpC6lM>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 15 Nov 2019 07:56:25 -0000


On 2019-11-13 11:25 p.m., Keith Moore wrote:
>
> On 11/13/19 10:07 AM, Phillip Hallam-Baker wrote:
>
>> Maybe what we need is a structure that assigns multiple reviewers for
>> some projects and rubber stamps others.
>
> Seems like ADs already have a fair amount of discretion to ask for
> multiple in-depth reviewers vs. getting minimal review.   If having a
> human make such decisions isn't your idea of an appropriate
> "structure", I'd be curious to know what is.
>

The issue is that is only so much senior security clue to go around.
There is a non-trivial amount of effort for an-out-area reviewer to spin
up enough understanding about what a WG is doing.  There are a lot of
documents that simply allocate a new attribute from an existing registry
and then use it for something.  Determining if this has a trivial or
non-trivial security impact can be difficult.  If it turns out to be
trivial, then we've wasted the reviewers time (opportunity cost).  If it
turns out not to be trivial (and the reviewer missed that), then if we
are lucky, we catch it at IESG time, and then it might be a year later.

WGs are given security advisors, and most don't use them, and many of
them are AWOL.