Re: Quality of Directorate reviews

Stewart Bryant <stewart.bryant@gmail.com> Wed, 06 November 2019 13:01 UTC

Return-Path: <stewart.bryant@gmail.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6F258120874 for <ietf@ietfa.amsl.com>; Wed, 6 Nov 2019 05:01:43 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level:
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id s0iMWi9UyTgR for <ietf@ietfa.amsl.com>; Wed, 6 Nov 2019 05:01:41 -0800 (PST)
Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 20C88120878 for <ietf@ietf.org>; Wed, 6 Nov 2019 05:01:41 -0800 (PST)
Received: by mail-wm1-x32c.google.com with SMTP id t26so3220517wmi.4 for <ietf@ietf.org>; Wed, 06 Nov 2019 05:01:41 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=N7RgpC1zJzV2OQSwAJ2b1BQvbn3aT7M/JL12eqjJ+7s=; b=ZMwv3AGqHs+SbkF76zZu3656WD0Y5K1brTgq8PXNnPW+sMsCcRNkv94lb+UhbLKgig H2k2/wm7g2nLuuExXRoMIUKgB12A0iRnfb2sEVYesCahSVEYDSc/NwEYQINwmrHXzMK0 dx2h0RoqVmwqnIQPT6rPjzlbiQcZk5n/fMiWhmlwEvWxb66HX7G0pCGYnM7/92hqyYmF 7yDwqU36AU2sxKVB4sWeIfUQfdmuep/Mu9fotjGGUa6mTv1+Xc7VK7TqE+bBVuM+POZj KlOJt43LWRMB9PfNvXupfHMmXHZNvfCJLj3xvkCjfBCrVq/8FUCbt8E6BvJWhMPW0IMQ jVYg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=N7RgpC1zJzV2OQSwAJ2b1BQvbn3aT7M/JL12eqjJ+7s=; b=iOL7gtst3E4MQXhox12swAVxhhd7WDFK7UJuye/NZS46YrrE3qy3dBTDXC8mLYmpaC oe7aU64mxx+2wZSGfr6U+0pxwmGPEFY/OhI1qCoYAMkiOqWosQnhV82fHN/KJd4wbwH5 3qfYlmLrKBuojuH5R8MfEuD3YqGderJYNxUeL3yJ5vhiN4rZeuPOcLZjPtADiiGLwx/0 t1Gs3+2yQW4cqHMCQbmrlHL0aPSV81kL4SSZOpSVJP/m5Vs1mAB/TQKEzMKdYItFXEnp 1ihojigHvp6wgFg3Bt3/wL1v1iNgi/KTKE+Fx4I36tcCn+3ROrFrAnec973JGCM299lh V9nw==
X-Gm-Message-State: APjAAAUF0JLqCQA5DPu76YCPBLjK44GAuLCu24jBU2hV2jsBh3blxrD5 +slu1zHaZbPKysUNN3CL3mA=
X-Google-Smtp-Source: APXvYqwp/P92odMhFWc3QSkSDFp9JR/TiWQxT6pzMXMie9s1JYX6Rtz4jPjpMOSQq/cWtjNv1OO9Sw==
X-Received: by 2002:a05:600c:2046:: with SMTP id p6mr2494033wmg.165.1573045299504; Wed, 06 Nov 2019 05:01:39 -0800 (PST)
Received: from appleton.fritz.box ([62.3.64.16]) by smtp.gmail.com with ESMTPSA id j11sm19895934wrq.26.2019.11.06.05.01.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 06 Nov 2019 05:01:38 -0800 (PST)
Content-Type: text/plain; charset="us-ascii"
Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3601.0.10\))
Subject: Re: Quality of Directorate reviews
From: Stewart Bryant <stewart.bryant@gmail.com>
In-Reply-To: <MN2PR11MB43669E4CEF13CDA51A764F9AB5790@MN2PR11MB4366.namprd11.prod.outlook.com>
Date: Wed, 06 Nov 2019 13:01:37 +0000
Cc: Ralph Droms <rdroms.ietf@gmail.com>, Alexey Melnikov <alexey.melnikov@isode.com>, Michael Richardson <mcr+ietf@sandelman.ca>, ietf <ietf@ietf.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <DB7A7C12-52B0-4BAF-AE7B-395B8552F83C@gmail.com>
References: <157279399807.13506.13363770981495597049.idtracker@ietfa.amsl.com> <0EF64763-BA25-468A-B387-91445A61D318@gmail.com> <CAJU8_nUovmFmgNiYx0ez_1f+GPdU9xGViDYWfowEEomrn0pyDw@mail.gmail.com> <alpine.LRH.2.21.1911040841160.27600@bofh.nohats.ca> <CE06CC6D-E37F-4C90-B782-D14B1D715D4B@cable.comcast.com> <38E47448-63B4-4A5D-8A9D-3AB890EBDDDD@akamai.com> <09886edb-4302-b309-9eaa-f016c4487128@gmail.com> <26819.1572990657@localhost> <2668fa45-7667-51a6-7cb6-4b704c7fba5a@isode.com> <2C97D18E-3DA0-4A2D-8179-6D86EB835783@gmail.com> <MN2PR11MB43669E4CEF13CDA51A764F9AB5790@MN2PR11MB4366.namprd11.prod.outlook.com>
To: "Rob Wilton (rwilton)" <rwilton@cisco.com>
X-Mailer: Apple Mail (2.3601.0.10)
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/Ln72E0rdNbRYOeIqjQ3re2y7seo>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 06 Nov 2019 13:01:43 -0000

That is a good idea.

I would also like to see a statement from the responsible AD that they have reviewed the document in detail and believe that it is fully ready to go to for review by people other than the WG and their own directorate. In some cases I really wonder it a draft has been read end to end by anyone before it is passed on for review by the wider community. 

- Stewart

 

> On 6 Nov 2019, at 11:42, Rob Wilton (rwilton) <rwilton@cisco.com> wrote:
> 
> Perhaps RFCs could list (within the document) who reviewed/approved them, and in which role/capacity the review had been performed.
> 
> This could serve two purposes:
> - some minimal reward for those individuals taking the time to review the document,
> - encouragement for the reviewers to ensure that an adequate review has been performed based on the role/capacity in which they are acting. 
> 
> Rob
> 
> 
>> -----Original Message-----
>> From: ietf <ietf-bounces@ietf.org> On Behalf Of Ralph Droms
>> Sent: 06 November 2019 11:03
>> To: Alexey Melnikov <alexey.melnikov@isode.com>
>> Cc: Michael Richardson <mcr+ietf@sandelman.ca>; ietf <ietf@ietf.org>
>> Subject: Re: Quality of Directorate reviews
>> 
>> 
>> 
>>> On Nov 6, 2019, at 5:34 AM, Alexey Melnikov <alexey.melnikov@isode.com>
>> wrote:
>>> 
>>> Hi Michael,
>>> 
>>> On 05/11/2019 21:50, Michael Richardson wrote:
>>>> 
>>>> Brian E Carpenter <brian.e.carpenter@gmail.com> wrote:
>>>>> If we want the IESG job to be more reasonably sized, we have to take
>>>>> work away from the ADs. As far as I can see, that means taking away
>>>>> their duty of acting as final reviewers. I don't want to name names
>>>>> because I don't think the ADs are to be blamed individually, but
>>>>> some of them spend *enormous* effort on detailed reviews.
>>>> 
>>>> +1
>>>> 
>>>> I think that there is a lack of trust by ADs of the various
>> directorates.
>>> 
>>> Other ADs have commented on this, but I think I need to repeat what
>>> they said and expand on it.
>>> 
>>> Results are vary varied. Some are quite good (e.g. Gen-Art) and others
>>> really depend on reviewer. ADs responsible for Directorates are faces
>>> with the choice of firing half of their Directorates (which has some
>>> rather unfortunate consequences) and/or raise the bar on who should be
>>> allowed to join. We already struggle to recruit people at all levels
>>> of our organization.
>>> 
>>> Best Regards,
>>> Alexey
>> 
>> We shouldn't be depending on last-minute quality checks to maintain the
>> quality of our output.  Working groups should be producing documents that
>> are ready to publish, and develop trust that their documents are high
>> quality.
>> 
>> - Ralph
>> 
>>> 
>