Re: Quality of Directorate reviews

Mark Nottingham <mnot@mnot.net> Fri, 29 November 2019 21:54 UTC

Return-Path: <mnot@mnot.net>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 071E81200F6 for <ietf@ietfa.amsl.com>; Fri, 29 Nov 2019 13:54:26 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.699
X-Spam-Level:
X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, MIME_QP_LONG_LINE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=mnot.net header.b=PQsDzyxX; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=AbhcdT3Y
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lHzS4Yk2lBU3 for <ietf@ietfa.amsl.com>; Fri, 29 Nov 2019 13:54:22 -0800 (PST)
Received: from wout1-smtp.messagingengine.com (wout1-smtp.messagingengine.com [64.147.123.24]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 936D11200DE for <ietf@ietf.org>; Fri, 29 Nov 2019 13:54:22 -0800 (PST)
Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id B62B48D3; Fri, 29 Nov 2019 16:54:20 -0500 (EST)
Received: from mailfrontend1 ([10.202.2.162]) by compute3.internal (MEProxy); Fri, 29 Nov 2019 16:54:21 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mnot.net; h= content-type:content-transfer-encoding:from:mime-version:subject :date:message-id:references:cc:in-reply-to:to; s=fm1; bh=pWYGZ3r 7lB1nXyt95warAqndKoU/DC8IhQ2BMZdU+F4=; b=PQsDzyxXv/gfJo3dIJsfrRI TOjHUK4GsOjLVn5YvQlTBZvjwfMReXtQOOFdXFCLfGD7jTCt29F6gEHj7829FIt2 ElMXRCNM3NM7c0WxdM7+iAZwx2IjuWMhUy/8+nJVxsEnlx9UN4N9/rkfGMF06S8e mVpuXx51fMtic6hFNPv/cUyJnbhEtbCAadHyocZuHR0VR5PPQCIiTKic9DfywDl/ vO0tlHlfIOb8tf6PD0ft173X/tgd73nGiXXEP4tX45JlfkUkWcBt9MaSWg+oITbv lv6IkXtRj+t2SLCmrjeXGjQy1CauxLmDc8/J2uwVvb5xWgUIfpz6LgGfrW6DXzA= =
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm1; bh=pWYGZ3r7lB1nXyt95warAqndKoU/DC8IhQ2BMZdU+ F4=; b=AbhcdT3YSfyw3m0su7ofNFl4gHlsrkTVfdQNWFK2cz0DaHDeCeSFbHD22 VSo/yxIRlsgEFLL8qDsNBkRoBqpsViQOawp6Y6g8advKy0ciJlrfyuSSnZ3jfrkf HJTtorih/ptdxHiqeg5qPQvGg2xD46oDp9Optgl6jieX1gygEDJse6ygjw6ofbgt EMrk2xs8N7dqW2DHOaKqAkTEUUggWNR6S4zSXkWv+QT2qMasNqnpfdGcw5e3SN0P sh+b2c5EOGhYr46OQIexxs35x9NOU9EpMmrBK9OsVFssKUDWFcCTUJStHz0VfNnX txwAYKra7GzH1Jyg09Qx8y0u2PFNQ==
X-ME-Sender: <xms:i5PhXRyxkWycnuaaHX17Brp_Yj3tqaYNthyp-Wkig3xgg2U2H3lZOA>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedufedrudeiledgudehudcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpegtgffhggfufffkfhgjvffosegrjehmrehhtdejnecuhfhrohhmpeforghr khcupfhothhtihhnghhhrghmuceomhhnohhtsehmnhhothdrnhgvtheqnecuffhomhgrih hnpegtohguvggrshgtrhgrfhhtrdgtohhmnecukfhppedurddugeefrdehkedrleeknecu rfgrrhgrmhepmhgrihhlfhhrohhmpehmnhhothesmhhnohhtrdhnvghtnecuvehluhhsth gvrhfuihiivgeptd
X-ME-Proxy: <xmx:i5PhXTpbGXVoH9YYriCj81f3bC4iV9l-EGx1MKwPCQ9b6_FGW8eA6Q> <xmx:i5PhXV5Ontyg9a7X9L_82pmy_nHMdRc5Ch5uG-kE5NzjmnweN0jR7g> <xmx:i5PhXeA04nNRCGKOb7g_sl6n88fYXdWSXODN428eedfu7i2qfvbQpA> <xmx:jJPhXXWA3FwKAnwlzsb8sX4bipPtD5ENM9XDIV3GpEfiAxogztJppQ>
Received: from [IPv6:2001:8004:1202:9c52:5039:630f:cc7e:f1e0] (unknown [1.143.58.98]) by mail.messagingengine.com (Postfix) with ESMTPA id 8D23C80060; Fri, 29 Nov 2019 16:54:18 -0500 (EST)
Content-Type: multipart/alternative; boundary="Apple-Mail-A34057C4-B63C-4518-80ED-60BC805CCABF"
Content-Transfer-Encoding: 7bit
From: Mark Nottingham <mnot@mnot.net>
Mime-Version: 1.0 (1.0)
Subject: Re: Quality of Directorate reviews
Date: Sat, 30 Nov 2019 08:54:14 +1100
Message-Id: <FC0E2225-E277-42C2-8173-9E8213EC2FF7@mnot.net>
References: <CADnDZ89At83GGr8pUb8QfEvaeXN8qQ=XmjtJt5QeDcYm96fPUg@mail.gmail.com>
Cc: alexey.melnikov@isode.com, ietf <ietf@ietf.org>, rwilton@cisco.com, Michael Richardson <mcr+ietf@sandelman.ca>
In-Reply-To: <CADnDZ89At83GGr8pUb8QfEvaeXN8qQ=XmjtJt5QeDcYm96fPUg@mail.gmail.com>
To: Abdussalam Baryun <abdussalambaryun@gmail.com>
X-Mailer: iPhone Mail (17B111)
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/1GvVHB2vCzwtxOCRWgnWYQRZpzU>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 29 Nov 2019 21:54:26 -0000

There is a lot of evidence that a culture of blame leads to bad engineering outcomes. 

John Allspaw wrote something about this a while back:
  https://codeascraft.com/2012/05/22/blameless-postmortems/

I think it’d be better for this community to be  focused less on blame and more on becoming what John refers to as a just culture. 

Cheers,

Sent from my iPhone

> On 30 Nov 2019, at 5:30 am, Abdussalam Baryun <abdussalambaryun@gmail.com> wrote:
> 
> 
> Hi Alex and Ralph,
> 
> The quality will never increase if authors are not answering positively to discussions in their WG, however, I always blame first WG-authors and second WG chair and then thirdly the AD, for low quality of discussions/drafts/works.
> 
> > From: ietf <ietf-bounces@ietf.org>; On Behalf Of Ralph Droms
> > Sent: 06 November 2019 11:03
> > To: Alexey Melnikov <alexey.melnikov@isode.com>;
> > Cc: Michael Richardson <mcr+ietf@sandelman.ca>;; ietf <ietf@ietf.org>;
> > Subject: Re: Quality of Directorate reviews
> > 
> > 
> > 
> > > On Nov 6, 2019, at 5:34 AM, Alexey Melnikov <alexey.melnikov@isode.com>;
> > wrote:
> > >
> > > Hi Michael,
> > >
> > > On 05/11/2019 21:50, Michael Richardson wrote:
> > >>
> > >> Brian E Carpenter <brian.e.carpenter@gmail.com>; wrote:
> > >>> If we want the IESG job to be more reasonably sized, we have to take
> > >>> work away from the ADs.
> 
> I agree
> 
> >>>>As far as I can see, that means taking away
> > >>> their duty of acting as final reviewers. I don't want to name names
> > >>> because I don't think the ADs are to be blamed individually, but
> > >>> some of them spend *enormous* effort on detailed reviews.
> 
> 
> The WG Chair/manager is blamed, they need to encourage discussions/works and deep investigation before submitting drafts to AD. The WG Chair should push WG and authors for best reviews and cooperation.
> 
> 
> > >>
> > >> +1
> > >>
> > >> I think that there is a lack of trust by ADs of the various
> > directorates.
> > >
> > > Other ADs have commented on this, but I think I need to repeat what
> > > they said and expand on it.
> > >
> > > Results are vary varied. Some are quite good (e.g. Gen-Art) and others
> > > really depend on reviewer. ADs responsible for Directorates are faces
> > > with the choice of firing half of their Directorates (which has some
> > > rather unfortunate consequences) and/or raise the bar on who should be
> > > allowed to join. We already struggle to recruit people at all levels
> > > of our organization.
> 
> 
> It is easy to know when the draft is ready, YOU just check if there was many discussions on the WG list, if non then send it back to WG to work.
> 
> > >
> > > Best Regards,
> > > Alexey
> > 
> > We shouldn't be depending on last-minute quality checks to maintain the
> > quality of our output. 
> 
> Agree
> 
> >Working groups should be producing documents that
> > are ready to publish, and develop trust that their documents are high
> > quality.
> 
> Agree, the authors need to answer all questions on the list as if all participants are ADs. WG authors reply differently with AD than with WG participant, this needs to change.
> 
> AB