Re: Author and attendance measurements [Was: Re: Thought experiment [Re: Quality of Directorate reviews]]

Carsten Bormann <cabo@tzi.org> Sat, 09 November 2019 14:36 UTC

Return-Path: <cabo@tzi.org>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 041B512010D for <ietf@ietfa.amsl.com>; Sat, 9 Nov 2019 06:36:26 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.865
X-Spam-Level:
X-Spam-Status: No, score=-0.865 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_SBL_CSS=3.335, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NGp9TKFkfprc for <ietf@ietfa.amsl.com>; Sat, 9 Nov 2019 06:36:24 -0800 (PST)
Received: from gabriel-vm-2.zfn.uni-bremen.de (gabriel-vm-2.zfn.uni-bremen.de [134.102.50.17]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D6DBC1200C5 for <ietf@ietf.org>; Sat, 9 Nov 2019 06:36:23 -0800 (PST)
Received: from client-0178.vpn.uni-bremen.de (client-0178.vpn.uni-bremen.de [134.102.107.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gabriel-vm-2.zfn.uni-bremen.de (Postfix) with ESMTPSA id 479KT175Nbzyv9; Sat, 9 Nov 2019 15:36:21 +0100 (CET)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
Subject: Re: Author and attendance measurements [Was: Re: Thought experiment [Re: Quality of Directorate reviews]]
From: Carsten Bormann <cabo@tzi.org>
In-Reply-To: <26D1F92D-67A9-4F96-9FFF-76D525C4BE6E@piuha.net>
Date: Sat, 9 Nov 2019 15:36:21 +0100
X-Mao-Original-Outgoing-Id: 595002979.1446511-1279e1201da606fdb997055e9bbbde1f
Content-Transfer-Encoding: quoted-printable
Message-Id: <C8225595-4B6E-462D-BB7D-31D8E7ADFA1C@tzi.org>
References: <157279399807.13506.13363770981495597049.idtracker@ietfa.amsl.com> <0EF64763-BA25-468A-B387-91445A61D318@gmail.com> <CAJU8_nUovmFmgNiYx0ez_1f+GPdU9xGViDYWfowEEomrn0pyDw@mail.gmail.com> <alpine.LRH.2.21.1911040841160.27600@bofh.nohats.ca> <CE06CC6D-E37F-4C90-B782-D14B1D715D4B@cable.comcast.com> <38E47448-63B4-4A5D-8A9D-3AB890EBDDDD@akamai.com> <09886edb-4302-b309-9eaa-f016c4487128@gmail.com> <26819.1572990657@localhost> <2668fa45-7667-51a6-7cb6-4b704c7fba5a@isode.com> <2C97D18E-3DA0-4A2D-8179-6D86EB835783@gmail.com> <91686B28-9583-4A8E-AF8A-E66977B1FE13@gmail.com> <012b9437-4440-915c-f1f9-b85e1b0be768@gmail.com> <8b387d57-f056-7de3-9851-a431b17b67b4@network-heretics.com> <c97440e1-c6bb-8219-2970-357d9c45beb2@gmail.com> <fa406d85-babb-9821-047f-ad7efbdd23f5@network-heretics.com> <11796.1573157656@localhost> <26D1F92D-67A9-4F96-9FFF-76D525C4BE6E@piuha.net>
To: IETF Discussion <ietf@ietf.org>
X-Mailer: Apple Mail (2.3445.9.1)
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/mz_jB5Bj9znOmKEHn2jNHg_yk2A>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 09 Nov 2019 14:36:26 -0000

On Nov 8, 2019, at 08:11, Jari Arkko <jari.arkko@piuha.net>; wrote:
> 
> 2015	698
> 2016	711
> 2017	630
> 2018	541
> 2019	370

>> 370*(365.0/Time.now.yday)
=> 431.4696485623003

More importantly, the RPC is constipated by the v3 switch, so those numbers aren’t very indicative.  Maybe looking at

>> 370*(365.0/Time.parse("2019-09-16").yday)
=> 521.4285714285714

gives a clearer indication that we are about where we were in 2018.  But really, there should be some metric run over the RFC editor queue as well.

OK, so much for playing scientist here.  Now for the process wonk in me (which I’m definitely only playing).

It seems to me all the proposals to add work for everyone to get better metadata for making these measurements are non-starters.  A little more information can be extracted out of the existing information in the datatracker, which would make a nice project for a student maybe, but the main contributors all have way more important things to do.

So what do we learn?  
Citing this snippet again:

> My impression from the regular plenary reports on attendance my impression is
> that the number of people involved in the IETF is declining, yet the number
> of RFCs being processed is going up.

There are lots of impressions here, which as we see often aren’t supported (or even are contradicted) by the indicators we have.
E.g., my “impression” also is that conflicts are going up, but maybe that is just a facet of the IoT work I’m engaged in going mainstream and impacting/being impacted by more WGs.

The original idea in some of the contributions to this thread, which in my words I’d summarize as lobotomizing the quality engineering we do so we can get more candidates for ADs, is a non-starter as well.  Most of the arguments in this thread for this are bolstered by statements about what specific participants of the process *should* be doing (also known as process confabulation in software engineering), not by observations about reality.

That we don’t have a good alternative to the current process doesn’t mean we can’t tune some knobs, and I’d prefer if we’d focus on what these knobs are.  Strengthening the directorate review system (with the objective to reduce variance in review quality and timing) is very well worth some effort.  Feeding more early cross-area review into the WG process that precedes the IESG review also would help; we have been trying to do this but probably need to be more consistent in these efforts.

All in all I think that making someone responsible for proposing tuning actions and observing and evaluating the results, preferably over a longer time than a single AD tenure, might help.  Not in the sense of “wenn ich nicht mehr weiter weiß, bilde ich ‘nen Arbeitskreis” (not sure that this aphorism about the tendency of German bureaucracy to shift responsibility to useless committees is translatable), but in the sense of creating actual responsibility.  Sweeping actions would still need to go through our consensus process, which makes them very expensive, but continued attention on smaller tweaks might already help a lot.

Grüße, Carsten