Re: Author and attendance measurements [Was: Re: Thought experiment [Re: Quality of Directorate reviews]]
John C Klensin <john-ietf@jck.com> Fri, 08 November 2019 13:56 UTC
Return-Path: <john-ietf@jck.com>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EAE3B1200F9 for <ietf@ietfa.amsl.com>; Fri, 8 Nov 2019 05:56:25 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.898
X-Spam-Level:
X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_NONE=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ln_TnJCsGfe0 for <ietf@ietfa.amsl.com>; Fri, 8 Nov 2019 05:56:24 -0800 (PST)
Received: from bsa2.jck.com (ns.jck.com [70.88.254.51]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C73D212003F for <ietf@ietf.org>; Fri, 8 Nov 2019 05:56:23 -0800 (PST)
Received: from [198.252.137.10] (helo=PSB) by bsa2.jck.com with esmtp (Exim 4.82 (FreeBSD)) (envelope-from <john-ietf@jck.com>) id 1iT4kP-000Cz6-BH; Fri, 08 Nov 2019 08:56:21 -0500
Date: Fri, 08 Nov 2019 08:56:15 -0500
From: John C Klensin <john-ietf@jck.com>
To: Keith Moore <moore@network-heretics.com>, ietf@ietf.org
Subject: Re: Author and attendance measurements [Was: Re: Thought experiment [Re: Quality of Directorate reviews]]
Message-ID: <335EAA17262BD4585E4E8964@PSB>
In-Reply-To: <5ae8bb9f-3aa6-b930-70e7-d38d7bc46c07@network-heretics.com>
References: <157279399807.13506.13363770981495597049.idtracker@ietfa.amsl.com> <0EF64763-BA25-468A-B387-91445A61D318@gmail.com> <CAJU8_nUovmFmgNiYx0ez_1f+GPdU9xGViDYWfowEEomrn0pyDw@mail.gmail.com> <alpine.LRH.2.21.1911040841160.27600@bofh.nohats.ca> <CE06CC6D-E37F-4C90-B782-D14B1D715D4B@cable.comcast.com> <38E47448-63B4-4A5D-8A9D-3AB890EBDDDD@akamai.com> <09886edb-4302-b309-9eaa-f016c4487128@gmail.com> <26819.1572990657@localhost> <2668fa45-7667-51a6-7cb6-4b704c7fba5a@isode.com> <2C97D18E-3DA0-4A2D-8179-6D86EB835783@gmail.com> <91686B28-9583-4A8E-AF8A-E66977B1FE13@gmail.com> <012b9437-4440-915c-f1f9-b85e1b0be768@gmail.com> <8b387d57-f056-7de3-9851-a431b17b67b4@network-heretics.com> <c97440e1-c6bb-8219-2970-357d9c45beb2@gmail.com> <fa406d85-babb-9821-047f-ad7efbdd23f5@network-heretics.com> <11796.1573157656@localhost> <26D1F92D-67A9-4F96-9FFF-76D525C4BE6E@piuha.net> <5ae8bb9f-3aa6-b930-70e7-d38d7bc46c07@network-heretics.com>
X-Mailer: Mulberry/4.0.8 (Win32)
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
X-SA-Exim-Connect-IP: 198.252.137.10
X-SA-Exim-Mail-From: john-ietf@jck.com
X-SA-Exim-Scanned: No (on bsa2.jck.com); SAEximRunCond expanded to false
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/wadtxRiU11b4pgwm34jgclVk5vo>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Nov 2019 13:56:26 -0000
--On Friday, November 8, 2019 02:31 -0500 Keith Moore <moore@network-heretics.com> wrote: > On 11/8/19 2:11 AM, Jari Arkko wrote: > >> Pretty healthy numbers I'd say, even, if 600-700 people >> manage to publish RFCs and the meetings draw 900-1100 people. > > To me it sounds unhealthy. If meeting attendance is an > indicator of the number of active participants, it sounds like > about half of the participants are publishing an RFC every > year. With that many authors I have to wonder how many people > have time to participate in discussion of others' > documents. Of course a person can author one document > while discussing others, but it's easy for an author's > attention to others' drafts to diminish while revising his own > draft. Keith, As Jari cautioned, those numbers have to be interpreted with care. For example, the number of documents with many authors has has risen over the years since Jon Postel aggressively pushed back against the practice. We also have issued documents that are revisions of older ones. In general, original author names have been retained even when newer authors have been added on and the actual involvement of the original ones has been minimal. There may also be issues with listed contributors because, at the discretion of a WG or whomever is holding the pen, that category may include someone who contributed a concept or a few critical paragraph, but who was not otherwise significantly involved in the production of the document. Even Jari's comment that you quote is problematic because, while 600-700 people may have their names on RFCs that are published, sometimes it is accurate to say that they published them and sometimes some of them are just names. Even the definition of "author" may differ from one document to the next. We also aren't careful about distinguishing between authors and editors. For example, with a document that is really developed by a WG, the "author" may be mostly a compiler of WG comments and text rather than a contributor of and advocate for original ideas. Either may involve a great deal of time and effort, but the kind and level of involvement is different. I even know of at least a couple of often-cited RFCs in which there are several listed authors but none of them, independent of their intellectual or other contributions, actually provided any significant fraction of the published text. Similarly, comparing author counts to f2f meeting registrations does not quite work and is probably getting worse over time as meeting costs rise and active remote participation increases. We should have at least approximate numbers by now for total registered participants, but probably not reaching back to the point that Jari is comfortable with the authorship numbers. I can imagine ways to estimate numbers and control, in the statistical sense, for some (but not all) of those issues without, e.g., interviewing authors or doing a careful document-by-document evaluation. They all would require a lot of work. Maybe others disagree, but I'd rather see the people who would probably need to do that work spending their time on substantive technical contributions instead. But, without doing that work (at least), inferences about how many people are writing documents and therefore not evaluating those of others seems fairly close to pure speculation. best, john
- NomCom 2019 Call for Community Feedback NomCom Chair 2019
- Re: NomCom 2019 Call for Community Feedback Stewart Bryant
- Re: NomCom 2019 Call for Community Feedback Salz, Rich
- Re: NomCom 2019 Call for Community Feedback Kyle Rose
- Re: NomCom 2019 Call for Community Feedback Paul Wouters
- Re: NomCom 2019 Call for Community Feedback Kathleen Moriarty
- Re: NomCom 2019 Call for Community Feedback Livingood, Jason
- Re: NomCom 2019 Call for Community Feedback Salz, Rich
- Re: NomCom 2019 Call for Community Feedback Randy Bush
- Re: NomCom 2019 Call for Community Feedback Yoav Nir
- Re: NomCom 2019 Call for Community Feedback Kathleen Moriarty
- Re: NomCom 2019 Call for Community Feedback Salz, Rich
- Re: NomCom 2019 Call for Community Feedback Stephen Farrell
- Re: NomCom 2019 Call for Community Feedback Brian E Carpenter
- Re: NomCom 2019 Call for Community Feedback Salz, Rich
- Re: NomCom 2019 Call for Community Feedback Stephen Farrell
- Re: NomCom 2019 Call for Community Feedback Brian E Carpenter
- Re: NomCom 2019 Call for Community Feedback Kyle Rose
- Re: NomCom 2019 Call for Community Feedback Keith Moore
- Re: NomCom 2019 Call for Community Feedback Tim Wicinski
- Re: NomCom 2019 Call for Community Feedback Rob Sayre
- Re: NomCom 2019 Call for Community Feedback Benjamin Kaduk
- Re: NomCom 2019 Call for Community Feedback Bob Hinden
- Re: NomCom 2019 Call for Community Feedback Donald Eastlake
- Re: NomCom 2019 Call for Community Feedback Kathleen Moriarty
- Re: NomCom 2019 Call for Community Feedback Eric Vyncke (evyncke)
- Re: NomCom 2019 Call for Community Feedback Salz, Rich
- Re: NomCom 2019 Call for Community Feedback Keith Moore
- Re: NomCom 2019 Call for Community Feedback John Levine
- Re: NomCom 2019 Call for Community Feedback Salz, Rich
- Re: NomCom 2019 Call for Community Feedback Stewart Bryant
- Re: NomCom 2019 Call for Community Feedback Kathleen Moriarty
- Re: NomCom 2019 Call for Community Feedback Michael Richardson
- Re: NomCom 2019 Call for Community Feedback Kyle Rose
- AD workload [was Re: NomCom 2019 Call for Communi… Brian E Carpenter
- Re: AD workload [was Re: NomCom 2019 Call for Com… Mark Nottingham
- Re: AD workload [was Re: NomCom 2019 Call for Com… Rob Sayre
- Re: AD workload [was Re: NomCom 2019 Call for Com… Terry Manderson
- Re: AD workload [was Re: NomCom 2019 Call for Com… Salz, Rich
- Re: AD workload [was Re: NomCom 2019 Call for Com… Keith Moore
- Re: AD workload [was Re: NomCom 2019 Call for Com… Leif Johansson
- Quality of Directorate reviews Alexey Melnikov
- Re: Quality of Directorate reviews Ralph Droms
- RE: Quality of Directorate reviews Rob Wilton (rwilton)
- Re: Quality of Directorate reviews Alexey Melnikov
- Re: Quality of Directorate reviews Stewart Bryant
- Re: AD workload [was Re: NomCom 2019 Call for Com… Salz, Rich
- RE: Quality of Directorate reviews Paul Wouters
- Re: Quality of Directorate reviews Carsten Bormann
- Re: Quality of Directorate reviews Ralph Droms
- Re: Quality of Directorate reviews Keith Moore
- Re: Quality of Directorate reviews Ralph Droms
- Re: AD workload [was Re: NomCom 2019 Call for Com… Michael Richardson
- Re: Quality of Directorate reviews Michael Richardson
- Re: Quality of Directorate reviews Michael Richardson
- Re: Quality of Directorate reviews Keith Moore
- Re: Quality of Directorate reviews Michael Richardson
- Re: Quality of Directorate reviews Michael Richardson
- Re: AD workload [was Re: NomCom 2019 Call for Com… Stewart Bryant
- Re: Quality of Directorate reviews Stewart Bryant
- Re: Quality of Directorate reviews Michael Richardson
- Re: Quality of Directorate reviews Bob Hinden
- Re: Quality of Directorate reviews Bob Hinden
- Re: Quality of Directorate reviews Salz, Rich
- Re: Quality of Directorate reviews Bob Hinden
- Re: Quality of Directorate reviews Jared Mauch
- Re: Quality of Directorate reviews Salz, Rich
- Re: Quality of Directorate reviews Stewart Bryant
- Re: AD workload [was Re: NomCom 2019 Call for Com… Leif Johansson
- Re: AD workload [was Re: NomCom 2019 Call for Com… Salz, Rich
- Re: AD workload [was Re: NomCom 2019 Call for Com… Salz, Rich
- Re: AD workload [was Re: NomCom 2019 Call for Com… Leif Johansson
- Re: AD workload [was Re: NomCom 2019 Call for Com… Leif Johansson
- Thought experiment [Re: Quality of Directorate re… Brian E Carpenter
- Re: Thought experiment [Re: Quality of Directorat… Keith Moore
- Re: Quality of Directorate reviews Keith Moore
- Re: Quality of Directorate reviews Keith Moore
- Re: Thought experiment [Re: Quality of Directorat… Brian E Carpenter
- Re: Thought experiment [Re: Quality of Directorat… Keith Moore
- Re: Thought experiment [Re: Quality of Directorat… Nico Williams
- Re: Thought experiment [Re: Quality of Directorat… Brian E Carpenter
- Re: Thought experiment [Re: Quality of Directorat… Keith Moore
- Re: Thought experiment [Re: Quality of Directorat… Nico Williams
- Re: Thought experiment [Re: Quality of Directorat… Leif Johansson
- Re: Thought experiment [Re: Quality of Directorat… Bob Hinden
- Re: Thought experiment [Re: Quality of Directorat… Bob Hinden
- Re: Thought experiment [Re: Quality of Directorat… Keith Moore
- Re: Thought experiment [Re: Quality of Directorat… tom petch
- Re: Quality of Directorate reviews Michael Richardson
- Re: Quality of Directorate reviews Michael Richardson
- Re: Quality of Directorate reviews Stewart Bryant
- Re: Thought experiment [Re: Quality of Directorat… Nico Williams
- Re: Thought experiment [Re: Quality of Directorat… Nico Williams
- Re: Thought experiment [Re: Quality of Directorat… Michael Richardson
- Re: Thought experiment [Re: Quality of Directorat… Michael Richardson
- Re: Thought experiment [Re: Quality of Directorat… Michael Richardson
- Re: Thought experiment [Re: Quality of Directorat… Nico Williams
- Re: Thought experiment [Re: Quality of Directorat… Michael Richardson
- Re: Thought experiment [Re: Quality of Directorat… Salz, Rich
- Re: Thought experiment [Re: Quality of Directorat… Keith Moore
- Re: Thought experiment [Re: Quality of Directorat… Keith Moore
- Re: Thought experiment [Re: Quality of Directorat… Keith Moore
- Re: Thought experiment [Re: Quality of Directorat… Keith Moore
- Re: Thought experiment [Re: Quality of Directorat… Michael Richardson
- Re: Thought experiment [Re: Quality of Directorat… Michael Richardson
- Re: Thought experiment [Re: Quality of Directorat… Michael Richardson
- Re: Thought experiment [Re: Quality of Directorat… Brian E Carpenter
- Re: Thought experiment [Re: Quality of Directorat… Michael Richardson
- Re: Thought experiment [Re: Quality of Directorat… Bob Hinden
- Re: Thought experiment [Re: Quality of Directorat… Keith Moore
- Re: Thought experiment [Re: Quality of Directorat… Keith Moore
- Re: Thought experiment [Re: Quality of Directorat… Nico Williams
- Re: Thought experiment [Re: Quality of Directorat… Nico Williams
- Re: Thought experiment [Re: Quality of Directorat… Nico Williams
- Re: Thought experiment [Re: Quality of Directorat… Christian Huitema
- Author and attendance measurements [Was: Re: Thou… Jari Arkko
- Re: Author and attendance measurements [Was: Re: … Keith Moore
- Re: AD workload [was Re: NomCom 2019 Call for Com… Eliot Lear
- Re: AD workload [was Re: NomCom 2019 Call for Com… Ole Troan
- Re: Author and attendance measurements [Was: Re: … John C Klensin
- Re: AD workload [was Re: NomCom 2019 Call for Com… Salz, Rich
- Re: Author and attendance measurements [Was: Re: … Michael Richardson
- Re: Author and attendance measurements [Was: Re: … Andrew G. Malis
- Re: AD workload [was Re: NomCom 2019 Call for Com… Keith Moore
- Re: Thought experiment [Re: Quality of Directorat… Bob Hinden
- Re: Author and attendance measurements [Was: Re: … Keith Moore
- Re: Author and attendance measurements [Was: Re: … Michael Richardson
- Re: NomCom 2019 Call for Community Feedback John Leslie
- Re: AD workload [was Re: NomCom 2019 Call for Com… Brian E Carpenter
- Re: AD workload [was Re: NomCom 2019 Call for Com… Keith Moore
- Re: AD workload [was Re: NomCom 2019 Call for Com… Michael Richardson
- Re: AD workload [was Re: NomCom 2019 Call for Com… Keith Moore
- Re: Author and attendance measurements [Was: Re: … John C Klensin
- Re: NomCom 2019 Call for Community Feedback John C Klensin
- Re: NomCom 2019 Call for Community Feedback Mary B
- Re: Author and attendance measurements [Was: Re: … Carsten Bormann
- Re: Author and attendance measurements [Was: Re: … Michael Richardson
- Re: Quality of Directorate reviews Phillip Hallam-Baker
- Re: Quality of Directorate reviews Keith Moore
- Re: Quality of Directorate reviews Michael Richardson
- Re: Quality of Directorate reviews Keith Moore
- Re: Thought experiment [Re: Quality of Directorat… Julian Reschke
- Re: Thought experiment [Re: Quality of Directorat… Michael Richardson
- Re: Quality of Directorate reviews Michael Richardson
- Re: Quality of Directorate reviews Benjamin Kaduk
- Re: Quality of Directorate reviews Keith Moore
- Re: Quality of Directorate reviews Michael Richardson
- Re: Quality of Directorate reviews Benjamin Kaduk
- Re: Quality of Directorate reviews Abdussalam Baryun
- Re: Quality of Directorate reviews Mark Nottingham