Re: Workload constants [was I-D Action: draft-rsalz-termlimits-00.txt]

Christian Huitema <huitema@huitema.net> Fri, 22 October 2021 16:54 UTC

Return-Path: <huitema@huitema.net>
X-Original-To: ietf@ietfa.amsl.com
Delivered-To: ietf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2EC773A1152 for <ietf@ietfa.amsl.com>; Fri, 22 Oct 2021 09:54:30 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.89
X-Spam-Level:
X-Spam-Status: No, score=-1.89 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, NICE_REPLY_A=-0.001, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01] autolearn=unavailable autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 14B-Q2C2d61Y for <ietf@ietfa.amsl.com>; Fri, 22 Oct 2021 09:54:27 -0700 (PDT)
Received: from mx36-out20.antispamcloud.com (mx36-out20.antispamcloud.com [209.126.121.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C87C43A1150 for <ietf@ietf.org>; Fri, 22 Oct 2021 09:54:27 -0700 (PDT)
Received: from xse260.mail2web.com ([66.113.197.6] helo=xse.mail2web.com) by mx136.antispamcloud.com with esmtp (Exim 4.92) (envelope-from <huitema@huitema.net>) id 1mdxoF-000Dff-Ci for ietf@ietf.org; Fri, 22 Oct 2021 18:54:24 +0200
Received: from xsmtp22.mail2web.com (unknown [10.100.68.61]) by xse.mail2web.com (Postfix) with ESMTPS id 4HbVnB6PrRz4nM for <ietf@ietf.org>; Fri, 22 Oct 2021 09:54:22 -0700 (PDT)
Received: from [10.5.2.14] (helo=xmail04.myhosting.com) by xsmtp22.mail2web.com with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.92) (envelope-from <huitema@huitema.net>) id 1mdxoE-000875-Ol for ietf@ietf.org; Fri, 22 Oct 2021 09:54:22 -0700
Received: (qmail 6606 invoked from network); 22 Oct 2021 16:54:19 -0000
Received: from unknown (HELO [192.168.1.103]) (Authenticated-user:_huitema@huitema.net@[172.58.43.63]) (envelope-sender <huitema@huitema.net>) by xmail04.myhosting.com (qmail-ldap-1.03) with ESMTPA for <moore@network-heretics.com>; 22 Oct 2021 16:54:19 -0000
Message-ID: <ffb8b5c6-8af1-794d-9766-a52b8eabe3cf@huitema.net>
Date: Fri, 22 Oct 2021 09:54:18 -0700
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.2.0
Content-Language: en-US
To: Keith Moore <moore@network-heretics.com>, "Salz, Rich" <rsalz@akamai.com>, Brian E Carpenter <brian.e.carpenter@gmail.com>, Barry Leiba <barryleiba=40computer.org@dmarc.ietf.org>
Cc: IETF discussion list <ietf@ietf.org>
References: <20211021005426.639E92B1D176@ary.qy> <e6d59712-ca73-0723-5cb2-b1f749e37577@network-heretics.com> <CAC4RtVDCxp7RveAXxTUU47fYXw5ebV+yJTMkDAJWGvcq-4DyTw@mail.gmail.com> <25a9f62e-1957-0e77-1e7a-733d9dae4a86@gmail.com> <02ccb205-d628-b490-946b-a518e963e210@network-heretics.com> <9538275C-EE23-456B-824A-492E0541FF48@akamai.com> <d765983a-7839-11e4-5ee7-513a0ac25588@network-heretics.com>
From: Christian Huitema <huitema@huitema.net>
Subject: Re: Workload constants [was I-D Action: draft-rsalz-termlimits-00.txt]
In-Reply-To: <d765983a-7839-11e4-5ee7-513a0ac25588@network-heretics.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Originating-IP: 66.113.197.6
X-Spampanel-Domain: xsmtpout.mail2web.com
X-Spampanel-Username: 66.113.197.0/24
Authentication-Results: antispamcloud.com; auth=pass smtp.auth=66.113.197.0/24@xsmtpout.mail2web.com
X-Spampanel-Outgoing-Class: unsure
X-Spampanel-Outgoing-Evidence: Combined (0.15)
X-Recommended-Action: accept
X-Filter-ID: Pt3MvcO5N4iKaDQ5O6lkdGlMVN6RH8bjRMzItlySaT9WLQux0N3HQm8ltz8rnu+BPUtbdvnXkggZ 3YnVId/Y5jcf0yeVQAvfjHznO7+bT5x9j7219Tb9QoiGKb6esGsuKj/EwzSHE5FGYwwjsNRPCIdX 6BQ3tZEguTNPRPRl22PmD6wdmZPcItWbGe10hXJtXL4FsauCVkDjmcYJdU3yWp7KuHNaaKdg7iBE ZefdsNUFWKwa/wzJUjmazeC7ImcaFnGazaZJ5IRUAEf2xItpzBQ6V51u76v35b1wNe/MvdLom48E g3of4Y9DlgiJ0nAJ2+J9PgaoF8SQHto3le4zsHTaeQtlKubP6iUTjj6yPARK6buALVaA782LKxg6 vRmng8N1aLhXqdc+jC1RcnVud53D5caUhbVtvqItBqoizkEt9O20UjkwI0v+LOlw05G4BS+iyyNq bT8dUMXMJ4tUCMj6G37ZfAMLceP5aNHPt26RBupu5v1nytoNnc138GfEJRQ2qC7jjynPIHPNqSn4 QTXUjLjYWQt1/5xnQymMoPsgr/U0flMcy2Vi/IcBgY4arPaiJ1W6hAyiRC61jekdwIcXNugoOEbH RyFULpSjm7jZ1h/HfDRQ5Ig8VhPsPE8NQ/T3Op6Um662jkOH4Bxha1aQO8Owdd/hwUqvChYXKEM8 ZvAPiDSnpLzkbV0PCyWVDRojSVizNl0ce/s7u0P9b7Oijoc3SCZfWp1RjkjWCw/vIUzTXkDAiiJi mGhLUFuS2lhaIetXfCg1JdAVrOwKfGDRJ9dL7joOXZZDvyYSSedUoFIvD3sIcP1fhJPM6B/82ynL mR50j7SN/KU1c/DV0GLua3RJ0N4pwp1zcex/7jhEY8m+dvJolEdXjj+kQUhPfNySVdID1SvWZkW9 NuwOcr13qFZSq8Fx+9otn0aqja8VKPqpdskk5LxBR/9t1zMMkdu6/R2FM84kxYRFSvC1IDg1BRW7 hzp8w3iHcOwbVtsmWfnQGGis4EvbR3jXsI0ESXwhBU2hwt/J18C+HygJl/jEzm1SsR8v3aJbN/NZ fa8pHhHaz+HPa0HAgEx4sWDF
X-Report-Abuse-To: spam@quarantine11.antispamcloud.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf/i2lJRzCJra1F9E-1lnefredWFFY>
X-BeenThere: ietf@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF-Discussion <ietf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf>, <mailto:ietf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf/>
List-Post: <mailto:ietf@ietf.org>
List-Help: <mailto:ietf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf>, <mailto:ietf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Oct 2021 16:54:31 -0000

On 10/22/2021 7:01 AM, Keith Moore wrote:
> On 10/22/21 9:37 AM, Salz, Rich wrote:
>
>>>    Is the relevance of a typical IETF RFC going up or down?
>> How do you tell?  Can you measure it, other than have opinions?
>
> I don't know how to measure it.  (Also, I think there's a general 
> problem in society these days which is that we pay more attention to 
> things that are easily measured, than to things that aren't, even when 
> the things that are easily measured are less important.)

I did try to estimate that, see RFC 8963. I tried various metrics, 
including citation counts, deployment of the technology, and web 
references. The very high level summary is that there are considerable 
differences between RFCs. The majority of the RFCs that I sampled is 
cited a couple of times if at all, deployed moderately, and hardly 
referenced outside of RFC indices and repertories. The highly popular 
ones are cited a lot, deployed a lot, and referenced a lot.

Comparisons between time periods are difficult, because any document is 
likely to accrue more citations and references over time. When 
compensating for that, there were no huge differences between the median 
of samples of RFC from 2018, 2008 and 1998 -- largely because the median 
RFC is not read very much. In my samples, the value of the IETF appears 
to come from a very small number of big hitters, such as for example RFC 
8446 (TLS 1.3) in 2018, or RFC 2267 (Network Ingress Filtering) in 1998. 
There are probably a few big hitters like that every year, someone may 
want to do the study.

There was a huge difference in another metric, the time from initial 
draft to published RFC. Back in 1998, the median delay was one year. It 
was already more than 3 years in 2008, and still is. Much of the delays 
happen in the working group, from initial draft to last call.

-- Christian Huitema