Re: Workload constants [was I-D Action: draft-rsalz-termlimits-00.txt]

[Christian Huitema <[e-mail]>]

On 10/22/2021 7:01 AM, Keith Moore wrote:
> On 10/22/21 9:37 AM, Salz, Rich wrote:
>
>>>    Is the relevance of a typical IETF RFC going up or down?
>> How do you tell?  Can you measure it, other than have opinions?
>
> I don't know how to measure it.  (Also, I think there's a general 
> problem in society these days which is that we pay more attention to 
> things that are easily measured, than to things that aren't, even when 
> the things that are easily measured are less important.)

I did try to estimate that, see RFC 8963. I tried various metrics, 
including citation counts, deployment of the technology, and web 
references. The very high level summary is that there are considerable 
differences between RFCs. The majority of the RFCs that I sampled is 
cited a couple of times if at all, deployed moderately, and hardly 
referenced outside of RFC indices and repertories. The highly popular 
ones are cited a lot, deployed a lot, and referenced a lot.

Comparisons between time periods are difficult, because any document is 
likely to accrue more citations and references over time. When 
compensating for that, there were no huge differences between the median 
of samples of RFC from 2018, 2008 and 1998 -- largely because the median 
RFC is not read very much. In my samples, the value of the IETF appears 
to come from a very small number of big hitters, such as for example RFC 
8446 (TLS 1.3) in 2018, or RFC 2267 (Network Ingress Filtering) in 1998. 
There are probably a few big hitters like that every year, someone may 
want to do the study.

There was a huge difference in another metric, the time from initial 
draft to published RFC. Back in 1998, the median delay was one year. It 
was already more than 3 years in 2008, and still is. Much of the delays 
happen in the working group, from initial draft to last call.

-- Christian Huitema