Re: [Rfcplusplus] Conversation as metaphor

[Martin Thomson <[e-mail]>]

On Thu, Jul 12, 2018 at 10:20 AM Brian E Carpenter
<[e-mail]> wrote:
> On 12/07/2018 12:04, S Moonesamy wrote:
> > reader is not interested in the review process.  One of the intrinsic
> > property of a RFC is availability.  Another property, if I can
> > describe it as such, is that a RFC may be viewed as authoritative.
>
> But falsely so, according to the boilerplate in many RFCs. However,
> I'm not sure this is different from academic articles: whether or
> not they are authoritative depends on many factors, and each reader
> has to make their own judgment. Being labelled "RFC", or being published
> in "IEEE Transactions", is only a starting point.

I have come to a slightly different conclusion to this.  The value of
a thing that is primarily a specification derives largely from how
closely it accurately describes reality.  As Brian (T) observes, an
academic publication might only be concerned with a superficial
treatment of the subject and so there are plenty of cases where any
reference will do (I theorize that this is what dramatically inflates
citation counts on certain documents).

For a document to be valuable beyond that simple purpose, it needs to
describe reality accurately.  To the extent that the IETF publishes
documents that are valuable, their congruence with reality is the
primary source of that value.  In many cases, we don't really get to
be sure about that until afterwards, because the documents are written
as a promise.  But one strength of our process is that the delivery on
promises is pretty high.

For many years, the value of certain specifications that I worked on
was diminished by neglect.  The draft said one thing, but practical
constraints meant that you had to do something else completely.
Whether that was to achieve interoperability, the expected level of
security, or whatever the reason, that had the effect of undermining
the credibility of those RFCs.

I realize that I'm about to expand the problem scope further, but...
We tend to hit more problems when we have aspirational protocol
specifications.  That is, protocols without a promise.  We're not
consistently able to distinguish between objectively good designs and
designs that will be deployed, partly because that requires the use of
time travel.  We do better with protocols that deploy before
publication, for example.

This is something that discussions of the level of review don't really
help with.  I consider the value of the RFC series to derive primarily
from its ability to accurately describe some part of the state of the
Internet.  Peer review in the sense of what an academic publication
demands doesn't serve that goal.  Current IETF processes and culture
are somewhat geared toward that outcome, which is what I believe has
produced this reputation.

That's not necessary a shared goal across the streams.

I suspect that the outcome here may depend more on whether you think
that the RFC series is a venue or a descriptor.  Conceived of as a
journal or publication outlet, I can see how people would conclude
there being no problem with the current situation.  This is merely
another place that you go to witness or participate in a discussion.
However, as a label or descriptor, the current situation isn't ideal.