Re: Idea for a process experiment to reward running code...

[John C Klensin <[e-mail]>]

--On Monday, December 03, 2012 11:28 +0000 Stephen Farrell
<[e-mail]> wrote:

>> Encouraging running code is a Good Thing. Publishing sloppy
>> specifications is a Bad Thing.
> 
> Sure. I guess I'd hope that we push back on sloppy specs as
> usual, but that the running code might make that less likely,
> or at least more likely to be just editorial.

Stephen,

I agree with Brian, but want to reinforce thinking about this a
little differently and in a way that may draw several other
sub-threads together.  The IETF is functioning here as a
standards body, not an certification body for implementability.
Knowing that something can be implemented, and implemented at
production quality, has traditionally been an IETF hallmark (and
hence part of Dave Clark's slogan) relative to other bodies who
have standardized the un-implementable.  IMO, when we approve
something as a Proposed Standard, it suggests that we have
evaluated and approved it along three quite separate dimensions:

	(1) Utility of the idea as something that should be
	implemented (or at least as a decent and reasonable way
	to do something)
	
	(2) Quality of the protocol and whether the
	specification adequately describes it.
	
	(3) Whether the specification can be implemented

Failures are possible on any of the three.  It is possible for a
well-specified and implementable idea to be utterly useless.  It
is possible to have a useful and well-designed protocol, with
implementations by insiders, to be so badly described that it is
unlikely that anyone else could figure out how to implement it.
It is, as Brian pointed out in his recent note, possibly to have
a hack-level implementation and/or one that doesn't work in the
edge cases.

Using any of those three in a way that enables shortcuts around
the other two puts us at risk or creating bad standards.  Even a
few bad standards could be used to hold the IETF up to ridicule
in ways that would weaken everything else that we do for a long
time... and the fact that we saw them as the result of a process
experience would be no protection at all.  Worse from the
standpoint of a speed-up procedure, someone's discovering a
problem after a hasty IESG approval could easily lead to an
appeal on the grounds that the process used did not allow for
adequate review, a result that would cost far more time than the
difference between the current and proposed procedure.

As others have pointed out, there are lots of other ways to
speed things up, most of which fall within the discretion of
individual WGs, ADs, and the IESG without any changes at all.
At least a few of them would involve "changes" back to what RFC
2026 already specifies, including treating Proposed Standards as
Proposed Standards, and eliminating, e.g., days or weeks or
post-IETF-Last-Call AD nitpicking over text in favor of
practicing the intent of the "no known technical defects" rule
of 2026 and letting the RFC Editor do their job.  But "you claim
to have done consideration 3, therefore you get to shortcut
considerations 1 and 2" should not be one of them.

best,
   john