Re: document requirements and IETF procedures (was: Re: polishing of draft-ietf-dnsext-rfc3597-bis-01)

[SM <[e-mail]>]

Hi Alfred,

[Follow-up to the IETF discussion list as you 
suggested.  The original message is from the namedroppers mailing list.]

At 06:02 19-02-10, Alfred Hönes wrote:
>Regarding openness, I have more serious concerns.
>(RFC-to-be's held off by IESG intransparently, etc.... )
>But that is off-topic here.
>
>I'm not sure where I last had read that phrase (flame?) above -- and it
>essentially doesn't matter here --, I bet it was one of the procedural
>drafts from Brian Carpenter in which he tried to overcome the road
>blocks to revising RFC 2026 and adopting it to long standing practice.
>
>I am tempted to quote a COMMENT posted yesterday by an AD, with
>respect to a technical document in the IESG and the IETF procedures
>followed in its (6 years lasting) genesis, and I challenge its more
>general applicability to the meta level of IETF procedural standards
>as well  (and so I've elided from the quote all keywords related to
>the triggering circumstances):
>
>    "[...]
>     This would seem to imply that the .... WG has decided to
>     deviate from the old IETF operating principle of "rough
>     consensus and running code".  For at least some of the
>     techniques described in this draft, they are generally
>     accepted and widely implemented on key implementations.
>     I ask what the reason is for divorcing IETF standards
>     from established best practices and actual running code?
>     ... RFCs are not sacred documents, they should reflect what
>     we want our implementations to do.  But maybe there are
>     important use cases for the actual standard ... behavior
>     in this space, just that I don't know about them.  Please
>     educate me about the background for this decision."

I am going to quote large extracts from RFC 3774.

   "The IETF is unsure what it is trying to achieve and hence cannot
    know what its optimum internal organization should be to achieve
    its aims."

   "Externally, the IETF is often classified with these conventional
    SDOs, especially by detractors, because the differentiation in the
    IETF's mission and processes and the rationale for those differences
    are not clear."

   "A major symptom of this lack is that WGs do not consistently produce
    timely, high-quality, and predictable output."

   "Failure to identify and articulate engineering trade-offs that may
    be needed to meet the deadlines that the WG has set without
    inappropriately reducing the 'fitness for purpose' for the
    intended customers."

   "Continued refinement of the solution beyond the point at which it
    is adequate to meet the requirements placed on it by the intended
    purpose."

We could call it mission creep.

   "Poorly defined success criteria for WGs and individual documents."

   "Lack of auditing against explicit criteria throughout the
    standards development process."

   "Lack of review, especially early review, by reviewers who are not
    directly interested members of the WG, and by subject matter
    experts for topics related to, but not necessarily the immediate
    focus of the document."

   "Lack of criteria for determining when a piece of work is
    overrunning and/or is unlikely to be concluded successfully,
    either at all or within an acceptable time frame."

   "One problem which the IETF does not appear to suffer from is
    excessive bureaucracy, in the sense that transfer of information is
    generally kept to the minimum necessary to accomplish the task.  It
    is important that any changes introduced do not significantly
    increase the bureaucratic load whilst still recording sufficient
    information to allow process improvement."

I note that the IESG provides a wealth of 
information nowadays through its minutes.  The 
same cannot be said of Working Groups.  The minima nowadays is XMPP logs.

   "One area of particular concern is the tendency for protocols to be
    assessed and issues resolved primarily through static analysis of the
    written specification rather than by practical experiment with
    'running code'."

It's one thing to read a 
specification.  Implementing it provides a 
different view as we can come across use-cases we 
would never have thought about.

   "The current hierarchy of Proposed, Draft, and Full Standard maturity
    levels for specifications is no longer being used in the way that was
    envisioned when the stratification was originally proposed. In
    practice, the IETF currently has a one-step standards process that
    subverts the IETF's preference for demonstrating effectiveness
    through running code in multiple interoperable implementations."

I guess that it should be called ossification instead of stratification.

   "It is widely perceived that the IESG has 'raised the (quality)
    bar' that standards have to pass to be accorded a PS status."

That's what happens when the RFC brand is considered as more important.

   "There appears to be a vicious circle in operation where vendors
    tend to deploy protocols that have reached PS as if they were
    ready for full production, rather than accepting that standards at
    the PS level are still under development and could be expected to
    be altered after feature, performance, and interoperability tests
    in limited pilot installations, as was originally intended.  The
    enthusiasm of vendors to achieve a rapid time to market seems to
    have encouraged the IETF in general and the IESG in particular to
    attempt to ensure that specifications at PS are ready for prime
    time, and that subsequent modifications will be minimal as it
    progresses to DS and FS, assuming effort can be found to create
    the necessary applicability and interoperability reports that are
    needed."

Maybe by trying to achieve perfection at Proposed 
Standard, the IETF has involuntarily encouraged 
that.  It is difficult to make changes after that 
as there are deployment considerations then.

   "The periodic review of protocols at PS and DS levels specified in
    are not being carried out, allowing protocols to persist in
    these lower maturity levels for extended periods of time, whereas
    the process would normally expect them to progress or be relegated
    to Historic status."

We accept long-lived experiments instead of 
putting in a timer.  Experimental status is a 
facing saving device.  Proposed Standard is the 
equivalent of "the IETF said so".  We reinterpret 
the text as a holy book.  Naturally, that leads 
to divergent interpretations.  That gives a new meaning to protocol maturity.

   "the specification is not updated to reflect parts of the
    specification that have fallen into disuse or were, in fact, never
    implemented."

As the years go by people forget why the text was written in such a way.

   "Although there may be large attendance at many WG meetings, in
    many cases, 5% or less of the participants have read the drafts
    under discussion or that have a bearing on the decisions to be
    made."

   "Little or no response is seen when a request for 'last-call'
    review is issued, either at WG or IETF level."

Fortunately there is the GEN-Art review and 
reviews by directorates from specific areas or 
else the cross-area review would be in name 
only.  Maybe consensus should be redefined as "lazy consensus".

  "The IESG members are overworked which is bad for their health,
   humor, and home life"

  "It appears that the IETF is showing a disturbing tendency to
   turn IESG 'rules of convenience' into rigid strictures that
   cannot be violated or deviated from."

  "Newcomers to the organization and others outside the affinity group
   are reluctant to challenge the apparent authority of the extended
   affinity group during debates and consequently influence remains
   concentrated in a relatively small group of people."

The old-boys (and girls) club is sometimes 
perceived as a subset of that affinity group.

   "Participants are frequently allowed to re-open previously closed
    issues just to replay parts of the previous discussion without
    introducing new material.  This may be either because the decision
    has not been clearly documented, or it may be a maneuver to try to
    get a decision changed because the participant did not concur with
    the consensus originally."

   "One cause that can lead to legitimate attempts to re-open an
    apparently closed issue is the occurrence of 'consensus by
    exhaustion'."

   "The IETF culture of openness also tends to tolerate participants who,
    whilst understanding the principles of the IETF, disagree with them
    and actively ignore them.  This can be confusing for newer
    participants, but they need to be made aware that the IETF does not
    exclude such people."

Regards,
-sm