Re: "Discuss" criteria

[Jari Arkko <[e-mail]>]

Hi Dave,

Joel and Sam already sent responses that I agree with,
but I wanted to add a couple of things. First of all, most
of the things that the IESG looks for in its review are
judgment calls. Its our job to think about whether, for
instance, underspecification problems in the document
are so bad that it calls for a Discuss even when we know
that interoperable implementations exist. Such a
situation should be rare, and I think most of us value
implementation experience very highly in all stages
of the IETF process. But I remember at least one
case from this year where "look, my implementation works"
claims from the proponents of a protocol were countered
with "no, its broken" from others, and it turned out that
the latter people were right. In any case, if the things
that IESG looks for weren't judgment calls, I would be
calling the tools team to work on the Discuss tool
and have the IESG do something else...

Secondly, when one of us makes a Discuss, there
is significant pressure to keep it within the Discuss
criteria guidelines, make it actionable, and explain
it in a believable manner with justifications. The
first test is whether the rest of the IESG believes
in it. Brian, for instance, gives a lot of feedback on
the other AD's Discusses, and this results often
in changes or removal of the Discuss. To survive
a laugh test in the telechat a Discuss has to have
some substance. Of course at the end of the day
we may disagree on whether there is a problem,
but my experience is that Discusses that stay
have well explained justifications.

Also, you wrote:
> Meta-point:
>
>      Something quite basic that is missing from the draft on
>      Discuss Criteria is a requirement that any Discuss not only
>      explain its precise normative basis, but that it give a clear
>      statement of what actions must be taken to clear the Discuss.
The Discusses have to be actionable in the sense
that author must know what specific problem there
is, and he needs to be able to fix it in this document.
But I hope you are not saying that the AD has to find
the fix...

Finally, I think we should just ship the discuss criteria
document. We tend to hold on to documents for too
long in the expectation that we might find an
improvement later. But I think the right documentation
format for this type of documents is IONs.

--Jari

Dave Crocker kirjoitti:
>> Dave is probably correct that the specific criteria are of broader
>> interest than just ADs, WG chairs, editors, and process wonks, and
>> might become even more perfect with broader review, but that's
>> another issue.
>>
>> And, since the criteria are public, I'm sure the IESG would be
>> interested in feedback on the criteria, especially now that WGs and
>
>
>
> Meta-point:
>
>      Something quite basic that is missing from the draft on
>      Discuss Criteria is a requirement that any Discuss not only
>      explain its precise normative basis, but that it give a clear
>      statement of what actions must be taken to clear the Discuss.
>
>
> From the draft:
>
>> * The specification is impossible to implement due to technical or
>> clarity issues.
>
> When this assertion is made, is it sufficient to cite existing
> implementations
> based on the current version of the specification?  Is the AD at least
> required to explain the assertion in detail?
>
>
>> * The protocol has technical flaws that will prevent it from working
>> properly, or the description is unclear in such a way that the reader
>> cannot understand it without ambiguity.
>
> Will demonstrations of interoperability be sufficient to counter this
> claim?
>
>
>> * It is unlikely that multiple implementations of the specification
>> would interoperate, usually due to vagueness or incomplete
>> specification.
>
> Will demonstrations of interoperability be sufficient to counter this
> claim?
>
>
>> * Widespread deployment would be damaging to the Internet or an
>> enterprise network for reasons of congestion control, scalability, or
>> the like.
>
> Beyond the simple assertion of this claim, what requirements are there
> for the AD to substantiate it?
>
>
>> * The specification would create serious security holes, or the
>> described protocol has self-defeating security vulnerabilities (e.g.
>> a protocol that cannot fulfill its purpose without security
>> properties it does not provide).
>
>
>> * It would present serious operational issues in widespread
>> deployment, by for example neglecting network management or
>> configuration entirely.
>
> There is often a failure to distinguish between new and peculiar
> problems created by a particular specification, versus general
> problems that already exist.
>
> A classic example of this is citing basic DNS problems, for
> specifications that are merely consumers of the DNS and, hence, are
> not creating any new problems.
>
>
>> * Failure to conform with IAB architecture (e.g., RFC1958 (Carpenter,
>> B., “Architectural Principles of the Internet,” June 1996.) [2], or
>> UNSAF (Daigle, L., “IAB Considerations for UNilateral Self-Address
>> Fixing (UNSAF) Across Network Address Translation,” November 2002.)
>> [3]) in the absence of any satisfactory text explaining this
>> architectural decision.
>
> This is an interesting item.
>
> 1.  At what point in time did publication of IAB preferences take on
> the force of law?
>
> 2.  Note that the list gives some examples, but does not supply a
> complete list. How are working groups to know which IAB documents have
> been declared normative standards for all IETF work and which have not?
>
> 3.  Why is the IAB allowed to create normative standards that cover
> all IETF work, without requiring that they first gain IETF-wide approval?
>
>
>> * The specification was not properly vetted against the I-D
>> Checklist. Symptoms include broken ABNF or XML, missing Security
>> Considerations, and so on.
>
>> * The draft omits a normative reference necessary for its
>> implementation, or cites such a reference merely informatively rather
>> than normatively.
>
>> * The document does not meet criteria for advancement in its
>> designated standards track, for example because it is a document
>> going to Full Standard that contains 'down references' to RFCs at a
>> lower position in the standards track, or a Standards Track document
>> that contains only informational guidance.
>
>> * IETF process related to document advancement was not carried out;
>> e.g., there are unresolved and substantive Last Call comments which
>> the document does not address, the document is outside the scope of
>> the charter of the WG which requested its publication, and so on.
>
>> * The IETF as a whole does not have consensus on the technical
>> approach or document. There are cases where individual working groups
>> or areas have forged rough consensus around a technical approach
>> which does not garner IETF
>>  consensus. An AD may DISCUSS a document where she or he believes
>> this to be the case. While the Area Director should describe the
>> technical area where consensus is flawed, the focus of the DISCUSS
>> and its resolution should be on
>>  how to forge a cross-IETF consensus.
>
> This is perhaps the scariest of the criteria.  It says that a
> knowledgeable, motivated constituency can spend months on solving a
> problem that it needs to have solved, and then others who have not
> participated in the work can come along and sabotage it.
>
> Yes, I know that is not the intend of this criterion, but it is what
> the effect will be -- and in some cases already is.
>
> Once upon a time, the rule in the IETF was that working group
> consensus was what mattered, absent a clear technical basis for saying
> that a specification "would not work".
>
> That is quite different from now saying that after a working group is
> done we somehow must be assured that the specification has acquired an
> IETF-wide politically correct acceptance.
>
> If this last criterion is meant to be taken as it is stated, there is
> a pretty straightforward basis for believing that ever bringing new
> work to the IETF is a very questionable decision.
>
> The criterion presumes that the IETF, as a whole, must agree on the
> One True Solution to a problem.
>
> In the IETF's history, that has been a requirement in some special
> cases, but not others.  The default view has been to let the market
> decide among choices. Requirement for a single choice has been
> asserted only when there is a strong argument that having multiple
> choices will cause damage.
>
> d/


_______________________________________________
Ietf mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/ietf