Re: [newtrk] [Fwd: I-D ACTION:draft-carpenter-newtrk-twostep-00.txt]

[Bruce Lilly <[e-mail]>]

[trimming to address the most important issues]

On Tue June 14 2005 10:29, Brian E Carpenter wrote:
> >>-- clarifies that moving up from PS is explicitly tied
> >>to interoperability

> > 2. a list of partially- or non-interoperating features (see below) is
> >    not the same as actual interoperability.  The relevant "problem"
> >    is that some people don't believe that mere markings on virtual
> >    paper -- moreover with no regard for the factual accuracy of what
> >    those markings represent -- constitute interoperability, and that
> >    "problem" is "solved" by explicitly *removing* ties to actual
> >    interoperability. 
> 
> Nothing we write is the same as actual interoperability. The proposal
> doesn't remove the ties between what we write and interoperability;
> it just moves them around a bit.

I believe that the change from BCP 9 (a.k.a RFC 2026) "'interoperable'
means to be functionally equivalent or interchangeable components of
the system or process in which they are used" and "The requirement
for at least two independent and interoperable implementations applies
to all of the options and features of the specification" to the draft
under discussion "applies to each of the options and features of the
specification considered individually" is much more than "moves them
around a bit"; it changes the character of the requirement from assessing
interoperability in terms of functional components to something which
is considerably less meaningful w.r.t. interoperability as currently
defined in BCP 9.

> >>-- removes the sense of struggling towards an unattainable
> >>goal (yet another bureaucratic step to finally reach STD status)
> > 
> > 
> > "unattainable" is too strong; as there are full standards, achieving
> > that status clearly is attainable.
> 
> So rarely that nobody cares, IMHO.

If nobody cares, why are we here?  Let the sleeping dog lie. [I believe
that "nobody" is too strong an assertion also]

> > The third bullet point addresses lack of enforcement of implementation
> > or deployment experience, and further weakening the "requirement" for
> > interoperable implementations doesn't address the lack of enforcement,
> > it codifies it -- the problem is exacerbated.
> 
> But my proposal doesn't weaken (or strengthen) that requirement.
> It just moves it around a bit.

See above.

> > The penultimate bullet point discusses failure to carry out periodic
> > reviews, and that appears to be "addressed" by the draft proposal in
> > the same manner as the standards track process itself (throwing out
> > the baby by eliminating the requirement).  As the IESG has been tasked
> > with carrying out such reviews (BCP 9), it is difficult to see this
> > as anything other than "appoints itself as both judge and jury on
> > process changes".
> 
> This isn't the IESG, it's newtrk, and newtrk is here to propose
> changes. But, with my limited experience of the IESG workload, I
> want to state as my opinion that the probability of any IESG operating
> under current conditions actually implementing such a periodic
> review is zero. What use is a provision that, as a matter of running
> code, is certain not to be executed?

Presumably the provision is there because the WG, the IETF (via Last
Call review), and the IESG felt that it was an important step in
managing the collection of specifications that constitute Standards
Track RFCs.  IESG has apparently decided to second-guess that
consensus decision by not carrying out the process specified in BCP 9.
Not merely judge and jury, but (non-)executioner as well.

> > The last bullet point discusses lack of maintenance, and the first
> > bullet point of the section notes that specifications rarely progress;
> > the practice of closing down WGs noted in the last bullet point has
> > been recognized as a major contributing factor in the lack of
> > progression, as discussed here and on the IETF discussion list. The
> > draft proposal doesn't address that issue.
> 
> It's somewhat decoupled, but I think the point is valid. On the one hand
> we don't want self-perpetuating WGs, but on the other hand if we close
> them down ASAP we lose any chance of a reasonably prompt interoperability
> review. I agree something needs to change in this area.

This is also related to the periodic reviews, since per BCP 9 the WG
Chair is responsible for documenting interoperability in a report for
advancement to DS.  No WG, no Chair, no report, no advancement.
 
> > i.e. "lowering the quality bar"
> 
> No, not in reality. Abolishing the bar that almost nobody tries to jump over
> has no particular effect at the entry level, and most documents stay at the
> entry level.

There are other factors in lack of advancement (see above).  Changing the
requirement for interoperability to something (difficult to concisely
characterize exactly what that is) considerably less related to true
interoperability certainly lowers the quality bar.  Whether or not that's
a de-facto process change vs. a de-jure process change is another matter,
outlined in detail in my June 8 "Interoperability" message, and related
to the "judge and jury" comments.

> >>-- moving to Interoperable Standard specifically may *not* require a
> >>    new RFC even if some features are listed as not shown to interoperate
> > 
> > 
> > Apparently already the de-facto situation.  Usually there is a new RFC
> > because of correction of errata, boilerplate changes, etc., but non-
> > interoperating features has not prevented progression on the current
> > Standards Track, at least recently.
> 
> Hmm. Examples?

We discussed (in the sense of BCP 9 section 6.5.2 second paragraph) the
case of advancement of the 2476bis draft to DS despite lack of
demonstrated functional component interoperability (See
http://www.ietf.org/IESG/Implementations/rfc-2476-and-draft-gellens-submit-bis-implementation.txt
and the spreadsheet matrix derived from it at
http://users.erols.com/blilly/rfc-2476-gellens-implementation.sxc
).  There is also the advancement of a 2234 successor draft to DS
despite the fact that *both* cited implementations (
http://www.ietf.org/IESG/Implementations/RFC2234-implementation-report.txt
) explicitly fail to
use the 2234 and draft provisions specifying CRLF line endings, one fails
to accept binary, decimal, and hexadecimal literals using %B, %D, %X in
spite of the fact that 2234 and the draft explicitly state case
independence of the 'b', 'd', and 'x' (it could be argued that the
specification is not sufficiently clear, since the case-independence part
is in a separate section), one accepts (illegal) empty rules, one fails
to accept a specified string literal of the form %x0D.0A.0B.  And there
are specification features known to cause confusion which have not been
used, and which could and should be removed as part of the fine-tuning
of the specification.

> > Simply listing non-interoperating features does not constitute actual
> > interoperability (see above and my June 8 list message).
> 
> No,of course not - it would however avoid the need to republish an RFC
> if only minor features had not be demonstrated.

I believe that the provision for removal of features not demonstrated as
interoperable removes features which may be used to exploit security
vulnerabilities and enhances interoperability (in the BCP 9 functional
component sense).  I further believe that that is a valuable step in
refining a specification as it progresses from entry level through
fine-tuned to cast in concrete.

> >>(Remember that my starting position was to move to a one level
> >>standards track.
> > 
> > 
> > Effectively, we already have that via BCP RFCS, which (per BCP 9) are
> > supposed to meet the requirements for a full Standard.  If -- as sometimes
> > claimed -- the quality bar has been raised to require interoperating
> > implementations at Standards Track entry, and if there is some
> > hypothetical specification which meets the requirements for full Standard
> > when the specification is prepared, there is a mechanism to give the
> > specification full Standard status in one step via BCP.
> 
> Well, BCPs are not meant for that as I read 2026.

I agree that BCP 9 specifies other uses.  However, in the hypothetical
case of a fully-fledged, truly interoperable specification with wide
deployment experience in mission-critical applications, BCP *could* be
used.  Indeed there is a proposal currently in Last Call for consideration
as BCP which may well be more suitable as a Standards Track specification
precisely because of fine-tuning and deployment issues.
 
> > There are more than 2 questions of significance.  Obviously, one could
> > ask:
> > 3. What significant problems would the proposed changes introduce?
> 
> I haven't spotted any, but maybe I have rose-tinted glasses.

I have mentioned the functional component interoperability issues, and
the current provision for removal of unimplemented potential security
loopholes/interoperability sandbars.

> > 4. Do we really care "to make the Internet work better", i.e.
> >    interoperabilty, or is the Mission Statement a hollow one?
> >    Indeed, that question is fundamental; if we do care about quality --
> >    making the Internet work better in reality -- then anything that
> >    lowers the quality bar is counterproductive.
> 
> I think that's arguable. You may make it work better by shipping something
> in a big hurry despite imperfections.

Well, I'm not sure that "a big hurry" is possible with current procedures
unrelated to the Standards Track (e.g. IESG review *after* IETF Last Call
rather than *concurrent* review, RFC Editor backlog), and the nature of
the beast (getting e.g. security considerations right takes time and
effort).  And if the "imperfections" impair interoperabilty or cause
havoc, one may well make the Internet work much worse.

> But in any case, I don't agree 
> that reducing the number of standards levels need have any adverse effect
> on the quality of our specs.

Agreed that all else being equal (particularly the quality-assurance
provisions w.r.t. interoperability and large-scale deployment), the number
of steps need not be related to quality.  However, if weakening or removing
those QA provisions is specified as (part of) the reduction of steps, then
clearly quality may suffer.
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html