RE: Last Call: <draft-housley-implementer-obligations-01.txt> (Expectations of Implementers of IETF Protocols) to Informational RFC

[<[e-mail]>]

Even the title is ambiguous: Expectations *for* implementers...

Obligations (in the filename)? there are no obligations.

Discussion of Postel's law is simplistic; interop is often improved by exactness, and debugging when being liberal can be an exercise in frustration.

Not sure I see the point of this doc. Implementers often read specs in isolation, they won't see this unless it's a mandatory normative reference. And it's not good enough to be that or BCP. 'Follow the BCPs' could be read as 'follow the doc references', and this won't be one of them.

What Elwyn said.

Lloyd Wood
http://about.me/lloydwood
________________________________________
From: ietf <[e-mail]> on behalf of Elwyn Davies <[e-mail]>
Sent: Monday, 12 May 2014 9:53 AM
To: Brian E Carpenter
Cc: [e-mail]; [e-mail]
Subject: Re: Last Call: <draft-housley-implementer-obligations-01.txt> (Expectations of Implementers of IETF Protocols) to Informational RFC

Hi.

I have some sympathy with both Brian's and Adrian's points of view on
this document.

I am with Brian in that I think there is a useful document buried in
here and trying to get out.

I am with Adrian in that it is currently relatively content free and, to
my mind, has both the wrong 'tone of voice' (particularly the abstract,
but generally) and misrepresents what the IETF is trying to do.
Specifically I would not use 'expectation' but try to explain the
(three) strands that the IETF provides to smooth the implementers path
to a useful and interoperable implementation.

I also think there are some important points that could be added to make
it less platitudinous:
- We are assuming (I believe) that the implementer isn't first up with
their implementation - probably ought to make this clearer as the
intende audience - and so there will be one or more existing
implementations out there.  New implementers should look at them for
various reasons.
- RFCs contain requirements language:  MUST etc are clear, but what
about SHOULD/RECOMMEND and MAY?  The (naive) implementer deserves some
advice here.  S/he needs to think about any SHOULDs... Why would the
implementation not follow the SHOULD? Maybe because 'everybody else'
ignored it as well (in the prior implementations), but see the next
point.  MAYs also need to be considered... they can represent options.
What did the prior implemntations do? Is this the right choice for *this
implementation*?
- What are the implementer's aims? A tool for some target audience? The
best and most comprehensive implementation ever? A basis for pulling the
protocol to pieces and making a better protocol?  If the first, then the
audience needs to be asked what they are expecting... they may have
views on the SHOULDs and MAYs ... or they may hedge their bets and want
configuration options.  If its the second or third, then .. you're on
your own sport! However, the implementer still needs to think about
potential users.
- A reminder that RFCs only mandate the order and meaning of the bits on
the wire, together with message flow and state machines and possibly
some configuration options that need to be provided.
- If an RFC talks about data structures in nodes, they tell you what
probably ought to be stored and a abstraction of a storage structure; it
is *not* mandatory that the implementer follows this dogmatically - if
there is a better way s/he can think of, then go for it.
- Not all prior implementations are right: It is probably worth the
implementer capturing some sample traffic and using it to verify whether
it matches the specification.  In a very few cases, we say through
gritted teeth that the implementation is right.  In some other cases
(TCP - RFC 793 - being a case in point) significant parts of the
specification have never been implemented or have fallen into disuse:
Regrettably this is not easy to cope with but is less true of more
recent specs.
- Many RFCs are *very complicated*: the captured traffic can also be
used to check whether the prospective implementer has understood the
specification correctly.
- The IETF doesn't provide text specs: Plan your testing as you go
along.
- Oh, and check the ERRATA!

Regards,
Elwyn


On Sun, 2014-05-11 at 08:31 +1200, Brian E Carpenter wrote:
> Hi Adrian,
>
> On 11/05/2014 06:34, Adrian Farrel wrote:
> > While I appreciate the effort the author put in to resolve previous
> > discussions, I do not support the publication of this document.
> >
> > The thrust of my previous comments were to say "It is all platitude,
> > but probably harmless" and "at whom is this document aimed". This
> > review is in a little more detail. I still find the whole document to
> > be one big platitude that does not need to be published,
>
> I can certainly see how you could reach that view, but I think you're
> looking at it from the viewpoint of somebody who's been around the
> IETF for a long time and knows what we do here and why we do it.
>
> I like to think of somebody else: a young programmer working far,
> far away, who will probably never attend an IETF meeting or join
> an IETF mailing list. For this person, we need to state things that
> are obvious to us. For example:
>
> "It is not sufficient to do an initial implementation of the protocol.
>  Maintenance is needed to apply changes as the come out in the future,
>  especially to fix security issues that are found after the initial
>  publication of a protocol specification."
>
> That isn't a trivial statement. It says that a protocol design may have
> zero-day vulnerabilities and if you implement it, it's your job to
> watch out for future updates and apply them. Is that going to be obvious
> to someone starting a garage company in Africa?
>
> I think the document is valuable.
>
> One specific comment: I think it should be mentioned that implementers
> should always read the references cited in a specification, especially
> but not only the normative references, and apply them as relevant.
>
> I also suggest mentioning implementation of extensions. Faulty extensions
> are harmful to interoperability and security. We have a couple of RFCs
> about that too (4775 and 6709).
>
>    Brian
>
> > but I have also
> > found a number of things that I think need to be fixed.
> >
> > ---
> >
> > This document significantly conflates advice to implementers wishing to
> > ensure interoperability, best practice for people claiming to have
> > implemented (i.e., claiming conformance to) a specification, and
> > constraints to the freedom for implementers of IETF specifications.
> >
> > As the text notes, IETF specifications (please don't call them standards
> > unless that is what you really mean) are not mandatory to implement. So
> > the text really must not tell people what they must or must not do. For
> > example, the Abstract says "By choosing to implement..." This is
> > nonsense! I can choose to do what I like. If I choose to implement stuff
> > and tweak it and make it better, that is entirely my choice.
> > Maybe you could have said "By claiming to have implemented..." But even
> > that is marginal. We are not the Internet Police and we have no
> > influence in the world of advertising or marketing. Nor do we run
> > conformance labs. This attempt at constraining implementations is bogus
> > and needs to be removed from the document.
> >
> > As far as interoperability is concerned, it is great to give concrete
> > advice. When the Introduction says
> >    This document provides advice to implementers of IETF protocols to
> >    improve interoperability of their implementations.
> > it would be wonderful if the document lived up to the claim. But this
> > claim seems to be at odds with the document Title and Abstract and all
> > I find in the document is effectively "If you want two implementations
> > to interoperate, they need to implement the same thing." Well, if that
> > comes as a surprise to anyone perhaps they are in the wrong business.
> >
> > So what is the document actually trying to do and say, and to whom?
> >
> > ---
> >
> > The document also contains a lot of passive voice that hides the
> > motivation for the text. For example, the Abstract says "...one is
> > expected..." Expected by whom? Why? Perhaps you can attribute the
> > expectation to make this meaningful.
> >
> > ---
> >
> > Introduction
> > "IETF protocols foster interoperability."
> > I don't believe this is true. A protocol cannot of itself achieve this.
> > Possibly the clear specification of a consensus-based protocol can do
> > that. Possibly.
> >
> > ---
> >
> > Introduction
> >    Yet, IETF standards are
> >    voluntary standards.  No one is required to implement them.
> >    Implementation is a choice.  By making this choice, an implementer is
> >    expected to:
> >
> >       (1) Follow the protocol specification;
> >
> > Please clarify "IETF standards" since you almost certainly don't mean
> > what you have written.
> >
> > But note that this text is circular. Implementation of a specification
> > is, by definition, following the specification. So this text doesn't
> > say anything! It is the *claim* of implementation that has an
> > expectation attached to it. What I do in the darkness of my own room is
> > not a matter for anyone's expectations.
> >
> > ---
> >
> > Introduction
> >
> >    When implementers meet these expectations, protocols interoperate as
> >    intended by the IETF.
> >
> > This is a mixed message. Are you trying to set out the expectations on
> > people who implement (or claim implementation), or are you giving advice
> > on how to achieve interoperability?
> >
> > ---
> >
> > Section 2
> >
> >    An implementer needs to maintain their implementation
> >    into the future.  It is not sufficient to do an initial
> >    implementation of the protocol.  One needs to apply changes as they
> >    come out.
> >
> > While the example given is highly desirable, it does not go as far as
> > "needs to". An implementation is a snapshot, a moment in specification
> > history, and is correctly described as such. There is no moral or other
> > binding on the implementer to make a change, just as there was no
> > requirement that the implementer select a particular specification to
> > implement.
> >
> > When I implement a protocol as specified in an RFC I am not making a
> > commitment to update my implementation to fix bugs in the specification
> > or to add features.
> >
> > Furthermore, not all protocol extensions are desirable in all
> > environments. There is no requirement for an implementer to add a widget
> > as there is no requirement for an implementer to implement version 2 of
> > a protocol or to pick up fixes to version 1.
> >
> > ---
> >
> > Section 3
> >
> > I am not comfortable with this attempt to define the purpose and meaning
> > of BCPs over and above RFC 2026. The statements in this section might
> > reflect how BCPs have been used in the past, but this text is too strong
> > in the way it looks to the future. It might be appropriate to put this
> > into the past tense...
> >
> >    Best Current Practices (BCPs) about IETF protocols (not the BCPs that
> >    define IETF processes and procedures) have often been used to
> >    document IETF consensus about operational or implementation practices
> >    pertaining to IETF protocols.
> >
> > By *your* definition of BCP, why is this document not a BCP?
> >
> > ---
> >
> > Section 3
> >
> >    By following the BCPs,
> >    implementers, operators, and administrators are able to provide a
> >    common experience when using the protocol, regardless of their point
> >    of attachment to the Internet.
> >
> > Do you mean "provide" or "obtain". If "provide" then provide to whom?
> >
> > ---
> >
> > Section 3
> >
> >    Sometimes BCPs are referenced in the protocol specification.  Often
> >    the implementer needs to look through the BCP index to find related
> >    BCPs.
> >
> > The implication here might be that by checking the list of BCPs an
> > implementer will find all of the relevant advice and guidance outside the
> > specification itself. This is not true. There are plenty of
> > Informational RFCs describing ways to build and deploy protocols. And
> > there is this (under-used) thing called an Applicability Statement
> > [RFC 2026 - section 3.2].
> >
> > Adrian
> >
> >
>