Re: [spring] Proposed policy on reporting implementation and interoperability

["Eric Vyncke (evyncke)" <[e-mail]>]

Bruno, Jim, Joel,

Without any hat, or only with the experience of reviewing so many I-Ds from different areas and having implemented a couple of proof-of-concept implementations of various IETF drafts.

The following are just comments for discussion, not criticisms because indeed running code is important at the IETF.

RFC 7942, an IETF stream BCP (i.e., with IETF consensus), is already useful but *optional* while the proposal below would be mandatory. Hence, I share Dhruv’s question: will it also apply to experimental or informational documents or BCP? Joel, thank you for your reply to Dhruv, it seems a sensible solution.

You may be aware of a long-standing project/idea/wish of the IESG to have a *living document concept* that could be updated for years after RFC publication. As you can imagine there are too many gears to make fast progress on this topic, but the current proposal is to move the ‘living’ part of the doc to a github repo under the IETF organization (e.g., done in MOPS). Wouldn’t it be a better fit than a section ‘carved in stone’ forever (even if time-stamped) in an RFC?

Getting an implementation could sometimes be easy in Python or Go in Linux at a hackathon  ;-), or in “slow path”, a little more complex in P4, even more in silicon... the ultimate step is of course a *deployment* outside of a virtualized lab:, i.e., in a production network. Of course, deployment (and interoperation) in a production network is the Graal ;-) But, where to put a useful but realistic needle *without delaying* the standardization process ?

Finally, while I understand that having a detailed (up to the MUST level) description could be useful, this may also quickly change over time and will probably require too much of work by the I-D authors and implementers. The IETF should also be very careful in the wording of this section as the IETF is not a compliance testing lab, i.e., it is not because a published RFC claims that vendor foo has implemented protocol bar that it has really implemented protocol bar.

Again, written w/o any hat and hoping to help a useful discussion

Regards

-éric

From: spring <[e-mail]> on behalf of Joel Halpern <[e-mail]>
Date: Wednesday, 3 August 2022 at 16:56
To: SPRING WG List <[e-mail]>
Subject: [spring] Proposed policy on reporting implementation and interoperability


SPRING WG:

At the suggestion of our AD, the WG Chairs have been discussing whether it would be helpful to be more explicit, in I-Ds and RFCs we produce, about the announced implementations and known interoperability tests that have occurred.  If the WG agrees, we would like to institute and post on the WG wiki the following policy.  The period for discussion and comment runs until 9-Sept-2022, to allow for folks who are on summer break:

All I-Ds that reach WG last call shall have an implementation section based on, but somewhat more than, that described in RFC 7942 (BCP 205, Improving Awareness of Running Code: The Implementation Status Section).  Authors are asked to collect information about implementations and include what they can find out when that information is available for public disclosure.  Documents will not be blocked from publication if the authors fill in the section as "none report" when they have made an effort to get information and not been able to.

There are a couple of important additions to what is called for in RFC 7942.  We have confirmed with leadership that these changes are acceptable in terms of IETF process:

1) We will retain the implementation status section when the draft is published as an RFC.  In order to do so, the section will begin with "this is the implementation status as reported to the document editors as of <date>"

2) Each implementation description MUST include either a statement that all MUST clauses in the draft / RFC are implemented, or a statement as to which ones are not implemented.

3) each implementation description may include reports of what optional elements of the draft / RFC are implemented.

Reports of interoperabiity testing are strongly encouraged.  Including the reports in the document is preferred.  This may include a reference to longer and more detailed testing reports available elsewhere.  If there are no reports of interoperability tests, then the section MUST state that no such reports were received.

Yours,

Bruno, Jim, and Joel