Re: [Sidrops] Requiring Two Implementations - before exit of WGLC

[Andrew Newton <[e-mail]>]

"Production-ready" can be subjective, and it is not even the standard
used to progress from Proposed Standard to Draft Standard. RFC 7127
instead uses "operational experience" which I believe is a better
metric.

Given that a proof-of-concept implementation is better than no
implementation, that may be good enough to improve the confidence in
the specifications. All that said, something does need to be done to
address Tim's concern.

Also, might all this be the purview of the IESG? Here is RFC 7127 on
this matter:

   The IESG may require implementation and/or operational experience
   prior to granting Proposed Standard status to a specification that
   materially affects the core Internet protocols or that specifies
   behavior that may have significant operational impact on the
   Internet.

-andy

On Fri, Oct 13, 2023 at 3:18 AM Ties de Kock <[e-mail]> wrote:
>
> Hi Tim,
>
> > On 13 Oct 2023, at 09:06, Tim Bruijnzeels <[e-mail]> wrote:
> >
> > Hi all,
> >
> > In principle, I agree with the proposal.
> >
> > However, I see potential issues that affect my own work on a delegated CA implementation. There are currently two open-source RPKI CA solutions that can act as a "child" CA: Krill and rpkid (by Dragon Research Labs) - there is nothing wrong with the latter, but it's not actively maintained (and yes, I wish it were - the ecosystem needs choice).
> >
> > Getting multiple implementations of the child side of possible new standard developments may be hard. Would one child implementation interoperating with another implementation acting as the parent be good enough in this context?
>
> You imply a very robust standard of interoperability here:
>
>   * Two full "production-ready” implementations
>   * Implemented by distinct teams
>
> I think requiring implementations beyond single-shot/proof of concept
> implementation of objects adds value. The single-shot/proof-of-concept
> implementations do not allow others to validate a concept because you can not
> integrate it into other systems (e.g. CA output -> continuously fed to a
> validator, including various objects).
>
> One example of an issue this chain would have caught before it hit production
> would be a recent issue of a RP implementation(s?) generating broken JSON for
> specific ASPA objects.
>
> Kind regards,
> Ties
>
> >
> > Tim
> >
> >> On 13 Oct 2023, at 00:08, Korsbaeck, Fredrik <fkback=[e-mail]> wrote:
> >>
> >> Hi,
> >>
> >> I think these implementation reports is extremely helpful fwiw, I use them in my day-to-day very often.
> >>
> >> To me, it seems like a sensible development to require running code, so +1 with Job and Ben here.
> >>
> >> /FK
> >>
> >> On 2023-10-06, 15:27, "Sidrops on behalf of Job Snijders" <[e-mail] <mailto:[e-mail]> on behalf of job=[e-mail] <mailto:[e-mail]>> wrote:
> >>
> >> Dear Andrew,
> >>
> >>
> >> On Fri, Oct 06, 2023 at 09:18:50AM -0400, Andrew Newton wrote:
> >>> On Wed, Oct 4, 2023 at 10:07 PM Chris Morrow <[e-mail] <mailto:[e-mail]>> wrote:
> >>>>
> >>>> This conversation sort of fizzled out...
> >>>>
> >>>> On Tue, 25 Jul 2023 07:27:45 +0000,
> >>>> "Dale W. Carder" <[e-mail] <mailto:[e-mail]>> wrote:
> >>>>>
> >>>>> Thus spake Job Snijders (job=[e-mail] <mailto:[e-mail]>) on Tue, Jul 25, 2023 at 04:23:53AM +0200:
> >>>>>> Proposed text:
> >>>>>>
> >>>>>> "Before SIDROPS Standards Track internet-drafts can progress to IESG
> >>>>>> review, interoperability must be demonstrated between at least two
> >>>>>> independent implementations for every aspect of the concepts in the
> >>>>>> specification. The chairs may waive this requirement when
> >>>>>> interoperability is of no concern (for example if the document is a
> >>>>>> BCP, problem statement, or requirements document).
> >>>
> >>> This is an interesting proposal, but it seems to conflate bcp,
> >>> information, and experimental with standards track. Why are chairs
> >>> waiving a requirement for documents that are not subject to the rule?
> >>
> >>
> >> Are you suggesting to remove the "(for example .. requirements document)"
> >> sentence?
> >>
> >>
> >>> It also seems to me that what is being asked for is an
> >>> interoperability report, not just implementation status. If so, "every
> >>> aspect" should be an enumerated list otherwise judging it is
> >>> guesswork. And it might be wise to consider an escape hatch for things
> >>> that might be more urgent but would otherwise get hung up on a formal
> >>> interoperability report.
> >>
> >>
> >> The phrase 'implementation report' is a reference to how the requirement
> >> for running code is handled in the IDR working group. A good example of
> >> how implementation & interopability are tracked is for example this wiki
> >> page: https://wiki.ietf.org/group/idr/implementations/draft-ietf-idr-large-community <https://wiki.ietf.org/group/idr/implementations/draft-ietf-idr-large-community>
> >> As part of the implementation report, often a (simple) interopability
> >> matrix is produced.
> >>
> >>
> >> I'd like the SIDROPS working group to take a similar approach as done
> >> here: https://wiki.ietf.org/group/idr/implementations <https://wiki.ietf.org/group/idr/implementations>
> >>
> >>
> >> Kind regards,
> >>
> >>
> >> Job
> >>
> >>
> >> _______________________________________________
> >> Sidrops mailing list
> >> [e-mail] <mailto:[e-mail]>
> >> https://www.ietf.org/mailman/listinfo/sidrops <https://www.ietf.org/mailman/listinfo/sidrops>
> >>
> >>
> >>
> >> _______________________________________________
> >> Sidrops mailing list
> >> [e-mail]
> >> https://www.ietf.org/mailman/listinfo/sidrops
> >
> > _______________________________________________
> > Sidrops mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/sidrops
>
> _______________________________________________
> Sidrops mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/sidrops