Re: [spring] Proposed policy on reporting implementation and interoperability

Ketan Talaulikar <ketant.ietf@gmail.com> Fri, 19 August 2022 04:28 UTC

Return-Path: <ketant.ietf@gmail.com>
X-Original-To: spring@ietfa.amsl.com
Delivered-To: spring@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BDDC1C152582 for <spring@ietfa.amsl.com>; Thu, 18 Aug 2022 21:28:37 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.104
X-Spam-Level:
X-Spam-Status: No, score=-2.104 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sRknog9MyRUU for <spring@ietfa.amsl.com>; Thu, 18 Aug 2022 21:28:33 -0700 (PDT)
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6B87FC14CE40 for <spring@ietf.org>; Thu, 18 Aug 2022 21:28:33 -0700 (PDT)
Received: by mail-ej1-x62e.google.com with SMTP id i14so6718116ejg.6 for <spring@ietf.org>; Thu, 18 Aug 2022 21:28:33 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=P/bPy5mkF5IX+pdCijbLLz0Ki9Ky8UFesRIQRkd4Ncc=; b=RdCVY3ICXyKXQVGTOPhjEFe9QqxGrRLDsB1k2oKzGcfw/DyQuuEYEYQ+IIiMialesK KJX7FvN2ES9ZkSjeXSHiZRUWvJC+NJ+ueXQjGyn5jCTkz7CCCCghtnuYru9WZIfuog32 eVremryeWMYd73QMpfT2ilVp1Inmm2iST0w7pxoE77fySiBKd62XP0lR0bVXy2dkjtdh paEpoBQyzMHex9Qy/2Uey35G+0gc0Pazk4A1j0fgmJGcfERyB3DZbjEFzpOnyBehlI4U 4Iu2DeFbn24rxclD9YWbNHNqOsTJsd0K9VOOZdmhfC8MzvFkoT0H7WZkkXiXKJ1AxWNT 8GQw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=P/bPy5mkF5IX+pdCijbLLz0Ki9Ky8UFesRIQRkd4Ncc=; b=Z8S7Y3/8VX8e5LdQlXT/oNHpjzUiipMRAOwX+omDP/URS37MJ9vQcy2B3rd758NEeH /x3C9jvH5jvrRGcXbUbKKlUy7rbgGD5sTeo3FN3hkH7enkI8yxb/F1B9pwaHqlUIsQC7 aEzFMRcM1vi8xii7LIEE12yc3ZAv/tMAdh0hiZBQoK1fCPPGLWxvkfvPXh8t14oA1KMR 1rFvRaZLg6GwYaCpP0sg0LZTJxQBaaYCPIen75B7z/qBoAiUE0HqdQkzJK42l9+4/VVM TGRUf8tnB+4nCIqC2b2v1o4oN/5NajFq5evP5y+NTjkm9so1Gy419AFOFlTId36dtw8i DWxw==
X-Gm-Message-State: ACgBeo1afzW2dntAI6fXa6R/SU66fKYrymyQIyuLrZ6hCvSHGEo4onZR flOXryDlUulM5kzMKfn+yP50TiXOsOQNiQa16iSAGNr3
X-Google-Smtp-Source: AA6agR55E36VlATwDUJ3XzZZrt4Fwvob8PqSwa80z3QJqJ9o2WpHrPXXgW2tZI//mHyQwp58YAJpM04ZPf/hT7mRjoc=
X-Received: by 2002:a17:907:7287:b0:73c:f216:fd0c with SMTP id dt7-20020a170907728700b0073cf216fd0cmr834107ejc.83.1660883310994; Thu, 18 Aug 2022 21:28:30 -0700 (PDT)
MIME-Version: 1.0
References: <9c7ac280-c1f7-956c-cdbb-2b0745aaf2fa@joelhalpern.com>
In-Reply-To: <9c7ac280-c1f7-956c-cdbb-2b0745aaf2fa@joelhalpern.com>
From: Ketan Talaulikar <ketant.ietf@gmail.com>
Date: Fri, 19 Aug 2022 09:58:19 +0530
Message-ID: <CAH6gdPzYmxoVbOaAp3waUWFOCJPVoW1R3iN75SoqimyrkY769w@mail.gmail.com>
To: "Joel M. Halpern" <jmh@joelhalpern.com>
Cc: SPRING WG <spring@ietf.org>
Content-Type: multipart/alternative; boundary="000000000000c3e96805e6908778"
Archived-At: <https://mailarchive.ietf.org/arch/msg/spring/nKsfb_SumrUvwro2s2Ow4ArhQaY>
Subject: Re: [spring] Proposed policy on reporting implementation and interoperability
X-BeenThere: spring@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: "Source Packet Routing in NetworkinG \(SPRING\)" <spring.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/spring>, <mailto:spring-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/spring/>
List-Post: <mailto:spring@ietf.org>
List-Help: <mailto:spring-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/spring>, <mailto:spring-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 19 Aug 2022 04:28:37 -0000

Hi Joel/All,

Can the policy clarify some of the following points which are not
explicitly covered in its currently proposed text?

a) Whether a single implementation is sufficient or if we require at least
2 *independent* (i.e., by different implementors) ones?
b) There are some MUSTs that are associated with optional features (i.e.,
the MUST come into play only when someone is implementing that specific
option). Should there not be a distinction between the basic/mandatory and
optional MUSTs?
c) SHOULDs fall between mandatory and recommendations. In most cases, they
are quite important/critical and at almost the same level as the MUSTs when
it comes to functionality. Should they not be covered similarly to the
MUSTs?

Please find inline below some observations/comments/feedback on this
proposed policy.


On Wed, Aug 3, 2022 at 8:15 PM Joel Halpern <jmh@joelhalpern.com> wrote:

> SPRING WG:
>
> At the suggestion of our AD, the WG Chairs have been discussing whether it
> would be helpful to be more explicit, in I-Ds and RFCs we produce, about
> the announced implementations and known interoperability tests that have
> occurred.
>
KT> Looking at the SPRING WG charter, the work done here is mostly
architectural in nature and the necessary protocol extensions to actually
deliver the feature(s) are done in other WGs. Does it not make sense to
extend or apply this policy across WGs (at least those in the RTG area that
actually manifest SR solutions)? Without that, I question if such a
policy's application to SPRING WG alone is effective. Has there been any
discussion on this policy amongst the RTG ADs and other RTG area WG chairs?
If so, would be good to get some insights into the wider thought process
(if there is one) from our ADs.


> If the WG agrees, we would like to institute and post on the WG wiki the
> following policy.  The period for discussion and comment runs until
> 9-Sept-2022, to allow for folks who are on summer break:
>
> All I-Ds that reach WG last call shall have an implementation section
> based on, but somewhat more than, that described in RFC 7942 (BCP 205,*
> Improving Awareness of Running Code: The Implementation Status Section*).
> Authors are asked to collect information about implementations and include
> what they can find out when that information is available for public
> disclosure.  Documents will not be blocked from publication if the authors
> fill in the section as "none report" when they have made an effort to get
> information and not been able to.
>
KT> The proof of the pudding is not in its recipe but in actually making
and then eating it :-) Therefore, I strongly support the inclusion of an
Implementation Section in the WG documents per RFC7942. It is a very good
idea to mandate this as part of SPRING WG policy.

There are a couple of important additions to what is called for in RFC
> 7942.  We have confirmed with leadership that these changes are acceptable
> in terms of IETF process:
>
> 1) We will retain the implementation status section when the draft is
> published as an RFC.  In order to do so, the section will begin with "this
> is the implementation status as reported to the document editors as of
> <date>"
>
KT> I am strongly opposed to this since such "transient" and "changing"
data (even with a date stamped) is unsuitable for publication in the form
of an RFC. Instead, I recommend that the WG follow
https://www.rfc-editor.org/rfc/rfc7942.html#section-3. We can pick any/all
of the options and make them available for consideration.

KT> Another aspect to consider is that publishing these details in the RFCs
may give an advantage to some vendors who are the first to implement (for
whatever reasons) and hence get mentioned in the RFC while some other
vendor that implements the feature post-publication has no chance to get on
the RFC. This might be seen as incentivizing vendors that are in a position
to implement first.


> 2) Each implementation description MUST include either a statement that
> all MUST clauses in the draft / RFC are implemented, or a statement as to
> which ones are not implemented.
>
KT> As someone who has been an author/editor of IDR WG documents, I can
share that getting this level of information has been a rather tedious and
challenging process. It may be a good target to set out - though I am not
sure about making this mandatory. My concern is that we might not always be
able to get this level of details and then the question will arise before
the WG on how long to delay the publication. Note that we do architecture
work in SPRING and the protocol extensions are done in other WGs. What
happens when those other WGs don't have a similar/consistent policy and the
protocol specs that are dependent on the SPRING WG architecture doc get
stuck awaiting implementation report details?


> 3) each implementation description may include reports of what optional
> elements of the draft / RFC are implemented.
>
KT> Same as the previous comment.

> Reports of interoperabiity testing are strongly encouraged.  Including the
> reports in the document is preferred.  This may include a reference to
> longer and more detailed testing reports available elsewhere.  If there are
> no reports of interoperability tests, then the section MUST state that no
> such reports were received.
>
KT> Capturing pointers to or details of interoperability testing in the
implementation report is a good idea. Subject to my response to (1) above.
Interoperability testing at forums like the EANTC is an ongoing (yearly)
process. More/new vendors get added as the feature matures and gets wider
adoption. Putting this into an RFC publication does not make sense.

Thanks,
Ketan

Yours,
>
> Bruno, Jim, and Joel
>
>
> _______________________________________________
> spring mailing list
> spring@ietf.org
> https://www.ietf.org/mailman/listinfo/spring
>