Re: [netconf] Adoption poll for draft-tgraf-netconf-yang-notifications-versioning

Andy Bierman <andy@yumaworks.com> Fri, 02 June 2023 15:24 UTC

Return-Path: <andy@yumaworks.com>
X-Original-To: netconf@ietfa.amsl.com
Delivered-To: netconf@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 25243C151075 for <netconf@ietfa.amsl.com>; Fri, 2 Jun 2023 08:24:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.096
X-Spam-Level:
X-Spam-Status: No, score=-2.096 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=yumaworks.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id p4eaCFE51Zvu for <netconf@ietfa.amsl.com>; Fri, 2 Jun 2023 08:24:43 -0700 (PDT)
Received: from mail-lj1-x229.google.com (mail-lj1-x229.google.com [IPv6:2a00:1450:4864:20::229]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AC9ADC151073 for <netconf@ietf.org>; Fri, 2 Jun 2023 08:24:43 -0700 (PDT)
Received: by mail-lj1-x229.google.com with SMTP id 38308e7fff4ca-2b1b6865c7cso3145271fa.3 for <netconf@ietf.org>; Fri, 02 Jun 2023 08:24:43 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yumaworks.com; s=google; t=1685719481; x=1688311481; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=mWHdsFRgwU/DC9KYPyZpOr5kxSiGbMyYm6X8Szyjwhk=; b=YRB0jGD1flZzuLsK9ESks7OmZW1WQqbLR/eGyeOciBPTGEE+WDsLfc46+xN76q9D/o XdRfnwYt4nlgmCYPQw4/2ZSD5OPzx+Bl7k6tynzCOvzpc+Wqrmk7BsV+xnypYN2HH0ns 2TnsRBexT21M6gxjINgFEUyDatckGmlbRBH/j7M7gKi1gASzdkgVLFmF+YKAR2w7WAJ4 qgQ9tInOPtqvV0JKnASRE2S/vHzOriqYLikgpmX8KOJTai+kIqUek9RPjCJrQvb3kU15 QlxTxjUjnv5gqC7+LzBIzwE1xEZsO1seSe4nsvK5Ej+KCoKpCL1Blnw2k6TJnKxKxUfY p2Gg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685719481; x=1688311481; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=mWHdsFRgwU/DC9KYPyZpOr5kxSiGbMyYm6X8Szyjwhk=; b=TpahUqJOPsY1uyRR0L9PTOW1035Dl4pWeHfCTfKyBPw7Ueb9yQcI3NoyQE5zT4xsQv AFUwhc1JvBB3wlrqBHGHT/ipOBF7QqFcAF4ZZ1qshPse/orWMiRuNd/6BLMgkVricV54 OFzfLm5ftmYxrNnxLiHcrRe7FEcLxfWJerSeyFWbSpaXzWrEjOK7+gSCSVnByI7H4gqB wF7A8YbfgEWNEODBHsPKR8IqorbZ8zDucXrrqGhNNDpVFTe5vEs5z9KAma9DnUvuzHJO JMEX1ToBTfZ7qNfoWfaRSf/ZNSyca7a4NfEKD2aV9Nx/e1czB01+XTNqCJbudn1mMhKd AA+w==
X-Gm-Message-State: AC+VfDySjXBIxh+qrhC8L5sCWPsbVMLPWBtG5y9RRrk9kvPh0rNpJS75 Ut3ZoNt3L3p0dOZcrYhsAGtkexVkY2BDhdsWn0uHzZGQbdihQfEH
X-Google-Smtp-Source: ACHHUZ4wDDgb0CfbzbQNQElk1tMne9VDjNDmh/pGlgSi14TUJsyyb1dVFBdDUVIssKwgaaZe9e8wQfdGe9qlQZq+amg=
X-Received: by 2002:a2e:9216:0:b0:2b1:b334:bfa6 with SMTP id k22-20020a2e9216000000b002b1b334bfa6mr248749ljg.12.1685719480766; Fri, 02 Jun 2023 08:24:40 -0700 (PDT)
MIME-Version: 1.0
References: <E78B3C0B-B415-4E8C-9080-ADFB69D4464E@gmail.com> <AC6138B1-ADA3-4D27-959A-47053D195914@imdea.org> <ghneulz67ajhfmlvs4kkx2ixvtl54ycjtr63dp6diqkqth5hdp@dt3zayd4keas> <BY5PR11MB41965C8427EF343AD6B8EC72B54B9@BY5PR11MB4196.namprd11.prod.outlook.com> <59b2e26f-c9e2-19b7-5985-c71500f21e62@huawei.com>
In-Reply-To: <59b2e26f-c9e2-19b7-5985-c71500f21e62@huawei.com>
From: Andy Bierman <andy@yumaworks.com>
Date: Fri, 02 Jun 2023 08:24:29 -0700
Message-ID: <CABCOCHQtTRVTpRa-ghT=yH-Tx3M+S8+M8_fDnm-_pjFY3JHQ0A@mail.gmail.com>
To: Benoit Claise <benoit.claise=40huawei.com@dmarc.ietf.org>
Cc: "Rob Wilton (rwilton)" <rwilton=40cisco.com@dmarc.ietf.org>, netconf <netconf@ietf.org>, netconf-chairs <netconf-chairs@ietf.org>
Content-Type: multipart/alternative; boundary="000000000000d7a59105fd2726ca"
Archived-At: <https://mailarchive.ietf.org/arch/msg/netconf/DTV3BH7jLkIFmKp-m1C8zABsgdk>
Subject: Re: [netconf] Adoption poll for draft-tgraf-netconf-yang-notifications-versioning
X-BeenThere: netconf@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: NETCONF WG list <netconf.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netconf>, <mailto:netconf-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netconf/>
List-Post: <mailto:netconf@ietf.org>
List-Help: <mailto:netconf-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netconf>, <mailto:netconf-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 02 Jun 2023 15:24:49 -0000

On Fri, Jun 2, 2023 at 7:28 AM Benoit Claise <benoit.claise=
40huawei.com@dmarc.ietf.org> wrote:

> Dear all,
>
> Typically, I try to avoid sending a message such as "As co-author, I
> support ..." which doesn't carry much weight IMO.
>
> However, I have been observing the discussion and it's about time to share
> my views.
>
> First, it seems that we have mixing the WG Call of Adoption of this draft
> with the "Joint WGLC on "semver" and "module-versioning" drafts" in NETMOD.
> Granted, the two issues are linked but I don't see the point to redo the
> NETMOD WGLC on this NETCONF new draft.
> This draft is not about supporting multiple YANG module revisions in a
> server.
>
> My interest in this draft is about YANG-PUSH tooling. Exactly like I can
> directly onboard a new IPFIX router to the analytic system:
>     - Template Records and Flow Records are sent automatically from the
> router,
>     - basically, the IPFIX Flow Records, with the IPFIX Information
> Elements and Template Record semantic are automatically known to the
> Collector and most importantly to the analytic tool.
>     - the data processing chain can digest the information
>
> YANG-PUSH is currently changing, with UDP export
> (draft-ietf-netconf-udp-notif), with distributed export from line card
> (draft-ietf-netconf-distributed-notif
> *) *You see the analogy with IPFIX. Similarly to IPFIX toolchain, we need
> to ingest any new router with YANG-PUSH subscription automatically into the
> analytic system.  This justifies the addition of revision/revision labels
> in YANG-PUSH.
> Granted, we could be sending YANG-PUSH without those fields, then use the
> IETF-YANG-LIBRARY to understand the revision, and keep a lookup table in
> YANG-PUSH receiver ... as long as the router has not changed: no software
> update, no reboot, no offline. Basically ,we constantly need to check that
> nothing has changed. (remember the interface counter SNMP Polling with
> sysUpTime discontinuity counter).
>
> If the router is upgraded with NBC YANG modules (and yes, that could
> happen) , it's best to stop streaming those not-supported-by-the-pipeline
> YANG module (revisions). Hence the revision selection. We have to stop
> thinking that the end goal is for the routers to spit telemetry data. The
> end goal is that the entire onboarding/collection system up to analytics,
> can make sense of the telemetry data.  So yes, having garbage in my
> analytic system is a big deal. In other words, we don't want to break the
> end to end data processing chain (the pipeline) b/c of NBC YANG modules.
>
> So, in case it is not obvious, I support the WG call for adoption based on
> these valid use cases. :-)
>

If there are systems that can upgrade the YANG modules on-the-fly even
without disrupting any
YANG Push subscriptions, I would be quite impressed.  This is not the real
use-case.

There is a real need to support a no-so-YANG-aware type of client
application.
Kafka support was mentioned.  This is the real use-case.

Arbitrary complexity in YANG Push filtering is not helping interoperability.
The common use-case implied by this draft is a filter for instances of a
single object.
The single (and simple) revision-date works for this common use-case.

It might be better to standardize this common single-node filter than to
make a complex solution that ends up dragging in half the YANG library.



> Regards, Benoit
>

Andy


>
> On 5/30/2023 1:09 PM, Rob Wilton (rwilton) wrote:
>
> Hi,
>
> I think that it would be helpful to decouple the discussion about whether the YANG version number should be changed to the NETMOD WGLC review of the module versioning draft.  I don't think that part of the discussion is particularly relevant to the adoption call for this document.
>
> My understanding of the goal here is to make it easier to export network device telemetry data (exposed via YANG push) in a way that can integrate well with message bus services like Kafka.  I think that this is a good aim for the industry, and thus I support efforts (like this draft) to try and achieve this.
>
> I'm not intimately familiar with Kafka but I understand that it assumes that schema for the data being published can changed over time (e.g., perhaps due to a software upgrade on a publisher, or different instances of publishers running slightly different versions of software).  Hence Kafka maintains a schema registry and allows a reference to the schema to be associated with the published data records.  This allows entities in the data pipeline to check and confirm that the telemetry data conforms to the schema, and potentially to handle the data differently, if needed.
>
> I appreciate that RFC 6020 and RFC 7950 both are defined to try and ensure that YANG definitions only evolve in a backwards compatible way, but I see many examples within the wider industry and some examples in standards bodies where definitions are changed in ways that may impact clients.
>
> Hence, I think that this problem space, and draft, is about how to easily/robustly connect YANG data sources into Kafka data buses:
> - one choice is to do this entirely off the box, having the entities managing the subscriptions also tracking changes to the device schema (e.g., through subscribing to YANG library updates).  Perhaps this can be made to work, but probably adds additional timing complexity relating to when the subscriptions actually start sending data relative to receiving notifications that the content in YANG library has changed.  Another choice here could be to always tear down all subscriptions if the device schema changes at all.
> - the alternative is to augment the subscription-started/modified notifications to give an indication that the data for that subscription has changed.  All data sent before this notification uses a previous schema, and all data sent after that notification uses the new schema.  This allows the schema to change without completely tearing down existing subscriptions.
>
> I'm less sure about the ability to restrict subscription requests to particular versions.  At a minimum I would suggest putting this under a feature statement, but I'm not quite sure whether it should be needed at all.
>
> I also agree with Andy's comment that we need to resolve how the revision-date/label applies to all the potential data under a subscription.  To reuse part of Andy's example:
>
>    <A:top>
>       <B:list1 />
>          <C:container1 />
>    <A:top>
>
> If the subscription is to B/list1, but during upgrade the only change was to the contents of C:container1 then for the subscription the revision-date/label for B:list1 wouldn't have changed, but the data received in the subscription would have changed.  I.e., at the moment, YANG has no way to version everything under a subscription.  Hence, I think that the notification would need to list all the module versions that are being used by the subscription, but I'm not convinced that is easy to do in practice.  The alternative would be to just version the entire schema of the datastore (e.g., either a YANG library checksum, or a version label for a YANG package instance that represents the datastore schema), but this has the downside that all subscriptions would see a reported change in schema if any single module is changed on the device, even one that isn't being used (directly or indirectly) by the subscription.  But then an off-device tool can figure out if the actual schema for the subscription has changed.
>
> In summary, I support NETCONF working in this space, and I believe that this draft is a reasonable starting point, but it still requires some refinement, which could be done as part of the WG work on this problem.
>
> Rob
>
> // As an individual contributor.
>
>
>
> -----Original Message-----
> From: netconf <netconf-bounces@ietf.org> <netconf-bounces@ietf.org> On Behalf Of Jürgen Schönwälder
> Sent: 22 May 2023 19:25
> To: Camilo Cardona <juancamilo.cardona@imdea.org> <juancamilo.cardona@imdea.org>
> Cc: netconf <netconf@ietf.org> <netconf@ietf.org>; netconf-chairs <netconf-chairs@ietf.org> <netconf-chairs@ietf.org>
> Subject: Re: [netconf] Adoption poll for draft-tgraf-netconf-yang-notifications-
> versioning
>
> On Mon, May 22, 2023 at 12:43:20PM -0500, Camilo Cardona wrote:
>
>
> - I like the insights shared by Jürgen and Andy regarding how hacky
>   versioning is for YANG 1.1. It makes sense, but I guess nobody
>   wants to start a 1.2 project..
>
> If it is too hard to change the YANG version number, why should I
> believe it will be easy to change all the future module version
> numbers?
>
> If people are serious about versioning, the first thing they should
> do is proposing to bump the YANG version number when they make
> incompatible changes.
>
> /js
>
> --
> Jürgen Schönwälder              Constructor University Bremen gGmbH
> Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
> Fax:   +49 421 200 3103         <https://constructor.university/> <https://constructor.university/>
>
> _______________________________________________
> netconf mailing listnetconf@ietf.orghttps://www.ietf.org/mailman/listinfo/netconf
>
> _______________________________________________
> netconf mailing listnetconf@ietf.orghttps://www.ietf.org/mailman/listinfo/netconf
>
>
> _______________________________________________
> netconf mailing list
> netconf@ietf.org
> https://www.ietf.org/mailman/listinfo/netconf
>