Re: [netconf] Adoption poll for draft-tgraf-netconf-yang-notifications-versioning

[[e-mail]]

Dear Andy,

Thanks for the feedback. We will address your input that an xpath and subtree-filter could reference more than one YANG module in the next revision.

AB> You are saying a dynamic subscription to a datastore will block an operator from making software changes to the device?  I would not have expected that. I expected the subscription would be suspended if the revision changed.

TG> The draft enables to perform a software upgrade of a network node where the subscribed YANG module revision can change and the network operator has the ability in a configured subscription to define wherever the subscription should be suspended or not by selecting the specific revision or selecting the backward compatible revision-label when the configured subscription is initially being configured.

AB> The revision-date is really only needed if there is a new module revision with NBC changes.
Otherwise, a normal YANG client is expected to handle any BC changes to the read-only data
retrieved from the server.   It seems fragile to force a specific revision and break if BC changes
are introduced.

TG> On the contrary. Wit this document we give the network operator the ability to subscribe to a specific revision or backward compatible revision-label ensure that semantic changes can only occur in the defined boundaries. The YANG Push receiver learns with each established and changed subscription the revision and revision-label for each subscribed YANG module along the subscribed xpath and the sub-tree filter.

AB> The client should decide if the revision is acceptable.
TG> I believe you are referring to the YANG push receiver correct? I argue that this is always a valid design. To me, a YANG push receiver should be in charge of retrieving YANG push messages and its semantics. It shall only drop messages if they do not conform to semantics, which should only occur in error condition. If the subscription permits semantic changes, wherever they are backward compatible or not, then the downstream system (TSDB, real-time data enrichment etc.) who is processing the data decides how the data is being processed. In case where the new semantics are backward compatible, we might process only the dimensions and measurements which are unchanged, or when the new messages have none backward compatible semantics, messages could be dropped. What I would suspect is that most network operators would opt for a subscription where backward compatible changes in semantics are allowed but if non backward compatible changes in semantics are performed, the subscription will be suspended, preventing that non processable data is even being exposed, dropping them as early as possible to prevent wasting resources.

Best wishes
Thomas

From: Andy Bierman <[e-mail]>
Sent: Friday, May 26, 2023 6:34 PM
To: Graf Thomas, INI-NET-VNC-HCS <[e-mail]>
Cc: [e-mail]; [e-mail]; [e-mail]; [e-mail]; [e-mail]
Subject: Re: [netconf] Adoption poll for draft-tgraf-netconf-yang-notifications-versioning




On Thu, May 25, 2023 at 11:05 PM <[e-mail]<mailto:[e-mail]>> wrote:
Dear Andy,

Thanks a lot for the comment. Let me answer chronological.

I assume you are referring to section 3 of the document which augments the xpath/sub-tree filter in the subscription state change notification messages with the revision and revision-label information.

I cannot find any such augment.

I was referring to the augments which modify the RPC operations:


  // Subscription parameters

  augment "/sn:establish-subscription/sn:input/sn:target" {

    description

      "Augment the establish-subscription RPC from the ietf-subscribed-notifications

      YANG module with the yang-push-module-version-config grouping.";

    uses ypm:yang-push-module-version-config;

  }

  augment "/sn:modify-subscription/sn:input/sn:target" {

  description

      "Augment the modify-subscription RPC from the ietf-subscribed-notifications

      YANG module with the yang-push-module-version-config grouping.";

          uses ypm:yang-push-module-version-config;

  }


These augments look incorrect.
The 'sn:target' node is a choice, which can only be augmented with a case (not 2 leafs).
You probably mean to augment the 'datastore' case in RFC 8641.



With netconf-capability-change you are referring to RFC 6470 correct? To my current understanding, and please correct me here, module revision is not a capability.

The server is required to send this event out if the loaded modules changes.
 It also send the YANG library events, which are more directly related to a module change.


With yang-library-change you are referring to RFC 7895 correct? To my understanding it has been superseded by RFC 8525. According to https://datatracker.ietf.org/doc/html/rfc8525#appendix-A, yang-library-update is a replacement for yang-library-change notification. A yang-library-change notification is sent when a change in YANG library such as a new module was added or the revision of the module occurred. If a change occurred, a notification is sent with the content-id to the YANG receiver. The YANG push receiver then needs to obtain the new information from the YANG library.


That is correct.
A YANG client is expected to use the server YANG library information to determine all
module/revision/feature/deviation settings for the server.



To my current knowledge, and please correct me if I understood it wrong, this mechanism does not prevent that the YANG push messages are published even though the semantics have changed. Nor does it prevent when the network node is being reloaded into a new software version where a YANG module revision has changed. It does address the case where during runtime the YANG module revision changes, however the described workflow is more complex to implement. Would you agree with my summary?


You are saying a dynamic subscription to a datastore will block an operator from making software
changes to the device?  I would not have expected that. I expected the subscription
would be suspended if the revision changed.

The server is not expected to manage module revision changes.
I am not sure how a client would use your proposed augments

The revision-date is really only needed if there is a new module revision with NBC changes.
Otherwise, a normal YANG client is expected to handle any BC changes to the read-only data
retrieved from the server.   It seems fragile to force a specific revision and break if BC changes
are introduced.



Going back to section 2. You raised the point that a subscription filter is not limited to 1 module. From the sentence " Only a single selection filter can be applied to a subscription at a time" in Section 3.6 of RFC 8641 https://datatracker.ietf.org/doc/html/rfc8641#section-3.6 we understood it is actually limited to one filter. Otherwise the semantic reference in the Subscription State Change Notification Messages would be vague. It would not be clear which filter applies to which subscription id which is the key in the push-update notification. What I believe, and please correct me if I misunderstood, you are referring to the case where a xpath is referencing to more than one YANG module. Correct? If that would be the case then we are back at the point similar to initial proposal (https://datatracker.ietf.org/doc/html/draft-tgraf-netconf-yang-notifications-versioning-00) where we proposed to add the module name for each revision as well.


No.  The 1 filter refers to a subtree or XPath filter.
There is no requirement that a filter is only limited to 1 module.

   <A:top>
      <B:list1 />
      <C:container1 />
   <A:top>

Does the revision-date refer to module A, B, or C?

   Xpath  "/interfaces"

Does the XPath filter refer to oc:interfaces or if:interfaces?
(XPath can get really complex -- the expression is not limited to an absolute path).




Regarding latest and latest-compatible-semversion from section 2 not matching YANG model in section 4.1 valid point. Thanks for spotting this. Noted to be corrected in -04 version.

https://author-tools.ietf.org/iddiff?url1=https://raw.githubusercontent.com/network-analytics/draft-tgraf-netconf-yang-notifications-versioning/main/draft-tgraf-netconf-yang-notifications-versioning-03.txt&url2=https://raw.githubusercontent.com/network-analytics/draft-tgraf-netconf-yang-notifications-versioning/main/draft-tgraf-netconf-yang-notifications-versioning-04.txt



How would server code deployed in 2023 know it wasn't the 'latest' anymore in 2026?
The client should decide if the revision is acceptable.


Best wishes
Thomas


Andy


From: Andy Bierman <[e-mail]<mailto:[e-mail]>>
Sent: Friday, May 19, 2023 5:25 PM
To: Graf Thomas, INI-NET-VNC-HCS <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
Subject: Re: [netconf] Adoption poll for draft-tgraf-netconf-yang-notifications-versioning



On Fri, May 19, 2023 at 7:42 AM Andy Bierman <[e-mail]<mailto:[e-mail]>> wrote:


On Fri, May 19, 2023 at 4:05 AM <[e-mail]<mailto:[e-mail]>> wrote:
Dear Andy, Jürgen, Nacho and Frank,

Thanks a lot for the feedback and comments. I understand your concerns that they are targeting section 2, extending the YANG push subscription capability to not only subscribing to an xpath but also be capable to select with revision or revision-label.

In section 1 we introduce the problem statement, here an excerpt:

---
This semantic reference is available when the subscription is being establish as described in Section 3.6 of [RFC8641] and being streamed from the publisher to receiver with the subscription state change notifications described in Section 2.7 of [RFC8639] where for each subscription a locally unique subscription ID described in Section 4.3.2 of [RFC8641] is being issued and streamed as metadata with the notification message in the YANG push message header.

The semantics can change between different YANG module revisions. The YANG module version statement is specified in Section 7.1.2 of [RFC6020] and states that the newer revision needs to be backward compatible to the previous revision. Section 3.1 of [I-D.ietf-netmod-yang-module-versioning] specifies that newer semantic versions introduced in [I-D.ietf-netmod-yang-semver] MAY not be backward compatible to the previous version when indicated with non-backwards-compatible keyword.
---

When the software of a network node is being upgraded, possibly the YANG module revisions might change. Thanks to the extension describes in section 3 of the document, the YANG push receiver is now able to learn through the subscription state change notification message that the revision or/and revision-label has changed and through netconf <get-schema> the new YANG module must be obtained outband to retrieve the new semantics for the push-update notification message.

So far so good. With this we prevented that the end to end data processing chain breaks because semantics can be maintained across network node software upgrades, YANG module changes, in configured subscription as well. However if the YANG module change involved a non-backward compatible change, messages are being dropped within the data processing chain. A network operator might ask, why should we publish YANG push messages from a network node when they cant be processed further down the data processing chain? Why shouldn't we give the network engineer the ability to express in the YANG push subscription that only YANG push messages complying to a certain revision or revision-label (semantic revision) should be published. This is what section 2 of the document tries to address.

I understood in the mail thread that it was assumed that the YANG data store would need to support more than one revision per YANG module. This is not the case. The point is that when the configured subscription is established and while the messages are being published, the revision or revision-label might change. Today we do not have the ability to subscribe to a specific revision nor to a specific compatible revision-label. This is what we try to enable in this document.




These are the network operators concerns and needs. From an implementer point of view, do you believe this makes sense and is implementable?



Good news: you are not trying to undo the 'implement 1 revision' requirement in RFC 7950.
Not so good news: There are already 2 events (netconf-capability-change and yang-library-change)
to notify the client of the extremely rare module update event.

In the rare event a module is updated, the subscriber gets a 3rd notification for this change,
and the data from the module starts to be filtered by the server so the subscriber is prevented
from receiving the "unwanted" data.  This can be quite complicated to implement,
forcing special processing for each subscription.

I am not in favor of changing YANG Push in this way.



I do not see any clear YANG definition or examples for these augments:


module: ietf-yang-push-revision



  augment /sn:establish-subscription/sn:input/sn:target:

    +--rw revision?         rev:revision-date-or-label

    +-- revision-label?   ysver:version

  augment /sn:modify-subscription/sn:input/sn:target:

    +--rw revision?         rev:revision-date-or-label

    +-- revision-label?   ysver:version

  augment /sn:subscription-started/sn:target:

    +--ro revision          rev:revision-date-or-label

    +-- revision-label?   ysver:version

  augment /sn:subscription-modified/sn:target:

    +--ro revision          rev:revision-date-or-label

    +-- revision-label?   ysver:version

  augment /sn:subscriptions/sn:subscription/sn:target:

    +--ro revision          rev:revision-date-or-label

    +--rw revision-label?   ysver:version

Since a subscription filter is not limited to 1 module, It is unclear how
this draft would be implemented.

The augment is not a module-name + revision, just a revision.
The purpose is unclear. These leafs are added to the establish-subscription input.


    leaf revision {

      type rev:revision-date-or-label;

      description

        "This references the YANG module revision to be sent in the subscription.";

    }

    leaf revision-label {

      type ysver:version;

      description

        "This references the YANG module semversion to be sent in the subscription.";

    }


The subscription never specifies a module, so what module name do these parameters affect?

Sec 2 describes some parameter values that do not exist (latest, latest-compatible-semversion)
The YANG module defines a yang-push-revision-supported feature, but there are no if-feature
statements that reference it.

Andy



Looking forward for your feedback and comments.

Best wishes
Thomas


Andy


From: netconf <[e-mail]<mailto:[e-mail]>> On Behalf Of Andy Bierman
Sent: Wednesday, May 17, 2023 7:27 PM
To: Jürgen Schönwälder <[e-mail]<mailto:[e-mail]>>; IGNACIO DOMINGUEZ MARTINEZ-CASANUEVA <[e-mail]<mailto:[e-mail]>>; Mahesh Jethanandani <[e-mail]<mailto:[e-mail]>>; netconf <[e-mail]<mailto:[e-mail]>>; netconf-chairs <[e-mail]<mailto:[e-mail]>>
Subject: Re: [netconf] Adoption poll for draft-tgraf-netconf-yang-notifications-versioning



On Wed, May 17, 2023 at 1:42 AM Jürgen Schönwälder <[e-mail]<mailto:[e-mail]>> wrote:
All of this has been pointed out before and does not come as a
surprise. So we should have a general answer how to handle the
"damage" created by an NBC update of YANG that according to some does
not even require a YANG version number change.

It is more than just a conformance issue.
It is hard to overstate the importance of RFC 7950 as the 1 well-written
document needed to learn YANG.  It may be OK to distribute the
YANG data model in lots of ad-hoc specifications, but not the
data modeling language definition.

People (way outside the IETF) will be expected to learn and use the "new" YANG correctly,
and that will be much more difficult without even a name, let alone a version number.
What is the new YANG called? "NBC YANG"?


Andy


..
/js
...