Re: [netmod] Comments on draft-ma-netmod-immutable-flag-06

["maqiufang (A)" <[e-mail]>]

Hi, Kent

Thank you for your valuable comments, please see inline.


I've just re-read the draft and this thread and have comments.

I support ensuring XC/Y remains transactional, such that a client can always move from valid config-A to valid config-B in a single update.  I also support requiring a "with-immutable" flag in client-requests in order for the "immutable" annotations to be returned (like "with-defaults").
Using an explicit parameter (e.g., with-immutable) avoids the case the client receives an "immutable" annotation yet doesn’t really understand that(clients don’t understand this annotation doesn’t even explicitly ask the server to return this). I agree that could be useful and should be added in the next version. I think this "with-immutable" flag should be supported even when a client retrieves a read-only target datastore (e.g., <system> or <operational>), though immutability only refers to restrictions on read-write datastores.
Any issues with the following statements?

1) Only the server can create/change/delete `immutable` data, which is seen in the <operational> and <system> datastores by default.  Clients are merely making immutable nodes visible/invisible in <running/start/candidate>.  If the client does not make a system-defined node visible in <running>, the node still exists in <operational> and <system>.
Among the common cases we see now are immutable data that is generated by system, and cannot be changed by the client. So the statement seems true for me.
I am unsure if there would be a case like a client creates a node instance but cannot modify that afterwards. That way it will not be present in <system> and the client may also be allowed to delete that. We used to have a BGP AS number case for that in previous version (https://datatracker.ietf.org/doc/html/draft-ma-netmod-immutable-flag-05#appendix-A.5)  but that is not a good practice and it was removed.

2) The "immutable" YANG extension statement (not the metadata annotation) designates, at the schema-level, config=true nodes that, when present in <running/startup/candidate>, are system-defined and hence immutable.
Note that NMDA does allow clients to create an interface entry with an interface-type value which is not yet physically present. Only when the interface physically appears, the type cannot be modified with another value not matching the real value the device is actually in use.


3) The "immutable" metadata annotation (not the YANG extension) designates, at the instance-level, config=true nodes that, when present in <running/startup/candidate>, are system-defined and hence immutable.    The metadata annotation is only needed to support config=true YANG lists that can contain a mix of client- and system- defined entries.
There is no such a case like a client creates several instances some of which are immutable while others are mutable. Generally I agree that “The metadata annotation is only needed to support config=true YANG lists that can contain a mix of client- and system- defined entries.” Which I believe is the most common.
Notice that leaf-list is not mentioned, cannot a particular leaf-list instance be immutable?

It seems to me all three above statements target that all immutable configuration are system-defined. But I assume some configuration may also be created by the client, e.g., interface type and BGP peer cases documented in the document. Thoughts?


4) The definition for "immutable" flag means is identical whether it is specified via the YANG extension or the metadata annotation.   Corollary: the "immutable" flag can only be used on data-nodes (e.g., not on "choice", "case", or "grouping").
Agree that it is a data node property, unless we define specifically what it means to use immutable flag on statements like choice/case/grouping, but I see it as out of scope for now.

A couple more musings:


1) I wonder if the "immutable" should be a boolean that is "false" by default, but can be explicitly set to "false" if a descendent of a "immutable=true" node.  That is, to let immutability be recursive but toggle-able.
If I understand you correctly, Kent, you’re proposing something similar to https://datatracker.ietf.org/doc/html/draft-ma-netmod-immutable-flag-06#section-3.2 except that im:immutable is changed to a Boolean type and of value “false” if not specified. That makes sense to me, if we agree that immutable should be a substatement to a "config true" leaf, leaf-list, container, list, anydata or anyxml statement rather than only terminal node like leaf or leaf-list.

2) I wonder is *what* the "immutable" flag means should be defined per data-type.  For instance:

    When a "leaf" node is immutable:

          - its value cannot change.
Yes, it’s quite easy.
    When a "leaf-list" node is immutable:

         - its value cannot change.

         - If specified at the schema-level, "ordered-by user"
           is invalid, because reordering changes the value.
My understanding is that,

·         if a leaf-list node is specified “immutable” at the schema-level, any leaf-list entries can neither be created or deleted. That means totally read-only to clients/users and I also agree that “ordered-by user” is invalid in this case.

·         if a specific leaf-list entry is specified “immutable” at the instance-level, only that specific entry cannot be deleted. Metadata annotation is only applied to existing instance, so the client can add/insert new entries; other entries without immutable flag can also be deleted.
Is this consistent with what you have in mind?
    When a "container" or "list" node is immutable:

          - all of its recursive descendants are "immutable=true"
            unless toggled back to "immutable=false" by a
            descendent node.
Yes, unless otherwise specified, immutability is inherited downwards towards the terminal leaves.
          - for lists, it is not possible to designate the list as a
            whole as immutable, because the list itself doesn't
            exist as a node in the data-tree.

              - If it is desired to communicate that list-entries
                cannot be added/removed by clients, it is sufficient
                for the list's `key` node(s) to be immutable=true.
I would rather see immutability of list the same way as leaf-list. That is, we do allow the list data node to be immutable at schema-level and it conveys that any list entry cannot be created and deleted, and likewise, “ordered-by user” is invalid in this case. I understand that list node may exist in multiple instances in the data tree but any issue you see with that?
              - if it is desired to communicate that a list-entries cannot
                by reordered, the list must be "ordered-by system".
                Corollary: an "ordered-by user" list is never immutable.
By the way, if a specific list instance is specified as immutable at instance-level, can a user create a descendant node instance inside that list instance?
    When a "anydata" or "anyxml" node is immutable


          - the node, and all of its recursive descendants, if any, is/are
            "immutable=true", unless toggled back to "immutable=false"
            by a descendent node.
Agree, I see it similar to a “container” node is immutable.
Thoughts?


Kent // contributor

Best Regards,
Qiufang




On Apr 17, 2023, at 5:29 AM, maqiufang (A) <[e-mail]<mailto:[e-mail]>> wrote:

Hi, Jan

Thank you so much for the follow-up, please see my reply inline.

From: Jan Lindblad (jlindbla) [mailto:jlindbla=[e-mail]]
Sent: Tuesday, April 11, 2023 10:05 PM
To: maqiufang (A) <[e-mail]<mailto:[e-mail]>>
Cc: Kent Watsen <[e-mail]<mailto:[e-mail]>>; Rob Wilton (rwilton) <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>
Subject: Re: [netmod] Comments on draft-ma-netmod-immutable-flag-06

Qiufang,

Thank you for your continued work on this. I think the critical point to decide now is which use cases are in and which are out.
Sure, agree.

If we decide that only the five or six use cases listed currently in -06 are included, I'm quite happy about standardizing some YANG extension and metadata keywords for them. As I mentioned earlier, I think a slightly simpler solution would be sufficient and preferable for the current, rather small and simple set of use cases. If someone prefers the solution proposed in -06 because it better caters for some not yet listed use case, please share that use case so we can see the whole picture and discuss.
The cases described in the current document now have covered the ones the authors would like to provide, so from the author’s perspective, I think I have no objections to your proposed simpler solution; and I honestly don’t have a plan to incorporate the invariant case into the draft.


I think that regarding the isInvariant concept in the liaison, 3GPP is asking to model the object which cannot be modified once created. While it's just a natural workaround for the clients to first delete it and then recreate with the really desired value, but not requiring that clients must do that.

Objects that cannot be modified once created cannot exist in a transactional protocol. If we introduce such objects, that would seriously detract from the value of NETCONF/YANG (XC/Y). Rather than watering down XC/Y to match the expectations by legacy management protocols, I'd suggest we ask servers that want to advertise themselves as XC/Y servers to add the missing functionality to their implementation. This is not unreasonable, as many, many server implementors have already done so across many industries.

If a particular vendor is unable or unwilling to implement a server to industry expectations, there is no way for me or anyone to prevent that vendor from releasing a YANG module with some proprietary extensions to describe that behavior. In fact, several already have, and that's probably fine. I just don't understand why IETF should spend valuable time to confuse he market and standardize such backward things. If 3GPP thinks the create-no-modify concept is valuable in the world of automation interfaces, they could release a 3GPP YANG module with such extensions.
Though I don’t have a very strong disagreement to the invariant behavior myself, if the WG thinks it as a really backward practice, I am pretty happy not to even mention that and keep the advancement of XC/Y.

As Jan pointed out, all the use cases written in the current document now are targeting configuration cannot be updated or deleted. I also feel it better to define identities to only support our current use cases now, and leave the door open to allow other implementation to extend more if needed.

While I like the flexibility with identities, they would really remove most of the value of standardization in this case. YANG already allows any vendor to invent their own extension keywords. If we standardize a new extension that relies on identities specified by vendors, we have not really gained much.
YANG extensions that would have an impact to the interoperability are good to be standardized IMO. Using identities is more like a compromise to me, providing an extensible solution to 3GPP without having to define that in IETF. But we can discuss more. This determines how we respond to the liaison. I assume we don’t really discuss a lot regarding the solution part.
Again, as mentioned, the objective is to document existing server behaviors, if the server already internally considers some configuration immutable for valid reasons. It doesn't apply to servers which don't have any immutable configuration, much less to encourage servers to add more such restrictions. The document should be updated if it failed to make that clear.

I'm fine with marking some parts of the configuration as immutable/impossible to change. This may cause some operational trouble in the field, but not as much as non-transactionality. It is the non-transactional behavior I find really hard to live with. Let's first decide which use cases we are addressing, then find the least intrusive solution to that set.
Operational trouble already exists today, immutable-flag tries to make some of that visible to client. See my comments above, I think no one is trying to introduce non-transactionality into the IETF.

By the way, can we also discuss a little bit about the systemCreated attributes raised in 3GPP liaison? Objects that can only be created/deleted by the system and the client is not allowed to create/delete.
Given that we already have a system-config document that defines a <system> datastore, I think if a client creates a system object with the same value as it is in <system>, that should be allowed (and if the system configuration is modifiable, a different value can be used to override that system-initialized value). System config copied from <system> into <running> should also be allowed to delete. While I can also see the requirements for this systemCreated attributes if some implementation directly populates system config into <running>.


Best Regards,
/jan

Best Regards,
Qiufang


From: netmod [mailto:[e-mail]] On Behalf Of Kent Watsen
Sent: Thursday, April 6, 2023 12:12 AM
To: Rob Wilton (rwilton) <[e-mail]<mailto:[e-mail]>>
Cc: Jan Lindblad (jlindbla) <jlindbla=[e-mail]<mailto:jlindbla=[e-mail]>>; [e-mail]<mailto:[e-mail]>
Subject: Re: [netmod] Comments on draft-ma-netmod-immutable-flag-06

Hi Rob,

My prior response to you focused on what the draft specifies (not the liaison), since you wrote:

"Fundamentally, I generally interpret this draft as saying:  NETCONF/YANG doesn't really match the existing management model/API in 3GPP and hence we want to make non-backwards compatible changes to NETCONF/YANG to match the 3GPP semantics."

Which isn't the draft's proposition.  Maybe you meant s/draft/liaison/?


I just re-read the liaison and I fail to see how the immutable-flag draft's goal of describing existing server behavior isn't aligned with the liaison.  At the end of the day, it doesn't really matter what extensions exist in the YANG, or annotations in the data, the client can send any request it wants to the server and, ultimately, the server must enforce whatever it wants to enforce.

It effectively comes down to a quality-assurance effort to determine if the YANG accurately describes the server's existing behavior.  To the extent that XC/Y tools might someday grok the YANG-extensions and/or data-annotations (to do some of the heavy-lifting) is out of scope, IMO.

I'm unsure if the set of the extensions and annotations in the current draft are robust enough to sufficiently support every use-case.  The WG could create taxonomies and the like to prove this out.  But a possibly better solution is for the immutable-flag draft to define behaviors using identities (instead of bits/boolean), so that 1) we don't have to define a perfect set of flags upfront and 2) applications (3GPP) can define additional flags if desired.

Whilst I agree that it is best for servers to be completely transactional, avoiding the need to delete a parent container in order to recreate a previously-immutable child, 3GPP has already decided to do this.  I see no issue in providing them an ability to capture this semantic using identities they defined themselves.

Kent







On Apr 5, 2023, at 6:49 AM, Rob Wilton (rwilton) <[e-mail]<mailto:[e-mail]>> wrote:

Hi Kent,

Some of my concern stems from the fact that during the NMDA architecture discussions there was a strong desire to make the configuration data stored in <running> to be owned by the client.  I.e., the server has the right to accept or reject a particular configuration but ultimately it is the client that should control what is in <running>.  Related to this, is the goal of ensuring the validation of the configuration is based on the state that the configuration represents, not the particular config edit that is being made to transition the configuration into that state.  I.e., it should be possible for a client to change the configuration from any valid state A to any valid state B, without requiring extra client orchestration steps to keep the server happy (e.g., you can transition from A to B, but must go via C, D and E on the way).  Or to put it another way, if such steps are required, then it is much simpler (for an automation client) to do those transitional steps on the server than it is to expose and force them on to the client.

As you point out, existing implementations don’t always follow these rules above, but I regard these as warts in the implementations relative to following the ideal architecture above.  As such, I have little issue with a YANG extension or metadata annotation to programmatically indicate to clients where these aberrations occur for this use case.

But this isn’t what the 3GPP liaison is asking for.  They are requesting to bake the edit-config constraints directly into the management APIs defined in YANG because this is how their underlying management model is defined and they don’t want to change their underlying paradigm.  No longer is YANG just defining the configuration data model, but protocol edit-config semantics are being merged and baked into the data model defining the management API.

These newly defined management APIs will not just work with existing generic YANG clients and orchestrators because I presume that the clients will require custom code to be able to successfully interface with the server implementing these models.  Perhaps these annotations provide sufficient information for YANG clients to generically work around these restrictions?  But even in the case that they do then I still question whether that is really helpful.  I.e., what is ultimately achieved other than the addition of some extra complexity in the automation, and what is the true goal here.  If the aim is to signal to the client that there are some properties that cannot be changed without tearing down the containing service in a traffic impacting way, then rather than marking the properties as being immutable, it might be sufficient to annotate them as service impacting if changed.  This might still provide the necessary visibility and awareness to the client, whilst avoiding additional orchestration complexity for clients.  After all, there are many properties in the network configuration models standardized in the IETF (and OpenConfig) that are similarly service impacting if changed, that don’t seemingly require any protection.

It also feels that we are on the path of fracturing the definition of the NETCONF/YANG management protocols, which is somewhat like how OpenConfig decided to interpret the pattern statement regex language differently (which they have now backtracked on) or their recent statement that servers are not generally expected/required to validate leaf ref constraints in the running configuration.  Each of these little cuts, whilst innocent and good intentioned on their own, risk gradually devaluing the common standards by creating many incompatible subvariants of the language and protocols.

Hence, this is why I think Jan’s approach of looking at the individual problems that are being solved and having a discussion as to what is the best way of solving these individual problems may result in a better overall solution.  Specifically, is there a compromise that can meet 3GPP’s and ITU’s goals without eroding the underlying NETCONF/YANG architecture?

Regards,
Rob

// Still no hats.

From: netmod <[e-mail]<mailto:[e-mail]>> On Behalf Of Kent Watsen
Sent: 03 April 2023 22:23
To: Rob Wilton (rwilton) <rwilton=[e-mail]<mailto:rwilton=[e-mail]>>
Cc: Jan Lindblad (jlindbla) <jlindbla=[e-mail]<mailto:jlindbla=[e-mail]>>; [e-mail]<mailto:[e-mail]>
Subject: Re: [netmod] Comments on draft-ma-netmod-immutable-flag-06

Hi Rob,





- In terms of properties that cannot be changed once written, I would rather see this issue framed more in the direction of it just being extra documentation written in a machine-readable way.  Specifically, using the annotation to give an indication that servers MAY reject requests to create/delete, or change, the configuration, but not requiring that they do so.  I.e., at the data model level, I don't think that we should preclude servers being able to handle this is in a more client friendly way (e.g., by breaking a single client transaction up into multiple internal transactional changes where needed).

I agree that the document does not make it clear enough, but this is already the case.   As I said at the end of this document's presentation on Friday's NETMOD, session, this document has no runtime-impact on servers (other than them needing to return annotated YANG and/or metadata).  There is also no runtime-impact on clients, as they as free to ignore all the annotations and metadata.   All this document does is define a mechanism for servers to describe the behavior they already implement.   The text in the document is confusing because the normative statements make it sound like the server needs to implement behavior to reject certain updates *because annotations/metadata said so*, but actually it's the other way around, as the server was already implemented to reject the changes.

1st paragraph in the Introduction:

This document defines a way to formally document as a YANG extension or YANG metadata an existing model handling behavior that is already allowed in YANG and which has been used by multiple standard organizations and vendors. It is the aim to create one single standard solution for documenting modification restrictions on data declared as configuration, instead of the multiple existing vendor and organization specific solutions. See Appendix B<https://datatracker.ietf.org/doc/html/draft-ma-netmod-immutable-flag#Existing_implementations> for existing implementations.¶<https://datatracker.ietf.org/doc/html/draft-ma-netmod-immutable-flag#section-1-1>








- For any immutable related metadata annotations, I think that this additional metadata should only be returned to clients when explicit requested via a separate RPC parameter, and I think that the draft needs to add text for protocol capabilities used to indicate whether this new option is supported (e.g., along the lines of RFC 6243, with-defaults).

Somewhat agree (Principle of Least Astonishment), though it's neither illegal, would cause client problems, or cause excessive network utilization (unlike with-defaults).

K.