Re: [netmod] Adoption poll for draft-ma-netmod-immutable-flag-08

["maqiufang (A)" <[e-mail]>]

Hi, Jürgen

Thanks a lot for the thorough review, please see my reply inline...

-----Original Message-----
From: netmod [mailto:[e-mail]] On Behalf Of Jürgen Sch?nw?lder
Sent: Saturday, September 9, 2023 5:09 AM
To: Jan Lindblad <[e-mail]>
Cc: [e-mail]
Subject: Re: [netmod] Adoption poll for draft-ma-netmod-immutable-flag-08

I have read draft-ma-netmod-immutable-flag-08 and my concerns remain. While I do see some valid use cases for this, I am concerned that this mechanism will be abused. The writing suggests that the server can dynamically decide what it considers immutable. If true, its a small step to have the server's decision depend directly or indirectly on other edits done by a client or on the operational state (the later is even described in the draft).
[Qiufang] I am sorry, but could you please give me more hints about how this mechanism could be abused? Is this related to the UC A.3? Because I think in -08, the authors have made an effort to make immutability to be independent of other edits done by a client or the operational state, we want immutability to be quite static based on the similar comments made by other folks, and this is already indicated in the document, e.g., sec 1.2:
" The immutability and configured value of an
   existing node must only change by software upgrade or hardware
   resource/license change. "
I am not sure which part of the draft has resulted in the understanding of "the server can dynamically decide what it considers immutable", could you point it out so that the authors can make further progress?

Thinking loud: If we can work out an immutable mechanism and we can then drop the system datastore, perhaps this is worth it. I do not see why we need a system datastore if we allow servers to add immutable data to <running>. 
[Qiufang] We are kind of touching system-config draft here, which defines a dedicated configuration datastore to hold system configuration which is either modifiable or non-modifiable (see https://www.ietf.org/archive/id/draft-ietf-netmod-system-config-02.html#name-modifying-overriding-system). While immutable mechanism makes it possible for the client to understand which system configuration can be overridden and which is not. Yes, we do allow servers to add system configuration to <running> but this is based on the premise: 1) the system configuration is referenced in <running> and all referential integrity constraints are need to be satisfied in <running>;2) the client has used a flag (we define it as "resolve-system" parameter) in the <edit-config> request so that servers populating configuration into <running> won’t be surprising for the client. 
System configuration will not be copied into <running> unless being referenced, this is regardless of the immutability of it. The system datastore and immutable mechanism can complement each other, but I think there is no function overlap. 

On the other hand, <running> was originally designed to be fully client controlled. The notion of server supplied immutable data nodes in <running> breaks this model. The other model is to have immutable data only in something like <system> and then that data gets merged into <intended> somehow. It seems the WG needs to settle what the right model is, for me the documents are still confusing and I am concerned about increasing client complexity and lowering interoperability. 
[Qiufang] I totally agree with you that <running> should be fully controlled by the client, actually, that is the reason why we'd like to define a "resolve-system" parameter to resolve references and the server should never write configuration into <running> unless this parameter is used. And there is indeed still a key issue in the system-config draft regarding whether we want <running> to always be valid, i.e., must all the referenced system configuration always be copied into <running>, and we can discuss it in a separate thread.
But I don’t think this draft will increase client complexity and lower interoperability, as we emphasize in the draft, it is already the case that immutable configuration exists in the real world, here we define some mechanism to improve its visibility to the clients. The client even without the knowledge of immutable extension/metadata annotation, can still send requests attempting to modify immutable configuration as it did before, and the server can still reject the request. Things just don’t get any worse, no extra complexity is introduced.

If the <system> datastore and immutable flags are designed to work tightly together, then we should be explicit about this and depending on how close they are tied together, perhaps even throw them into a single document.
[Qiufang] I am not sure we can put it like "they two are designed to work tightly together", because I'd rather to see immutable flag can even work without <system> datastore. If there is not a system datastore, immutable system configuration could also be present in <operational> or somewhere else. Maybe another reason we would like to document these two separately to keep each clean and concise. That said, if the WG decides to merge two, the authors are really okay with that. 

Detailed comments:

I am confused about the relationship of the proposed read-only (?) system data store and this immutable mechanism. They seem to interact or overlap, it is not clear to me how.
[Qiufang]I hope my clarification above could make it clear. Yes, system configuration datastore is read-only, but this just means that clients  cannot directly write into <system>; mutable system configuration can be modified by writing configuration to <running> that overrides the system configuration, while immutable system configuration cannot. 

The abstract says:

   Clients may use "immutable" statements in the YANG, and annotations
   provided by the server, to know beforehand when certain otherwise
   valid configuration requests will cause the server to return an
   error.

This seems to say that clients have to discover and work around immutable leafs in the data tree. The argument is that this improves the state of the art where modification requests simply fail without a clear indication of the reason. The solution assumes that clients retrieve the relevant portion of the data tree and then they can adapt. I think I also understood that immutable means 'immutable by a client', the data may be mutated by system, i.e., the data is mutable but just not mutable by the client.
[Qiufang] Yes, but even for the server, it is rather restrictive to change the immutability or the value of immutable leaf, which must only happen during software upgrade, license change, etc.

   Immutable configuration does not affect
   the contents of <running> by default.

What does this mean? I thought part of the motivation was that immutable data may be in config true nodes so that it can be referenced. If not in <running>, then where? What is the non-default case? For me, this is too fuzzy and handwaving.
[Qiufang]Sorry for being unclear,  immutable configuration is defined by the system so it is present in <system>(if exists) thus not affecting the contents of <running>. Once it is referenced and copied into <running>, the data object in <running> can never be changed by the client. We will better clarify it in the next version. 

Why can a client be expected to create immutable leafs in <running>?
[Qiufang] If there is other configuration which references immutable leafs, a client is expected to create in <running> with the same value in <system> to satisfy reference constraints.

Why do we need a system datastore if we allow immutable data (data only mutated by the server) in <running> et. al? Or we don't allow it in <running> and this depends on <system>?
[Qiufang]We need a system datastore so that the server can better expose the system configuration, regardless of being applied or being referenced or being immutable. While only referenced system configuration needs to be present in <running> if we need <running> to be valid. Make sense?

More generally, if we have something in <running>, then this does not always mean it gets applied. If there is immutable state, then non-matching config is likely not applied. If immutable config can be a function of operational state (and if it may even change with operational state), then we end up with other config depending on immutable config depending on mutable operational state, something we always tried to avoid.
[Qiufang] I am not sure I've fully catch you, but if we allow immutable configuration to be modified but not applied in <running>, I agree that other configuration depending on immutable configuration is likely not applied which might not meet user expectations. That's why we would suggest the server to directly reject modification to immutable configuration, instead of allowing it to be in <running> but not applied. Note that the draft tries to avoid making immutable configuration depending on mutable operational state by only allowing it to change on restrictive conditions.

Which error is returned when a client attempts to edit immutable data (with wrong values)? Is there a specific error that is returned when immutability causes an edit to fail? Or is the client expected to do a get when a somewhat generic error happens (with the necessary flags) to check whether something are some immutable leafs involved (and the client had the wrong value in the edit request)?
[Qiufang]Currently we are not defining any specific error caused due to immutability, "invalid-value" error-tag is reused to be backward compatible with the legacy, but I think it is a good point that we can define a more specific error-app-tag to point to it, will add it in the next version, thanks a lot!

   The error reporting is performed
   immediately at an <edit-config> operation time, regardless what the
   target configuration datastore is.

Why this statement, is this not generally how things work? What would be a delayed error report?
[Qiufang] Because here we would like to emphasize that the server don’t have to wait till a <commit> RPC operation to report the error if a different value is written into <candidate>, the server can detect and report it immediately. We would make it more explicit in the next version, thanks.

   Servers adding the immutable property which does not have any
   additional semantic meaning is discouraged.  For example, a key leaf
   that is given a value and cannot be modified once a list entry is
   created.

Not sure what this is trying to say. So what are the conditions under which a server can declare something to be immutable?
[Qiufang] If immutable flag is used but cannot provide extra information, it should not be used, like a key leaf node is already immutable once the list entry is created. Will reword this sentence in the next version.

And if I have some immutable data in a list, can I delete the list?
If not, what is the error that I can expect to get?
[Qiufang] Yes, the user can always delete any immutable configuration in <running>, which simply means make it invisible  in <running>. Since it is created by the server in <system>(if exists), this would not cause the server to delete and stop using it.

   The "immutable" flag is intended to be descriptive.

What does this mean? The YANG extension that is to be used at data model writing time. If a YANG module author declares a subtree to be immutable, then this means exactly what for an implementor (of servers and clients)?
[Qiufang] The intention is that, the immutable YANG extension-statement is used to describe an existing behavior, it applies to the server which already has the immutable configuration in the past. We are not defining mechanism to restrict server's behavior, but conveying what they've been doing all along in machine-readable behavior, there is nothing new for the implementor (of servers and clients).
 
   The "immutable" YANG extension can be a substatement to a "config
   true" leaf, leaf-list, container, list, anydata or anyxml statement.
   It has no effect if used as a substatement to a "config false" node,
   but can be allowed anyway.

I am not sure whether this fits the YANG design, are there other examples where YANG allows non-sensible statements just to ignore them? Perhaps Martin knows best.
[Qiufang]I think it is okay, e.g., see https://datatracker.ietf.org/doc/html/rfc7950#section-7.14.2, all "config" statements for the data nodes in  the input tree are ignored. Or "ordered-by" statement is ignored if it’s inside a state list, etc. Make sense to you?

Some of the text needs editorial work or better descriptions. An
example:

           "If servers always reject client modification attempts to
            some data node that can only be created, modified and
            deleted by the device itself, an 'immutable' YANG extension
            can be used to formally indicate to the client.

The logic feels backwards to me. The server rejects modification attempts because some data is considered immutable by the server. The text suggests it is the other way round, or worse, makes no sense (consider a server rejecting a request repeatedly due to a lack of permissions, this does not imply that the data is now immutable). And the immutable statement is about designing at data model design time that certain leafs are immutable. The server's behavior follows the model, not the other way round. And I would consider an immutable definition in the data model to actually be prescriptive in some form or the other.
[Qiufang]Sure, the authors will reword this sentence as well as preclude the factors of NAC model. I agree that the server's behavior follows the model, the immutability might be documented as plain-text in the description statement before, and now we hope there is a formal and machine-readable way to describe it.


            Adding immutable of an existing immutable statement
            is non-backwards compatible changes.
            Other changes to immutable are backwards compatible.";

What?
[Qiufang]This indeed needs editorial refinement! Thanks for pointing it out. 
 
I am also puzzled what the 'value' of the immutable extension statement is. The definition needs to be rewritten so that it defines precisely what this extension statement is.
[Qiufang] The value is a Boolean type with true or false, sure, will rewrite the definition of the extension-statement. 

The description of the immutable annotation has similar issues, except that for this one we know that the value is a boolean value.

           "define the with-immutable grouping.";

This is the kind of descriptions nobody needs. Perhaps explain what this definition is good for in order to establish the context for the stuff in the grouping and how it is used later.
[Qiufang] Thanks, will fix in the next version.

The example in A.1 is not really convincing. For the leafref to work, you need a set of immutable leafs, one for each timer value supported by the server. Are people really doing such constructions for timer values?? I understand that you want a leafref pointing to one out of N supported values, the timer value story, however, sounds weird. Do you have more convincing examples that are still easy to understand?
[Qiufang]Hope Balazs can chime in, given he provides this example.  It seems okay to me, such configuration is hardware-related, and its value is determined by the hardware capability/characteristic, it has to be declared as "config true" because of the need of being referenced by other configuration.

The example in A.2 confuses me. Interface configurations are applied to physically present interfaces. There is a name binding of the interface configuration to a physical interface. This section seems to suggest something very different from how I understand RFC 8343.
[Qiufang] I don’t see much inconsistency with RFC 8343, Can you point out what you think is problematic? Don’t you think it’s appropriate to define interface-type as immutable? As my understanding, it is a system-defined value and cannot be changed by the client with different values that cannot match its real type.

I have no idea why A.2.1 is there, it seems to show normal NC behavior or did I miss something? That said, what I am indeed missing is a clear definition which error is returned if an attempt is made to mutate an immutable leaf (well to a value it does not have).
[Qiufang] Good catch. We will define a dedicated error message for this specific case in the next version. Thanks a lot.

In A.3, if we have entries in lists that are immutable because the server believes this is good for the customer, then I am concerned that this stuff is not a static rare thing but something that will come and go dynamically. I see great potential for abuse of such a mechanism. Once the client needs to adapt to server logic to determine the mutability of things, we are on a slippery slope.
[Qiufang]Not so sure about this since immutability is defined to be rather static now. Some servers provide built-in configuration which cannot be modified by the client, because the configuration might be referenced by other clients, like predefined rules/policies, do you think it as an abuse?

/js

Best Regards,
Qiufang


On Tue, Sep 05, 2023 at 06:40:09PM +0200, Jan Lindblad wrote:
> Jürgen,
> 
> I agree completely with you regarding transactional management. Well said. 
> 
> In my reading of -08, however, I don't see the non-transactional behavior you describe in there any more. It was there in -01, but based on feedback from me and others, I think it has been washed out. If you still find non-transactional behavior in the latest rev, could you please point it out to me as well? My support too depends on the absence of such language.
> 
> I believe the document targets config true data that can never really be changed by any client. This data remains unchanged/unchangeable except possibly during a software version upgrade, entitlement/license change, hardware insertion or similar system-redefining event. Such data is config true simply because there are config true dependencies, such as leafrefs or must conditions on min and max range values. I think this kind of usage could be a relevant use case, and I'd support discussing ways to describe that in the WG.
> 
> Best Regards,
> /jan
> 
> 
> 
> > On 5 Sep 2023, at 18:05, Jürgen Schönwälder <[e-mail]> wrote:
> > 
> > I do not support this work. Yes, it is some effort on the server 
> > side to figure out how move from config state A to config state B. 
> > This proposal essentially pushes the work towards the clients that 
> > now have to figure out how to single step a server from config state 
> > A to config state B - and if things fail on intermediate steps it is 
> > the clients that have to figure out how to roll back the changes to 
> > get back to config state A (if that is possible at all). The 
> > original promise of NETCONF was that it is not necessary anymore to 
> > have clients that please the server, which was comiing out of many 
> > years of experience with protocols that required clients to please the servers.
> > 
> > If implementations have constraints on what kinds of edits are 
> > possible, then this is an implementation limitation and hence this 
> > should be documented in deviation statements but not in the data 
> > models.
> > 
> > /js
> > 
> > On Tue, Sep 05, 2023 at 12:42:35PM +0000, Kent Watsen wrote:
> >> NETMOD WG,
> >> 
> >> This email begins a 2-week adoption poll for: 
> >> https://datatracker.ietf.org/doc/draft-ma-netmod-immutable-flag/08
> >> 
> >> There is no known IPR on this draft (IPR call <https://mailarchive.ietf.org/arch/msg/netmod/_S-cKw5jIBmDKEPBRq8KeAbNLGg/>).
> >> 
> >> Please voice your support or technical objections to adoption on the list by the end of the day (any time zone) Sep 19.
> >> 
> >> Thank you,
> >> Kent (as co-chair)
> >> 
> > 
> >> _______________________________________________
> >> netmod mailing list
> >> [e-mail]
> >> https://www.ietf.org/mailman/listinfo/netmod
> > 
> > 
> > -- 
> > Jürgen Schönwälder              Constructor University Bremen gGmbH
> > Phone: [phone]         Campus Ring 1 | 28759 Bremen | Germany
> > Fax:   [phone]         <https://constructor.university/>
> > 
> > _______________________________________________
> > netmod mailing list
> > [e-mail] <mailto:[e-mail]> 
> > https://www.ietf.org/mailman/listinfo/netmod
> 

-- 
Jürgen Schönwälder              Constructor University Bremen gGmbH
Phone: [phone]         Campus Ring 1 | 28759 Bremen | Germany
Fax:   [phone]         <https://constructor.university/>

_______________________________________________
netmod mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/netmod