Re: [netmod] Adoption poll for draft-ma-netmod-immutable-flag-08

[Jürgen Schönwälder <[e-mail]>]

Reading and responding is easier if people use email clients that can
properly quote text...

On Wed, Sep 13, 2023 at 08:11:25AM +0000, maqiufang (A) wrote:
Hi, Jürgen

Thanks a lot for the thorough review, please see my reply inline...

-----Original Message-----
From: netmod [mailto:[e-mail]] On Behalf Of Jürgen Sch?nw?lder
Sent: Saturday, September 9, 2023 5:09 AM
To: Jan Lindblad <[e-mail]>
Cc: [e-mail]
Subject: Re: [netmod] Adoption poll for draft-ma-netmod-immutable-flag-08

I have read draft-ma-netmod-immutable-flag-08 and my concerns remain. While I do see some valid use cases for this, I am concerned that this mechanism will be abused. The writing suggests that the server can dynamically decide what it considers immutable. If true, its a small step to have the server's decision depend directly or indirectly on other edits done by a client or on the operational state (the later is even described in the draft).
[Qiufang] I am sorry, but could you please give me more hints about how this mechanism could be abused? Is this related to the UC A.3? Because I think in -08, the authors have made an effort to make immutability to be independent of other edits done by a client or the operational state, we want immutability to be quite static based on the similar comments made by other folks, and this is already indicated in the document, e.g., sec 1.2:
" The immutability and configured value of an
   existing node must only change by software upgrade or hardware
   resource/license change. "
I am not sure which part of the draft has resulted in the understanding of "the server can dynamically decide what it considers immutable", could you point it out so that the authors can make further progress?
{JS} The definition given right at the start of section 5.1

Thinking loud: If we can work out an immutable mechanism and we can then drop the system datastore, perhaps this is worth it. I do not see why we need a system datastore if we allow servers to add immutable data to <running>. 
[Qiufang] We are kind of touching system-config draft here, which defines a dedicated configuration datastore to hold system configuration which is either modifiable or non-modifiable (see https://www.ietf.org/archive/id/draft-ietf-netmod-system-config-02.html#name-modifying-overriding-system). While immutable mechanism makes it possible for the client to understand which system configuration can be overridden and which is not. Yes, we do allow servers to add system configuration to <runningbut this is based on the premise: 1) the system configuration is referenced in <runningand all referential integrity constraints are need to be satisfied in <running>;2) the client has used a flag (we define it as "resolve-system" parameter) in the <edit-configrequest so that servers populating configuration into <runningwon’t be surprising for the client. 
System configuration will not be copied into <runningunless being referenced, this is regardless of the immutability of it. The system datastore and immutable mechanism can complement each other, but I think there is no function overlap.
{JS} the possible implementation behaviours clients have to deal with depending on which drafts are implemented by a server concern me.

On the other hand, <runningwas originally designed to be fully client controlled. The notion of server supplied immutable data nodes in <runningbreaks this model. The other model is to have immutable data only in something like <systemand then that data gets merged into <intendedsomehow. It seems the WG needs to settle what the right model is, for me the documents are still confusing and I am concerned about increasing client complexity and lowering interoperability. 
[Qiufang] I totally agree with you that <runningshould be fully controlled by the client, actually, that is the reason why we'd like to define a "resolve-system" parameter to resolve references and the server should never write configuration into <runningunless this parameter is used. And there is indeed still a key issue in the system-config draft regarding whether we want <runningto always be valid, i.e., must all the referenced system configuration always be copied into <running>, and we can discuss it in a separate thread.
But I don’t think this draft will increase client complexity and lower interoperability, as we emphasize in the draft, it is already the case that immutable configuration exists in the real world, here we define some mechanism to improve its visibility to the clients. The client even without the knowledge of immutable extension/metadata annotation, can still send requests attempting to modify immutable configuration as it did before, and the server can still reject the request. Things just don’t get any worse, no extra complexity is introduced.
{JS} another parameter adds another dimension of complexity clients have to deal with; the argument "what is out there is bad so it does not matter how complex this solution is" does not convince me

If the <systemdatastore and immutable flags are designed to work tightly together, then we should be explicit about this and depending on how close they are tied together, perhaps even throw them into a single document.
[Qiufang] I am not sure we can put it like "they two are designed to work tightly together", because I'd rather to see immutable flag can even work without <systemdatastore. If there is not a system datastore, immutable system configuration could also be present in <operationalor somewhere else. Maybe another reason we would like to document these two separately to keep each clean and concise. That said, if the WG decides to merge two, the authors are really okay with that.
{JS} I am looking for ideally a simple single solution to the problem, I am concerned by a large number of possible combinations to solve the problem.

Detailed comments:

I am confused about the relationship of the proposed read-only (?) system data store and this immutable mechanism. They seem to interact or overlap, it is not clear to me how.
[Qiufang]I hope my clarification above could make it clear. Yes, system configuration datastore is read-only, but this just means that clients  cannot directly write into <system>; mutable system configuration can be modified by writing configuration to <runningthat overrides the system configuration, while immutable system configuration cannot. 

The abstract says:

   Clients may use "immutable" statements in the YANG, and annotations
   provided by the server, to know beforehand when certain otherwise
   valid configuration requests will cause the server to return an
   error.

This seems to say that clients have to discover and work around immutable leafs in the data tree. The argument is that this improves the state of the art where modification requests simply fail without a clear indication of the reason. The solution assumes that clients retrieve the relevant portion of the data tree and then they can adapt. I think I also understood that immutable means 'immutable by a client', the data may be mutated by system, i.e., the data is mutable but just not mutable by the client.
[Qiufang] Yes, but even for the server, it is rather restrictive to change the immutability or the value of immutable leaf, which must only happen during software upgrade, license change, etc.

   Immutable configuration does not affect
   the contents of <runningby default.

What does this mean? I thought part of the motivation was that immutable data may be in config true nodes so that it can be referenced. If not in <running>, then where? What is the non-default case? For me, this is too fuzzy and handwaving.
[Qiufang]Sorry for being unclear,  immutable configuration is defined by the system so it is present in <system>(if exists) thus not affecting the contents of <running>. Once it is referenced and copied into <running>, the data object in <runningcan never be changed by the client. We will better clarify it in the next version.
{JS} so is <system> required? What if there is no <system>?

Why can a client be expected to create immutable leafs in <running>?
[Qiufang] If there is other configuration which references immutable leafs, a client is expected to create in <runningwith the same value in <systemto satisfy reference constraints.
{JS} Why?

Why do we need a system datastore if we allow immutable data (data only mutated by the server) in <runninget. al? Or we don't allow it in <runningand this depends on <system>?
[Qiufang]We need a system datastore so that the server can better expose the system configuration, regardless of being applied or being referenced or being immutable. While only referenced system configuration needs to be present in <runningif we need <runningto be valid. Make sense?
{JS} so <system> is just an added feature to provide more context while the prime mechanism is immutable and the server adding (perhaps removing) data to running?

More generally, if we have something in <running>, then this does not always mean it gets applied. If there is immutable state, then non-matching config is likely not applied. If immutable config can be a function of operational state (and if it may even change with operational state), then we end up with other config depending on immutable config depending on mutable operational state, something we always tried to avoid.
[Qiufang] I am not sure I've fully catch you, but if we allow immutable configuration to be modified but not applied in <running>, I agree that other configuration depending on immutable configuration is likely not applied which might not meet user expectations. That's why we would suggest the server to directly reject modification to immutable configuration, instead of allowing it to be in <runningbut not applied. Note that the draft tries to avoid making immutable configuration depending on mutable operational state by only allowing it to change on restrictive conditions.
{JS} suppose a hardware card is plugged into a system, the server adds immutable state to <running>, config refers to this immutable state, the card is removed or replaced, what happens? Or is this scenario not applicable?

Which error is returned when a client attempts to edit immutable data (with wrong values)? Is there a specific error that is returned when immutability causes an edit to fail? Or is the client expected to do a get when a somewhat generic error happens (with the necessary flags) to check whether something are some immutable leafs involved (and the client had the wrong value in the edit request)?
[Qiufang]Currently we are not defining any specific error caused due to immutability, "invalid-value" error-tag is reused to be backward compatible with the legacy, but I think it is a good point that we can define a more specific error-app-tag to point to it, will add it in the next version, thanks a lot!
{JS} it may be fine to use an existing error but this needs to be defined and not be implied by way of an example

   The error reporting is performed
   immediately at an <edit-configoperation time, regardless what the
   target configuration datastore is.

Why this statement, is this not generally how things work? What would be a delayed error report?
[Qiufang] Because here we would like to emphasize that the server don’t have to wait till a <commitRPC operation to report the error if a different value is written into <candidate>, the server can detect and report it immediately. We would make it more explicit in the next version, thanks.
{JS} <candidate> is a scratch pad, whether references resolve is determined during commit or validate processing, not during edit processing.

   Servers adding the immutable property which does not have any
   additional semantic meaning is discouraged.  For example, a key leaf
   that is given a value and cannot be modified once a list entry is
   created.

Not sure what this is trying to say. So what are the conditions under which a server can declare something to be immutable?
[Qiufang] If immutable flag is used but cannot provide extra information, it should not be used, like a key leaf node is already immutable once the list entry is created. Will reword this sentence in the next version.

And if I have some immutable data in a list, can I delete the list?
If not, what is the error that I can expect to get?
[Qiufang] Yes, the user can always delete any immutable configuration in <running>, which simply means make it invisible  in <running>. Since it is created by the server in <system>(if exists), this would not cause the server to delete and stop using it.
{JS} which immutable data is the server adding (removing?) to/from <running> and when? which immutable data is the client adding (removing?) to/from <running>? do you ensure that there are not any ping-pong effects?

   The "immutable" flag is intended to be descriptive.

What does this mean? The YANG extension that is to be used at data model writing time. If a YANG module author declares a subtree to be immutable, then this means exactly what for an implementor (of servers and clients)?
[Qiufang] The intention is that, the immutable YANG extension-statement is used to describe an existing behavior, it applies to the server which already has the immutable configuration in the past. We are not defining mechanism to restrict server's behavior, but conveying what they've been doing all along in machine-readable behavior, there is nothing new for the implementor (of servers and clients).
{JS} this does not align well with what was said elsewhere; you now tell us that servers can do what they do anyway and the immutable mechanism just exposes the server's behaviour to the clients; this is exactly what I am concerned about; server's getting a lot of freedom and the clients have to pay the price to adapt, which is a pain for clients dealing with many different servers
 
   The "immutable" YANG extension can be a substatement to a "config
   true" leaf, leaf-list, container, list, anydata or anyxml statement.
   It has no effect if used as a substatement to a "config false" node,
   but can be allowed anyway.

I am not sure whether this fits the YANG design, are there other examples where YANG allows non-sensible statements just to ignore them? Perhaps Martin knows best.
[Qiufang]I think it is okay, e.g., see https://datatracker.ietf.org/doc/html/rfc7950#section-7.14.2, all "config" statements for the data nodes in  the input tree are ignored. Or "ordered-by" statement is ignored if it’s inside a state list, etc. Make sense to you?

Some of the text needs editorial work or better descriptions. An
example:

           "If servers always reject client modification attempts to
            some data node that can only be created, modified and
            deleted by the device itself, an 'immutable' YANG extension
            can be used to formally indicate to the client.

The logic feels backwards to me. The server rejects modification attempts because some data is considered immutable by the server. The text suggests it is the other way round, or worse, makes no sense (consider a server rejecting a request repeatedly due to a lack of permissions, this does not imply that the data is now immutable). And the immutable statement is about designing at data model design time that certain leafs are immutable. The server's behavior follows the model, not the other way round. And I would consider an immutable definition in the data model to actually be prescriptive in some form or the other.
[Qiufang]Sure, the authors will reword this sentence as well as preclude the factors of NAC model. I agree that the server's behavior follows the model, the immutability might be documented as plain-text in the description statement before, and now we hope there is a formal and machine-readable way to describe it.


            Adding immutable of an existing immutable statement
            is non-backwards compatible changes.
            Other changes to immutable are backwards compatible.";

What?
[Qiufang]This indeed needs editorial refinement! Thanks for pointing it out. 
 
I am also puzzled what the 'value' of the immutable extension statement is. The definition needs to be rewritten so that it defines precisely what this extension statement is.
[Qiufang] The value is a Boolean type with true or false, sure, will rewrite the definition of the extension-statement. 

The description of the immutable annotation has similar issues, except that for this one we know that the value is a boolean value.

           "define the with-immutable grouping.";

This is the kind of descriptions nobody needs. Perhaps explain what this definition is good for in order to establish the context for the stuff in the grouping and how it is used later.
[Qiufang] Thanks, will fix in the next version.

The example in A.1 is not really convincing. For the leafref to work, you need a set of immutable leafs, one for each timer value supported by the server. Are people really doing such constructions for timer values?? I understand that you want a leafref pointing to one out of N supported values, the timer value story, however, sounds weird. Do you have more convincing examples that are still easy to understand?
[Qiufang]Hope Balazs can chime in, given he provides this example.  It seems okay to me, such configuration is hardware-related, and its value is determined by the hardware capability/characteristic, it has to be declared as "config true" because of the need of being referenced by other configuration.
{JS} multiple config true leafs for all possible timer values in order to reference them?

The example in A.2 confuses me. Interface configurations are applied to physically present interfaces. There is a name binding of the interface configuration to a physical interface. This section seems to suggest something very different from how I understand RFC 8343.
[Qiufang] I don’t see much inconsistency with RFC 8343, Can you point out what you think is problematic? Don’t you think it’s appropriate to define interface-type as immutable? As my understanding, it is a system-defined value and cannot be changed by the client with different values that cannot match its real type.
{JS} I think the RFC 8343 model is that discovered physical interfaces appear in operational and if interface configuration refers to a physical interface, then the config gets applied if possible. If, for example, the type does not match, the interface config is not applied. Making type immutable assumes that an interface can only support a single type. RFC 8343 does not require this. Perhaps the only point you want to make is that the type leaf could have been marked as immutable and I am just confused by the discussion that is not needed to make that point.

I have no idea why A.2.1 is there, it seems to show normal NC behavior or did I miss something? That said, what I am indeed missing is a clear definition which error is returned if an attempt is made to mutate an immutable leaf (well to a value it does not have).
[Qiufang] Good catch. We will define a dedicated error message for this specific case in the next version. Thanks a lot.
{JS} but why is this example there in the first place? RFC 8343 already documents that the server can reject certain type values.

In A.3, if we have entries in lists that are immutable because the server believes this is good for the customer, then I am concerned that this stuff is not a static rare thing but something that will come and go dynamically. I see great potential for abuse of such a mechanism. Once the client needs to adapt to server logic to determine the mutability of things, we are on a slippery slope.
[Qiufang]Not so sure about this since immutability is defined to be rather static now. Some servers provide built-in configuration which cannot be modified by the client, because the configuration might be referenced by other clients, like predefined rules/policies, do you think it as an abuse?
{JS} the actual definition is the problem as explained above

/js

Best Regards,
Qiufang


On Tue, Sep 05, 2023 at 06:40:09PM +0200, Jan Lindblad wrote:
Jürgen,

I agree completely with you regarding transactional management. Well said. 

In my reading of -08, however, I don't see the non-transactional behavior you describe in there any more. It was there in -01, but based on feedback from me and others, I think it has been washed out. If you still find non-transactional behavior in the latest rev, could you please point it out to me as well? My support too depends on the absence of such language.

I believe the document targets config true data that can never really be changed by any client. This data remains unchanged/unchangeable except possibly during a software version upgrade, entitlement/license change, hardware insertion or similar system-redefining event. Such data is config true simply because there are config true dependencies, such as leafrefs or must conditions on min and max range values. I think this kind of usage could be a relevant use case, and I'd support discussing ways to describe that in the WG.

Best Regards,
/jan



On 5 Sep 2023, at 18:05, Jürgen Schönwälder <[e-mail]:

I do not support this work. Yes, it is some effort on the server 
side to figure out how move from config state A to config state B. 
This proposal essentially pushes the work towards the clients that 
now have to figure out how to single step a server from config state 
A to config state B - and if things fail on intermediate steps it is 
the clients that have to figure out how to roll back the changes to 
get back to config state A (if that is possible at all). The 
original promise of NETCONF was that it is not necessary anymore to 
have clients that please the server, which was comiing out of many 
years of experience with protocols that required clients to please the servers.

If implementations have constraints on what kinds of edits are 
possible, then this is an implementation limitation and hence this 
should be documented in deviation statements but not in the data 
models.

/js

On Tue, Sep 05, 2023 at 12:42:35PM +0000, Kent Watsen wrote:
>NETMOD WG,
>
>This email begins a 2-week adoption poll for: 
>https://datatracker.ietf.org/doc/draft-ma-netmod-immutable-flag/08
>
>There is no known IPR on this draft (IPR call <https://mailarchive.ietf.org/arch/msg/netmod/_S-cKw5jIBmDKEPBRq8KeAbNLGg/>).
>
>Please voice your support or technical objections to adoption on the list by the end of the day (any time zone) Sep 19.
>
>Thank you,
>Kent (as co-chair)
>

>_______________________________________________
>netmod mailing list
>[e-mail]
>https://www.ietf.org/mailman/listinfo/netmod


-- 
Jürgen Schönwälder              Constructor University Bremen gGmbH
Phone: [phone]         Campus Ring 1 | 28759 Bremen | Germany
Fax:   [phone]         <https://constructor.university/>

_______________________________________________
netmod mailing list
[e-mail] <mailto:[e-mail]
https://www.ietf.org/mailman/listinfo/netmod


-- 
Jürgen Schönwälder              Constructor University Bremen gGmbH
Phone: [phone]         Campus Ring 1 | 28759 Bremen | Germany
Fax:   [phone]         <https://constructor.university/>

_______________________________________________
netmod mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/netmod

-- 
Jürgen Schönwälder              Constructor University Bremen gGmbH
Phone: [phone]         Campus Ring 1 | 28759 Bremen | Germany
Fax:   [phone]         <https://constructor.university/>