Re: [netmod] Balazs Review of draft-ma-netmod-with-system-02

[Balázs Lengyel <[e-mail]>]

Hello Qiufang,
While it is possible to understand the draft in a correct way, I still believe that a few things should be much more explicit, would require strong straight statements:
- Definition of what is system configuration
- Update of <system> datastore has no effect on the <running> datastore even if some of it <system> has been copied over to <running> earlier.
- Contents of <running> are modifiable or not following the general YANG and NACM rules even if some of it <system> has been copied over to <running> earlier. Any further restriction will be defined in a separate “immutable” draft.
Etc.

I understand that the immutable draft can work without the system draft, but I  don’t know if the system draft can work without the immutable draft. What is your opinion?
Regards Balazs

From: maqiufang (A) <[e-mail]>
Sent: Thursday, 31 March, 2022 11:43
To: Balázs Lengyel <[e-mail]>; Jan Lindblad <[e-mail]>
Cc: '[e-mail]' <[e-mail]>
Subject: RE: [netmod] Balazs Review of draft-ma-netmod-with-system-02

Hi, Balazs, Jan
Please see my reply inline.

From: Balázs Lengyel [mailto:[e-mail]]
Sent: Thursday, March 31, 2022 5:31 AM
To: Jan Lindblad <[e-mail]<mailto:[e-mail]>>; maqiufang (A) <[e-mail]<mailto:[e-mail]>>
Cc: NetMod WG <[e-mail]<mailto:[e-mail]>>
Subject: RE: [netmod] Balazs Review of draft-ma-netmod-with-system-02

See BALAZS3 below.
regards Balazs

From: Jan Lindblad <[e-mail]<mailto:[e-mail]>>
Sent: Wednesday, 30 March, 2022 16:12
To: Balázs Lengyel <[e-mail]<mailto:[e-mail]>>; maqiufang (A) <[e-mail]<mailto:[e-mail]>>
Cc: NetMod WG <[e-mail]<mailto:[e-mail]>>
Subject: Re: [netmod] Balazs Review of draft-ma-netmod-with-system-02

Hello Balázs, Qiufang,

I've added some comments below as JANL.

Hi, Balazs,
Thanks for your thorough review and valuable comments!
To be brief, I will incorporate some of them to the update, but I think there are still a couple of comments that need further discussion, like:

·        Terminology to differentiate between the same(-path) data nodes in different datastores(e.g., system and running).
BALAZS2: MY best idea is to ALWAYS say:
               The “interface data node in <running>”
               “/running/interfaces/interface/name”  - so prefix the datastore name to the path

JANL: Actually, I think this notation is rather confusing. How about :running:/interfaces/interface/name ?
BALAZS3:OK, I tried to follow the Restconf notation, but I am ok with your proposal.


·        Should “copy-config” operation also be augmented to support “resolve-system” parameter?
BALAZS2: IMO yes. If you copy a complete configuration from a file to running, you will still need parts of <system> to make the new content of <running> valid.

JANL: I think this makes sense.


·        The potential NBC issue behind this principle: If the "resolve-system" parameter is not given by the client, the server MUST NOT modify <running> in any way not specified by the client.
BALAZS2: I very strongly oppose this restriction because it is: a bad idea, unenforceable, NBC and would be a problem for other SDOs.

JANL: I could accept watering down MUST NOT to SHOULD NOT.
BALAZS3: Sorry, I know system-set data has its problems, but my arguments still stand.
[Qiufang] SHOULD NOT is fine from my perspective.

·        Will the update of <system> be reflected into <running>? If not, will this cause an invalid <running> datastore?
BALAZS2: I see this similar as an upgrade problem. Following RFC7950 section 11 rules: During an upgrade you may obsolete a schema node, this remove it, or you may add a default value. Both changes may make an existing configuration invalid. System configuration should also be changed only very carefully because it may cause an invalid configuration, in which case the change to system-configuration probably should be rejected / not-allowed.

JANL: At system upgrade, I think autoconfig (system modifying its own configuration) is acceptable. Such changes could be seen as actions made by a system-internal client.
Automatic updates in running is a tricky problem. E.g. will you remove configuration of an interface if the HW is removed? There might be a lot of data nodes configured based on it. There might be leafrefs pointing at the interface.
[Qiufang] Agreed. Automatic updates in running when the system configuration changes should not be allowed. If the intention is to override a value of a system-defined node(the system-defined node is modifiable and has several dynamic default values), any update to the data node may not be cared to sync into <running>. The client can subscribe the <system> update, and determine itself whether any update should be reflected into <running>.

JANL: As many of you know, I'm not particularly happy about such changes (I prefer the configuration to be left intact, but the oper state to change to down/hardware-missing, etc) but if someone insists, these situations too could be seen as management operations executed over a different protocol (the operator-manipulates-the-hardware-protocol). In this case the system-internal client making config updates is responsible for ensuring running stays valid at all times.
BALAZS3:  I see this similar to an upgrade. At time of an upgrade or system-config changes you may need to adjust your configuration. E.g. confd does that. IMHO changing the system-configuration is just as serious matter as an upgrade. It should happen rarely and be done carefully. I don’t have a strong opinion about this yet, but I feel we should describe whatever should be happening.

In section 1) we indicate automatically updating the interfaces depending on HW changes.  I would REALLY !!!!! like a detailed description of this use-case. That would help me understand our plans better.
[Qiufang] I think that Appendix A has already defined some of the related use cases.
As I understand
when an interface is plugged in
- it will be automatically  created in <system> with list-entry, name, type.
- <running> will NOT be updated automatically
The client might copy over the created :system:/interfaces/interface into <running>
The client may use resolve-system property to implicitly copy over the :system:/interfaces/interface into <running>
[Qiufang] True, but note that the configuration will only be copied into <running> when there is a leafref refers to an existing interface, to make <running> valid.
Can a client set a different type to :running:/interfaces/interface[name=if0]/type then what is automatically set in :system:/interfaces/interface[name=if0]/type ?
- If yes, that might lead to confusion, misconfiguration (might use an operationalState leaf to indicate this)
- if no, this is a constraint between 2 datastores that needs to be cleanly explained in the draft and specified in the model
[Qiufang] No, the client cannot set an different value for an immutable system configuration in <running>. I think the draft has already mentioned this: “If a system configuration node is non-modifiable, then writing a different value for that node in <running> MUST return an error.”
One idea for this would be to introduce a new value for the immutable flag (in the other draft) for the schema node “system-defined-value” meaning that the value in <running> and/or <candidate> MUST be the same as the value in <system> (if system datastore is supported) otherwise the same as in <operational> (if that system datastore is not supported but operational is) or the same as a system defined value documented in an implementation specific manner (if neither system nor operational is supported).
[Qiufang] I think immutable flag work has covered this, the only difference is that immutable is defined as annotation in the current draft, not a schema.
Once interface[name=if0] is correctly configured in <system> and <running> what happens if
- the system decides to change the type of the interface? (e.g. swapping between 2 interface card) Will that impact the running configuration? Some parts of configuration maybe meaningless or incorrect for the new interface type. So it is potentially more than just an operationalState=down.

- the interface card is removed?
               -- remove the :running:/interfaces/interface[name=if0] subtree which might make some leaf-ref invalid
               -- always have an operational-state attribute that indicates that either that the substree of the configuration is down or that it is incorrect. The user will need to either delete the subtree via netconf/restconf/cli or update it to make the configuration correct. The second might be a better idea because it retains the user input configuration, it also handles the changed type use-case. However how do we enforce this rule about operational-state?

[Qiufang] The related configuration should not be removed from <running> automatically. NMDA has already defined this (https://datatracker.ietf.org/doc/html/rfc8342#section-5.3.2):

” we allow configuration for missing resources to exist in <running> and
<intended>, but it will not appear in <operational>.”
To be general, I think any system configuration being deleted from <system> should not cause it to be deleted from <running> automatically(if it exists in <running>), given it is very likely that it has already been referenced.

Please see more reply below.


Best Regards,
/jan



From: netmod [mailto:[e-mail]] On Behalf Of Balázs Lengyel
Sent: Thursday, March 24, 2022 2:47 AM
To: '[e-mail]<mailto:[e-mail]>' <[e-mail]<mailto:[e-mail]>>
Subject: [netmod] Balazs Review of draft-ma-netmod-with-system-02

Hello,
I did a detailed review of the system draft. My comments questions are below.
Regards Balazs

===========================================================

General)
I think this work is important and valuable, but it needs quite a lot improvements.

The term system-configuration is used confusingly. Does
system-configuration reside in the <system> datastore only or can it
reside in the <running> datastore too? If system-configuration is copied by
the client (<get-data>+<edit-config>) into the <running> datastore is it
still system-configuration? It is set by the client this time not the system.
[Qiufang] Yes, I agree.
System configuration is provided by the device in <system> datastore, though I think it may also be present in <operational> with origin=”system”.
If it is copied/pasted into <running>, the copied configuration in <running> should not be called as “system configuration”.
But I think it’s okay to say something like “copy system configuration into <running>”, the object to be copied is system configuration which is defined in <system>.
BALAZS2: So the definition of system-config is:
System configuration:  Configuration that is provided by the system itself. It is the configuration that is stored in the <system> datastore or the configuration data stored in the <operational> datastore with origin=system.
[Qiufang]  Would the following definition be better?
System configuration:   Configuration that is provided by the system itself.  System configuration is present in <system> once it's created, regardless of being applied by the device.  Applied system configuration also appears in <operational> with origin="system".
This means that
Some terminology is needed to indicate that you mean a specific data node
IN A SPECIFIC DATASTORE. The same data node (according to the path in the
data tree) in different datastores need to be referenced separately.
[Qiufang]Cross-datastores references is not the intention here.
Any suggestions to make it clear or what does the terminology looks like in your mind?

Does the solution allow conditional system configuration?
(E.g.,  if the client creates an OSPF interface the system inserts a child leaf into it)
[Qiufang] Yes, it does. System configurations which are provided and activated based on specific conditions being met in a system, it is defined as “Conditionally-Active system configuration”.
https://datatracker.ietf.org/doc/html/draft-ma-netmod-with-system-02.txt#section-2.2 describes this kind of system configuration.

1.1) If system shares the same schema as running that would force it to
populate mandatory nodes.  That might be a problem.
State that mandatory or min-elements might not be enforced in <system>.
[Qiufang] I think that mandatory or min/max-elements nodes should be enforced in <system>. There should be no exceptions.
What’s the problem that you think might happen?
1.3) “client may overwrite values of configurations defined in <system>”
However it also states: The contents of <system> datastore are read-only
These seem to contradict. Please clarify.
[Qiufang]It is confusing, indeed. When it says the contents of <system> datastore are read-only, I mean <system> is a read-only datastore, e.g., an <edit-config> operation towards <system> should be refused.  But the client can overwrite values of configurations defined in <system> by writing new values in <running>, which will take precedence over initial values set by the system.
How about this:
OLD:” client may overwrite values of configurations defined in <system>”
NEW:“client may overwrite values of configurations defined in <system> by configuring the intended values in <running>.”
BALAZS2:
New is good, but use the word override instead of overwrite. We don’t actually change the existing values, rather providing another value in <running> that will have precedence.
[Qiufang] Sure, thanks.
1.4) Shoudn't copy-config also be effected? Copy-config
might also need system configured items. It should be mentioned that the same
"resolution" is also needed after a node-restart.
[Qiufang] Are you suggesting to augment copy-config with “resolve-system” parameter also?
This operation is used to replace the target configuration, so you also want the server to write missing referenced system nodes automatically after the full replacement.
But is this still a “copy-config”? Copy usually means the same to me. Do you have any use cases in your mind?
I think we need further discussion, or let’s see if anyone has any other comments on this.
BALAZS2: IMO yes. If you copy a complete configuration from a file to running, you will still need parts of <system> to make the new content of <running> valid. If you think of conditional-system-configuration the above is specially true.
[Qiufang]Noted.
What does populate mean? Is it the same as "copy from system to running" ? If
yes please use that terminology. Populate is not as specific.
[Qiufang] Sure, thanks. I will reword it.

2) In the subchapters (and later) you use the terms provided, activated, applied. I am not
sure what this means. Is a not yet applied item present in the <system>
datastore or only when it is applied? If I do a get-data on <system>
will I receive not-applied data nodes?
[Qiufang] This might be confusing. I will try to define/delete these terms, if they are not defined in NMDA.
A data item will be present in <system> once it is provided by the system, regardless of whether it is applied.
Note that NMDA already defines the term “applied configuration”, which is configuration that is actively in use by a device.
“Activated” should be removed and replaced with “applied”.
What is the difference between an applied and an activated data node and an
applied but not activated data node?
[Qiufang]I think there is no difference, both mean that the configuration is in use by a server. Or maybe you want to ask what is the difference between a generated and applied data node and an generated but not applied data node? Regardless it is applied immediately when it’s generated, once system configuration is generated, it will appear in <system>. System configuration that are not applied yet is unlikely to appear in <operational>, but it may also depend on the device implementation.
I would rather see terminology like:
- is present in the <system> datastore
- is not yet present in the <system> datastore, but the system will create it
in the <system> datastore when a condition is fulfilled.
[Qiufang]Sure, will try to make it clear in next version.
How is it defined for specific schema nodes which kind of system-data it is ?
Free English text?
Is it needed to define this formally or is it enough if the server knows this?
[Qiufang] Current draft doesn’t define any specific schema to indicate which kind of system-data it is.
Although this work tries to explore all different kinds of system configuration, I don’t see a compelling reason to define a dedicated schema to identify each kind.
But I think that the client can understand each kind of system configuration by some ways, e.g., Configuration which is only present in <system> and not <operational> is  inactive-until-referenced, and configuration which is generated and present in <system> when a specific feature is enabled is conditionally-active system configuration.
2.2) Isn't the best example for this, when the functionality is licensed and
the license key is inserted?
[Qiufang]Sec.2.2 gives two examples for conditionally-active system configuration, one for hardware-related resource condition driven, and the other for software-related functionality resource driven.

Are you saying that QoS is not a good example? There used to be some discussion, I think we all agree that when QoS feature is enabled, QoS related system configuration will be generated. But licensing a functionality could also be the case from my perspective.

“I agree there can be dynamically added system config (e.g. create a new qos policy, and some queue list entries are automatically created inside that policy).”
https://mailarchive.ietf.org/arch/msg/netmod/ERmwCg6fVkPtbYn_Vkcgf9J6dJU/
https://mailarchive.ietf.org/arch/msg/netmod/919hV5ql3aI87ymvmbQX4yPwWCs/
3.1) <factory-default> is also read-only so why is that better to store
deletable data ? Did you mean that system-config originated data cannot be
delete even if it is copied over to running? Is that true both for explicit
NBI originated copy and copy due to resolve-system?
[Qiufang]I believe that this was discussed both on the mail list and at NETMOD's interim meeting last October.
A client can configure/override a system-instantiated object in <running>, it can also delete it from <running>( and I agree with you that once it’s present in <running> we cannot call it “system configuration” actually, and the value initialized by the system is overwritten even it’s the same value).
But anyway the contents defined in <system> will be merged into <intended> and <operational>. Therefore there is no way to delete a system configuration which is defined in <system>.
But <factory-default> can be used to initialize <running>, and once it’s deleted from <running>, it is totally deleted from the device.
And to answer your question, yes, system configuration defined in <system> cannot be deleted even it is copied over to running, it’s true both for explicit copy and copy due to resolve-system parameter.
3.2) If something was populated/copied over to running/candidate will/should
any changed system values be copied over again thereby updating the
running/candidate datastores?
Can this result in the running becoming invalid?
[Qiufang]Are you asking will any changes in <system> be reflected into <running>/<candidate>?
That’s a good question! Currently I suppose no, and I don’t think that will result in <running> becoming invalid, since the contents in <running> keep unchanged.
On the contrary, I think if the update being reflected into <running> MAY cause <running> to be invalid.
I don’t think all the update in <system> datastore should be reflected into <running>, e.g., an intentional override operation should not care the updated system-instantiated leaf instance value.
Since this document defines a <system> datastore, the client can use YANG notification to aware any system configuration updates, and reflect the update into <running> by itself if needed. Does this make sense?

4.1)  You write
"The client may reference nodes defined in <system>, overwrite values
   of configurations defined in <system>"
IMHO the data nodes in <running> and <system> are 2 different things even
if they reside on the same path in the data tree. You need to find
terminology to differentiate between the same(-path) data nodes in different
datastores. The current terminology is confusing, I need to guess which
datastore you mean. I think this guessing process might hide problems.
Do you mean here: "The client may reference nodes defined in <system> if they are
copied into <running>/<candidate> as a result of an explicit copy or
resolve-system parameter." For me referencing a data node in running and
referencing a data node in <system> (even if they share the same address in
the data tree) are 2 separate things. I don't think you want to create a
reference that point between datastores.
Do you mean here: "overwrite values of the data nodes that were created by
copying from the <system> datastore."
[Qiufang] Maybe I’ve caused some confusion.
Forget the <system> datastore for the moment. The client usually has the desire to reference a system-defined node, or overwrite values of system configurations, right?
The <operational> datastore only contains those which are actively in use, there is no standard mechanism for the client to see what system configuration is available in a server.
<system> is defined as a standard mechanism to allow the client to retrieve system configuration. So that it can reference/overwrite system configuration, by the client writing configuration to <running> that overrides/copies the system configuration in <system>.
<system> is only defined to retrieve available system configuration.
"<running> MAY overwrite and/or extend <system>" this means that the
data nodes in system are modified although they are readOnly.
Is this what you mean? Clarify!
[Qiufang] <system> itself is read-only. Any attempts to modification (e.g., <edit-config>) towards <system> datastore should be rejected.
But this doesn’t mean there is no way to overwrite/extend system configuration. Sec4.4 illustrates the details about overwriting and addition.
I will see how to make it clear in the next version.
BALAZS2: Again  as I understand it: we do not overwrite system configuration, we might overwrite data nodes in running that were originally copied from the <system> datastore.
[Qiufang]I am worried that we are paying excessive attention to the wording. The system initializes a value which the client does not like, and the client uses an <edit-config> to configure the desired value into <running>. <operational> will show that data with origin=”intended”. Isn't this overriding the system configuration? Actually, a system-initialized configuration is overriden.

"Note that only <system> aware clients copy
   referenced system nodes from <system>"
How does the server know if the client is system-aware? It would be better to
state something like:
'In order for the system configuration to affect validation the client needs to
either use the resolve-system parameter or explicitly copy system configuration
into running'

[Qiufang]Yes, the draft already says something similar to your proposal.

Sec4.1:” Clients MUST either

   explicitly configure system-defined nodes in <running> or use the

   "resolve-system" parameter.”


Last para: The server has no way to know if the client is system aware. Once
the data nodes are copied into <running> there is no need to say more.
[Qiufang] There is no need for the server to understand if the client is <system> aware.
But I’ll think it important to state in the draft that the copied configuration driven by the “resolve-system” will also be returned in a read back of <running> datastore, and this behavior also applies to legacy clients, to avoid any NBC issues.
4.2
"If the "resolve-system" parameter is not given by the client, the server
   MUST NOT modify <running> in any way not specified by the client."
I very strongly OBJECT.
- It is a bad idea.
- This is a big NBC change to Netconf/YANG.
- Other SDOs (3GPP, O-RAN) depend on the capability to modify <running>. They
have data nodes where it is stated that list entries are not created by the client.
- This would need a revision 2 of YANG.
- It is also unenforceable. It would be possible to work around it.
The system instantiates an onboard client to do the changes AND the system
prohibits the change for other clients.
However this is just a more complicated way of stating that the system itself
modifies running; we gain nothing but make the world more complicated.
[Qiufang]When we first start this work, a lot of folks agree that clients will benefit from a server which will not do anything the client doesn’t explicitly ask for.
We usually want a read-back of <running> contains only what was explicitly sent by the clients. This even used to be one of the objectives stated in the draft.
But I’ve seen your points here. Let’s see if there is any other comments or suggestions. Otherwise I’d prefer to change this with a recommendation behavior.
4.3
Paragraph-1 sentence 2 & 3 are trivial thus not needed. If you configure
something in running it becomes part of running independent of this draft.
[Qiufang] Yes, I agree. These can be removed.
Mention that the system itself  can also copy over parts or the complete
system configuration into running.
[Qiufang] Sure, will do, thanks!
4.4
In some cases, a server may allow some parts of system configuration
   to be modified.  List keys in system configuration can't be changed
   by a client, but other descendant nodes in a list entry may be
   modifiable or non-modifiable.

 This contradicts the statement that the <system> datastore is readOnly.
[Qiufang] See my clarification above, will see how to refine this statement.
"Client configuration statements in <running> take precedence over system
configuration nodes in <system>"

Instead of hiding this sentence in the middle of a subchapter, there should be
a separate chapter about merging running and system into intended, stating that running has precedence.
This a tier 1 important statement !
There could be some interesting corner cases.
[Qiufang] Okay, noted.
Once the data is in running, AFAIK the knowledge about why is it there is lost,
so terms like "client configuration" are hard to understand. That sounds more
like a use-case than a rule.
[Qiufang] Sure, I will just say “configuration defined in <running>”. Is this better?
BALAZS2: Yes
"While modifying (overriding) system configuration nodes may be
   supported by a server, there is no mechanism for deleting a system
   configuration node."

Once the node is in the <running> datastore if it is not mandatory it is
possible to remove it. What prevents it? What if it was the client that copied
the configuration into <running>? Is the client forbidden to remove something
that it created itself? I don't think so.
[Qiufang] No, the client is not forbidden to remove something that it created itself.
But even the node is removed from <running>, what’s defined in <system> datastore can still be merged into <intended> and applied by the device.
Thus there is no mechanism for deleting a system configuration node from the device’s perspective. Make sense?
BALAZS2: Makes sense and I agree, but I would propose the wording:
"While modifying (overriding) system configuration with data nodes in the <running> datastore may be
   supported by a server, there is no mechanism for deleting a
   configuration node from the <system> datastore."

5.
"datastore does not have to persist across reboots."
'I would say: The content of the datastore is removed at reboot and
re-created by the system with the same or changed content.'
IMHO it is important to state that there will be some reasonable content in the
<system> datastore even if it might have changed.
[Qiufang] Sure. What I have mentioned in the draft is that reboots will cause the contents of <system> to be lost.
I did not mention something in the draft that reboots will also cause contents change in the <system> datastore after re-loading/re-creating. But this is obviously true from my perspective.
BALAZS2: The re-0creation of (some) system configuration is what I am missing after a reboot.
7.1
"Comment: How does a RESTCONF client know if the RESTCONF server
   implements the "resolve-system" parameter?"
Make it a capability in the hello message like with-defaults.

[Qiufang] I am thinking that maybe no need to define a capability identifier, and YANG library can also help for RESTCONF client.

RFC8527 says that an NMDA-compliant RESTCONF server MUST support the <operational> datastore and MUST implement the ietf-yang-library module.

It also states:

”   A RESTCONF client can discover which datastores and YANG modules the

   server supports by reading the YANG library information from the

   operational state datastore.”

Make sense?

BALAZS2: Not yet. The resolve-system parameter might be supported without a <system> datastore as I understand. True?

SO does the support for the YANG module indicate the support for the resolve-system parameter? Isn’t that an overkill to use YANG modules to indicate support for a protocol capability? It might work, but is that the right way to do it?

[Qiufang] Yes, I just realize that there is no YANG module for RESTCONF resolve-system parameter. RFC8040 has defined a set of RESTCONF capability URIs to identify the query parameters supported by the server, it makes sense to define and register another capability identifier for RESTCONF resolve-system parameter. Thanks, balazs.



Best Regards,

Qiufang
7.2
The placement of resolve-system is sometimes incorrect. It shall be inside the
<edit-config> element.
[Qiufang] Good catch! I’ve fixed in my local version. Thanks, Balazs.

_______________________________________________
netmod mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/netmod