Re: [netmod] system configuration sync mechanism

[Andy Bierman <[e-mail]>]

On Sat, Jul 31, 2021 at 9:26 AM Balázs Lengyel <balazs.lengyel=
[e-mail]> wrote:

> Hello,
>
> Sorry for going back to the basics, but IMHO it is needed here. So as I
> see understand the problem:
>
> The purpose of the draft and some principles should be clearly stated. In
> my view:
>
>
>
> *P1) “There is a need to validate configuration data against data created
> by the system.”*
>
>
>
> We also have some principles:
>
> *P2) We want the running datastore to contain exactly what the operator
> has written there*
>
>
>
> From this it flows that
>
> *P3) the running/intended configuration MAY become invalid if the
> system-data changes. *
>
> But
>
> *P4) rfc8342#section-5.3.4 MUST always be valid*
>
>
>
> P3 and P4 contradict each other. The only way to resolve this IMHO is to
> say that system-data changes only at upgrade. If the upgrade can’t end up
> with a valid configuration (after whatever processing) then it MUST fail.
>
>
>
> IMHO
>
> If we move the system-data into a separate datastore that does not help.
> When I update the running configuration the validation will depend on
> system-data either in the system or in the intended datastore. Debugging a
> configuration distributed over multiple datastores is difficult.
>
> IMHO we should just introduce a new datatype beside config=true and
> config=false, let’s call it system-data or read-only-config=true.
>
> System-data would be the same type as config=true except it is not
> writable by Netconf/Restconf/CLI/SNMP, etc. only by the system itself.
>
> This would allow us to define and populate system-data in both
> running/intended datastores. It would be visible in the same context.
>
> System-data would be static, the operator would not be able to modify it,
> so impact on netconf/Restconf operations would be trivial.
>
>
>


I agree a new datastore will just add complexity without any value.
Your solution approach is better, but I think it would require a new YANG
version
to allow config node XPath to reference non-config nodes.

Another solution is to model the referenced node as config=true, but setup
automatic NACM rules so no user editing is allowed. This works well for
setting up an initial config that gets saved and not changed unless a reset
is done.

What if <intended> is what is NV-stored?  When that occurs, the config
changes from system to user config.
Routers have been saving the combined config for decades. IMO the standards
intentionally avoid discussing the conversion of a datastore to/from
NV-storage.


As I see it this is the same problem that we discussed in
> https://github.com/netmod-wg/yang-next/issues/41.
>
>
>
> I know this is a radical change, but I think using a new YANG extension to
> create a read-only config=true datatype is a much cleaner solution. One
> which some companies have already implemented.
>


+1

Actually, separate running and system would be a radical change, not this.
Cleaner and less disruptive.


If I misunderstood your intent, then sorry.
>
>
>
> Regards Balazs
>
>
>


Andy


> *From:* netmod <[e-mail]> *On Behalf Of *maqiufang (A)
> *Sent:* 2021. július 31., szombat 14:33
> *To:* Kent Watsen <[e-mail]>; [e-mail]
> *Subject:* Re: [netmod] system configuration sync mechanism
>
>
>
> Hi, Kent,
>
> Thanks for helping me revive this thread, which is exactly what I want to
> do.:)
>
> There was not a very fully discussion due to time constraints, but we did
> see some valuable points here, thank you everyone for sharing your views.
>
>
>
> Regarding option2,  I am still unsure how will things go if there is no
> <intended>(I think it was raised by Balazs, hopefully Balazs can also add
> something here)? Should <system> be implemented along with <intended>?
>
>
>
> Option 3 is still unclear, e.g., whether the <system> is copied into
> <running> automatically or manually? If auto-copy is not a good idea
> because it violates the definition of <running>, whether manual-copy is
> performed towards part or all of the system configurations created in
> <system>?
>
> Should we copy the entire <system> into <running>? Or should there be as
> few system configuration data items in <running> as possible?
>
> Anyway, I agree that option3 may still incur a failed validation of
> <running> when the operators reference the system configuration which is
> produced through the expansion of the system-defined templates.
>
> If the existing mechanism(e.g., edit-config)is sufficient to define
> referenced system data item in <running>, it seems that the flow marked in
> option3 from <system> to <running> can be removed, then it looks no
> difference between option1 and option3.
>
>
>
> Best Regards,
>
> Qiufang Ma
>
>
>
> *From:* netmod [mailto:[e-mail] <[e-mail]>] *On
> Behalf Of *Kent Watsen
> *Sent:* Thursday, July 29, 2021 1:09 AM
> *To:* [e-mail]
> *Subject:* Re: [netmod] system configuration sync mechanism
>
>
>
> WG,
>
>
>
> Regarding yesterday’s <system> datastore presentation, there seemed to be
> support for "Option #2”, which is to have <system> merge into <intended>.
>
>
>
> It was noted that this then would mean that client-validation of <running>
> would necessitate understanding how the merge works, to expand templates,
> resolve leafrefs, etc.
>
>
>
> My thoughts are, so?
>
>
>
> Firstly, a client that doesn’t understand that there may be some <system>
> defined configuration will, for the most part, be none the wiser.   The
> client *will* discover <system> configuration in <operational>, but this is
> already the case today.  One new thing is that <operational> should use
> “origin:system” for configuration originating from the <system> datastore.
> This last point might surprise clients…as the definition of “with-origin”
> doesn’t state that clients must ignore any unrecognized “origin”
> identities: https://datatracker.ietf.org/doc/html/rfc8527#section-3.2.2.
>
>
>
> Secondly, no shared object defined in <system> will be activated until
> client-supplied config references it.  But any client able to do this
> already knows how <system> merges into <intended> and is accounting for it.
>
>
>
> Thoughts?
>
>
>
> Kent
>
>
>
>
>
> On Jul 16, 2021, at 6:24 AM, maqiufang (A) <[e-mail]> wrote:
>
>
>
> Hi, Kent,
>
> Please see my reply inline.
>
>
>
> *From:* Kent Watsen [mailto:[e-mail] <[e-mail]>]
> *Sent:* Friday, July 16, 2021 2:55 AM
> *To:* maqiufang (A) <[e-mail]>
> *Cc:* [e-mail]
> *Subject:* Re: [netmod] system configuration sync mechanism
>
>
>
> Hi Qiufang,
>
>
>
> *            [snip]*
>
> The question is if the server implementation prunes dangling/unused
> objects when <intended> is applied, updating <operational>.  My assumption
> is that the server will discard any object that doesn’t actually impact the
> running configuration of the system (i.e., values are consumed by the
> underlying operating system, drivers, etc.).  Thusly, it is my opinion that
> only the referenced objects are applied.  Hence why, to answer your last
> question, I wrote that these configurations (manufacturer-defined objects)
> are not applied immediately but only after they are referenced.  Makes
> sense?
>
> *[Qiufang Ma] Yes, try to sum up our discussion about the categories of
> the system configuration:*
>
> ·       *Physical-resource-dependent--> whether this sort of system
> configuration exists in <system> dependents on if the physical resource is
> present(e.g., physical interface).*
>
> ·       *Physical-resource-independent-->which is provided by the device
> system*
>
> o   *Further classification from the perspective of “applied”
> time(dependents on whether the system configuration impacts the running of
> the system)*
>
> §  *Config that is applied immediately(e.g., the loopback, the predefined
> minimum length of password…)*
>
> §  *Config that is applied only after being referenced by other
> configs(e.g, definitions for applications ftp/tftp…)*
>
> o   *Further classification from the perspective of generation time*
>
> §  *Config that is generated unconditionally at each boot time(e.g,
> loopback, predefined minimum length of password, ftp/tftp…)*
>
> §  *Config that is generated conditionally during the device
> running(e.g., system-generated local-port and remote-port for a new
> established BGP connection)*
>
>
>
> Keep in mind that what is described above is just one aspect of what can
> be in <system>.  In addition to defining reference-able objects, <system>
> can also define/apply configuration immediately (e.g., the loopback
> interface).  That is, configuration not does not have to be referenced in
> order to become activated.
>
> *[Qiufang Ma] Noted.*
>
>
>
>
>
>
>
> Note that, <running> by itself would not pass validation, due to missing leafrefs.  Thankfully, NMDA never says that validation runs on <running>.  But once <running> and <system> have been merged, to become <intended>, the result does pass validation.
>
>  [Qiufang Ma] The referenced instance must also exist for the data to be valid since the require-instance defaults to true if not present. Is this what you had in your mind? Yes, NMDA says that it is <intended> which is subject to validation. But I also notice that In section 5.1.3 of the NMDA:”<running> MUST always be a valid configuration data tree, as defined  in Section 8.1 of [RFC7950]. ” So my thought here is that <running> should also conform to the YANG model constraints and that’s to say, a referenced system-defined data item should also exists in <running>.
>
> Therefore, if system configurations do not exist in <running>, they still need to be  configured in <running> manually in order for being referenced. In this case, the original purpose of predefining some system configurations for user convenience is lost. This is the reason why we would like to define some mechanism here to synchronize <system> into <running>.
>
>
>
>
>
> I see in RFC 8342 "<running> MUST always be a valid configuration data
> tree, as defined in Section 8.1 of [RFC7950]”.
>
>
>
> But the question remains if it is possible for the system is able to
> validate <running> without, e.g., expanding templates.  There may be a
> 'leafref' or ‘must’ expression somewhere that will fail because the
> evaluation occurs without expanding a template that supplies the missing
> parts.
>
> *[Qiufang Ma] On condition that <running> should be valid, the operators
> will need to retrieve from the <intended> or <operational> to get the
> template-expanded configurations and then create them in the <running>,
> right?  *
>
> *My feeling is that it loses the meaning of predefining and seems no
> differences between operator-defined configurations if operators have to
> create system configures in <running> before they use them. So I am
> beginning to think, if it’s possible to expand the system-defined template
> during the copying between <system> and <running>?*
>
>
>
> If this draft “updates” RFC 8342 (NMDA), then it can supply a clarifying
> statement about what it means that "<running> MUST always be a valid
> configuration data tree”.  Either that, or an Errata if it’s determined
> that the statement isn’t correct.
>
>
>
> You make a good technical point, but I think that we should *want* to
> avoid having to copy <system> (or <operational>) configuration into
> <running> if we can avoid it.  Agreed?
>
> *[Qiufang Ma] tend to agree. Maybe we should try to avoid it, unless we
> have to.*
>
>
>
> FWIW, also in RFC 8342, Section 5.1.4.:
>
>
>
>    <intended> is tightly coupled to <running>.  Whenever data is written
>
>    to <running>, the server MUST also immediately update and validate
>
>    <intended>.
>
>
>
>    <intended> MAY also be updated independently of <running> if the
>
>    effect of a configuration transformation changes, but <intended> MUST
>
>    always be a valid configuration data tree, as defined in Section 8.1 <https://datatracker.ietf.org/doc/html/rfc7950#section-8.1>
>
>    of [RFC7950] <https://datatracker.ietf.org/doc/html/rfc7950#section-8.1>.
>
>
>
>
>
>
>
>
>
> > I am wondering if these configuration will present in the <operational> (which contains all the configuration actually used by the device) before they’re referenced.
>
>
>
> I think that it would depend in the specific server’s behavior, regarding if *unused* predefine objects are present in <operational>.  Certainly the unused objects would not have to be present in <operational>.  If I were implementing the server, the unused objects would NOT be present in <operational>.
>
> [Qiufang Ma] Yes, if the predefined system configurations is unused, then I also tend to agree that they would not be present in <operational> but may also depend on the vendor implementation.
>
>
>
>
>
> Yup, this is the same conclusion is in my response above.
>
> *[Qiufang Ma] :)*
>
>
>
>
>
> > It would be good if we could determine if there are any other "resource-independent” configuration categories here.
>
> > [Qiufang Ma] Do you think there exists conditional system configuration (if the preceding configurations you mentioned above is not)? For example, if SSH is enabled on a device, SSH-related keys are automatically generated. Such configurations are generated at the moment when a special functionality is enabled.
>
>
>
> I’m unsure what you mean in general by "conditional configuration”, but I can speak to your specific example. Though I must preface my comments that I imagine there are a number of ways servers might go about enabling `sshd`.  What follows is my personal view, forged by being around systems for awhile  ;)
>
>
>
> In general:
>
>
>
>          - `sshd` is NOT enabled by default.
>
>          - `sshd` is enabled via a configuration knob.
>
>          - the SSH host key is dynamically generated the first time `sshd` is enabled.
>
>          - the SSH host key itself is in <operational> (not <running>)
>
>
>
> This view is consistent with the first paragraph in Section 3 of the “keystore” draft (reproduced below):
>
>
>
>          3.  Support for Built-in Keys
>
>
>
>             In some implementations, a server may support built-in keys.  Built-
>
>             in keys MAY be set during the manufacturing process or be dynamically
>
>             generated the first time the server is booted or a particular service
>
>             (e.g., SSH) is enabled.
>
>
>
> As a closing thought, this model (which I stated upfront may not be universal) would have no presence-in or interaction-with <system>…though, perhaps, there may be some predefined values for what key-algorithms and/or key-lengths to use when generating the SSH host key...
>
> [Qiufang Ma] By “conditional system configuration”, I was meaning some of the system configurations are not generated immediately after the device is powered on. Instead, they are generated when a specific condition is satisfied during the device running(e.g., a functionality is enabled due to some client configurations). I am not sure if it really exists, maybe not, just try to explore the possibilities of various resource-independent system configurations.:)
>
>
>
> To this point I agree..as does RFC 8342 (NMDA), Section 5.3.3.:
>
>
>
>    Sometimes, resources are controlled by the device and the
>
>    corresponding system-controlled data appears in (and disappears from)
>
>    <operational> dynamically.  If a system-controlled resource has
>
>    matching configuration in <intended> when it appears, the system will
>
>    try to apply the configuration; this causes the configuration to
>
>    appear in <operational> eventually (if application of the
>
>    configuration was successful).
>
> *[Qiufang Ma] See above, as I summarized as the system configurations that is generated conditionally during the device running.*
>
>
>
> Firstly, I again have to preface my comment that there are likely many ways that templating mechanisms can be defined.    But, in general, once a templating mechanism has been defined, then it stands to reason that templates could be defined either in <running> (by operators) or in <system> (by the manufacturer).  In one implementation I’m familiar with, the templates are objects that are referenced/parameterized by other parts of the configuration.  (Same as with the predefined objects discussion above.)
>
>
>
> To answer your questions:
>
>
>
> 1) Yes, it is my opinion that *activated* templates in <system> will be expanded and present in <intended>.
>
>
>
> 2) I would never suggest that the system-defined templates are present in <running>, though they may be referenced/parameterized by config in <running>.
>
>
>
> 3) if a config-template is configured in <running> (i.e., it is operator-defined) then, yes, the expanded configuration in <intended> is "client configuration” (note, "client configuration” is not a formal term).  That said, it seems fair to say that a template defined in <system> and then referenced by "client configuration” in <running> is also expanded as "client configuration” in <intended>.
>
>
>
> 4) I don’t not understand your last sentence, that the expansion of <system> templates are only present in <operational>.  Maybe you’re saying something subtle, e.g., that servers currently don’t support GET on <intended>.  But, in theory, the expansion of <system> templates should (IMO) be present in <intended>, so that they may be subject to validation.   Of course, all the <intended> configuration (whether originating in <running> or <system>) that is successfully “applied” will also be present in <operational>.
>
> [Qiufang Ma] Assume that there is no <system> and this work, the expansion of system templates are only present in <operational>. Because this is compatible with system configuration definition in NMDA.
>
> But if system configurations are only present in <operational>, the predefined system configurations still need to be retrieved and created into <running> explicitly when being referenced. I think we’ve reached an agreement on the need for <system> to exist, and our main point of disagreement is whether <system> should be copied into <running>. Your point is that being merged into <intended> is enough to make sure a success validation. But my understanding is that the referenced system configuration data item must also exist in the <running> to obey the model constraints.
>
>
>
> Yes, I believe that you provided an accurate description of the difference
> in our opinions.  Per my earlier response, you make a valid technical
> point, my goal is to waive that interpretation to the side so that a
> simpler solution can emerge.  It would be good to get other opinions on
> list, otherwise we’ll take it into the meeting.
>
> *[Qiufang Ma] OK.  Hopefully someone else would share some opinions here.
> Otherwise let’s take this into the IETF meeting.*
>
>
>
> [BTW, in keeping with this thread moving from the NETCONF to the NETMOD
> mailing lists, would it make sense to move the IETF 111 presentation slot
> from NETCONF to NETMOD too?  I think it does and, further, it would help
> with scheduling (NETCONF is over, NETMOD is under).  Would you be okay with
> this?   AD Rob and the NETCONF chairs discussed this morning, and think
> it's okay, but would still need to confirm with the NETMOD chairs.]
>
> *[Qiufang Ma] I am happy with the proposal, if it’s also okay for NETMOD
> chairs:-). I have sent an email to the NETMOD chairs to request to move
> this presentation slot from NETCONF to NETMOD. *
>
> *A new version of the draft will also be submitted to NETMOD when the
> draft-submitting window reopens.*
>
>
>
>  <big snip>
>
> > I’m beginning to think that:
>
> > ·         auto-copying into <running> is likely never a good idea, because it violates the definition of <running>
>
> > [Qiufang Ma] I am quite aware that different datastores in NMDA represents different views of data nodes.  And <running> represents a configuration datastore holding the current configuration of the device.
>
> > Should we consider system configuration also be part of current configuration of the device? From my perspective, the difference between system configuration and  client-configuration lies only in who provides it.
>
>
>
> <running> holds the current *operator-specified* configuration of the device.  System-provided configuration is NOT specified by operators (though system-defined objects may be referenced by operator-specified config in <running>).   I believe that this arrangement is consistent with the definition of <running>.  Agreed?
>
> [Qiufang Ma] Yes. Actually we are not trying to violate the principles of NMDA and the definition of <running>.  The issue we try to resolve here is that system configurations cannot be used(referenced or overwritten) by the operators directly and need to be created into <running> explicitly. This actually loses the meaning of “predefining and bringing convenience”. If auto-copying is not a good idea, what do you think about defining an RPC operation for the operators to do the copy(which is also what Rob suggests at the meeting)?
>
>
>
> If we have to copy into <running>, then I think that I agree an RPC
> (<edit-config>?) would be better.
>
> *[Qiufang Ma] From my perspective, <edit-config> is feasible but not
> efficient because operators still need to retrieve <system>/<operational>
> firstly. If we could define a RPC to copy the entire <system> into
> <running>, it seems more convenient for operators. However, some system
> configurations which are not going to be referenced or modified may also be
> copied into <running>. I don't have a strong feeling about which one is
> preferred. Anyway, we need to figure out whether it would be fine for
> <running> to missing referenced system configurations.*
>
>
>
> You mention “overwritten” by the operators?  Why wouldn’t the operators
> just define their own?  For instance, if they don’t like the vendor’s
> “vendor-foobar” object, they could copy/paste/edit their own “my-foobar”
> object with the values needed, yes?
>
> *[Qiufang Ma] Yes, defining their own would be okay. By overwriting, I
> mean sometimes the operators would like to modify the specific system
> configuration, e.g., the MTU value of a specified interface(identified by
> its name).*
>
> *If the operators want to modify the system configurations, there is no
> way but redefine them in <running>.*
>
>
>
>
>
> *Best Regards,*
>
> *Qiufang Ma*
>
>
>
>
>
>
>
>
>
> > ·         having in <operational> doesn’t make sense, since the tweaks wouldn’t go thru <running> --> <intended> validation.
>
> >
>
> > I’m wondering if a model like below would work for everyone - thoughts?
>
> > [Qiufang Ma] <intended> represents the configuration after all configuration transformations to <running> have been performed, so I think it is only coupled to <running>.
>
> > Anyway, the <system> should also interacts with <operational>.  Agreed?
>
>
>
> I don’t agree that <intended> must only be coupled to <running>.  Specifically, I think that it is okay (compatible with NMDA) to define a <system> that also impacts <intended>.   This is the only (IMO) sane approach, as it enables the combination <running> + <system> to be validated.
>
> [Qiufang Ma] Please see above. If <running> is OK to miss referenced system configuration, your proposal makes sense to me.
>
>
>
> Ack.
>
>
>
>
>
> * Best Regards,*
>
> *Qiufang Ma *
>
>
>
>
>
> Kent // contributor
>
>
> _______________________________________________
> netmod mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/netmod
>