Re: [netmod] leafref to lists that contain system-controlled entries

["Sterne, Jason (Nokia - CA/Ottawa)" <[e-mail]>]

Hi Rob,
Please see inline.
Rgds,
Jason

From: Robert Wilton [mailto:[e-mail]]
Sent: Tuesday, October 24, 2017 8:26
To: Sterne, Jason (Nokia - CA/Ottawa) <[e-mail]>; [e-mail]
Subject: Re: [netmod] leafref to lists that contain system-controlled entries


Hi Jason,

Please see further comments inline ...

On 24/10/2017 00:58, Sterne, Jason (Nokia - CA/Ottawa) wrote:
Thanks Rob.  Please see below.
Jason

From: Robert Wilton [mailto:[e-mail]]
Sent: Monday, October 16, 2017 6:40
To: Sterne, Jason (Nokia - CA/Ottawa) <[e-mail]><mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
Subject: Re: [netmod] leafref to lists that contain system-controlled entries


Hi Jason,

On 13/10/2017 19:43, Sterne, Jason (Nokia - CA/Ottawa) wrote:
Hi all,

There are a few threads on the mailing list that touch on the concept of system-controlled resources (mostly list entries):

https://mailarchive.ietf.org/arch/msg/netmod/3fTSHIh_MfHzmuDCoicAGiXA2E0
https://mailarchive.ietf.org/arch/msg/netmod/KIsSgKByQWpqYzA4i6Bwc8fuH3w
https://mailarchive.ietf.org/arch/msg/netmod/mjLJdiYErtNG41dJ5bJ5ji07cz0

A few drafts & RFCs also refer to the concept:
https://tools.ietf.org/html/draft-ietf-netmod-revised-datastores-04
https://tools.ietf.org/html/rfc7223

Several vendor implementations have list entries (instance data) that are populated by the server and can be referenced (leafref) from other places in the configuration.  These system entries are useful pre-created policies, interfaces, etc that can then be used (and referred-to) by operators in their explicit configuration.

If those entries are only expected to exist in the <operational> datastore, then in theory any references to them in user created configuration will cause a validation problem in the candidate/running (missing leafref target).

One solution discussed in the mailing lists is to change every reference to lists that could contain a system created entry to a "require-instance false" leafref.  But then some useful validation is lost.  In many cases the model is more correctly "require-instance true" but the set of targets includes the system create entries.

I agree that this is not a good general solution for system created configuration that is always expected to exist on the device because some of the useful validation is lost.

But I think that this solution does work well were the system created entries are truly dynamic in nature, e.g. it seems to work well for the topology YANG module where the topology may be explicitly configured, but would more normally be learned dynamically from protocol interactions, or perhaps be configured via a dynamic configuration protocol.

[>>JTS] OK - I think I follow you here.  You're saying that if there are references to truly dynamic entries, then since those entries will come and go (vs static bootup-time system entries that are always there), references to them will more likely be "require-instance false".   But that does mean the system has to allow references to non-existent entries, and you lose validation. You also risk errors like referencing a name that is just 1 char different from what you really wanted but the system can't tell you that you got it wrong.

Yes.  I don't think that you can solve this during existing YANG datastore validation, as it is defined today.

But I also think that NETCONF/YANG is potentially missing an RPC that is a step beyond validation, but before actually applying configuration.

YANG and the NMDA datastores draft makes it clear that both <running> and <intended> are always valid datastores. This means that the validation rules for these datastores really should not depend on the current hardware in a device if that hardware could be removed or change.  Otherwise, if you allow validation to depend on the current hardware capabilities, then if someone pulls out a linecard, that would cause a previously valid configuration to immediately become invalid, violating the rule that <running>/<intended> are always valid.
[>>JTS] In implementations that support templating today, the <running> is not always valid.  It is only upon expansion of the templates that you end up with a valid datastore (i.e. in the <intended>).  The template may provide missing mandatory parms, provide the actual targets of leafrefs, etc.
  I can't see a way to actually make <running> or <candidate> valid with the concept of templates  (unless we define validity as after a temporary expansion in some post-processed version of the <running> - but that would just be the <intended>).
  This problem of validity isn't specific to dynamic behavior (e.g. hardware coming & going), or to system provisioned objects.  It is a problem just with static config as well.


I think that a potential solution to this problem, is that a new NETCONF RPC could be defined that is a "<should-successfully-apply>" operation.   Processing during this new RPC would be able to check against current system resources, current hardware capabilities, etc, and would be designed to indicate whether the system expects (but does not guarantee) that the configuration would completely apply successfully without any errors or unapplied configuration.
[>>JTS] "completely apply successfully" does not necessarily mean the candidate or running are "valid" against the YANG model.
Configuration datastores would not have to always conform to this constraint, so that if an operator changed or removed a linecard, the configuration would still be "valid" (as per NMDA and RFC7950 rules) but would fail a subsequent "should-successfully-apply" check.
[>>JTS] I see it the other way around.  With template support, a <running> datastore will usually pass the "should-succesfully-apply" criteria but won't be 'valid'.






Another solution discussed is to have the system created entries appear in the <intended> datastore (as part of template/expansion).  That would make validation pass on the intended datastore, but then the candidate/running/startup datastores would not be valid (would be missing leafref targets if any part of the config refers to system created entries).  THis sounds similar to the problem that has been discussed in the past about the fact that templates (in the running) basically mean the running/candidate aren't necessarily valid (until after template expansion, which means only the intended would be valid).

I think that this solution is OK, but not necessarily ideal.

As you say, it means that <running>/<candidate>/<startup> may not be valid, which I see as quite a big down side.  Longer term, I think that it would be a good aim to allow the configuration to be validated off the device, if a client desired to do so.





Another approach could be to actually have those system created entries show up in running/candidate.  That would ensure that references to those entries are valid.  But if the whole concept of templates just cause the running/candidate to not be valid anyways maybe we wouldn't worry about the invalid aspect of references to system created list entries ?

I know that quite a few implementations do this today, but I'm generally not a fan of the system modifying <running>.  It seems that an overall architecture is much cleaner if <running> has a single source of truth and hence can be exclusively controlled by the client.

But I also like the approach where the client (rather than the device) explicitly writes these default entries into the configuration, if they are referenced elsewhere by the configuration, to make the configuration "complete".  E.g. if part of the configuration references the loopback0 interface then also explicitly add the necessary loopback0 configuration to instantiate the "loopback0" interface.  When this configuration is pushed to the device (i.e. using merge or replace operation semantics) then the system should silently accept/ignore the explicit configuration to create the loopback0 interface if it already exists on the system.

[>>JTS] Yes - that is another option and I like it.  If I follow you correctly, the server would never return these system entries in a <get-config> unless the client/operator had already explicitly 'created' them.
So the operator has the option to make the system entries visible or not.
Yes, exactly.




I think the server should still accept references to the system entries even if the client/operator hasn't explicitly created them.  The whole point is that those system entries are there and waiting for operators to use them from the start (without *having* to explicitly create or define them).   In that case the references would be 'dangling' (unresolved) as far as an offline validation is concerned (but a client could select to fix that by explicitly defining any entries they want to reference).
I think that this is OK.

Effectively, I see that being like a static system provided template for configuration that is merged with <running> to form <intended>, which is then validated.
[>>JTS] Yes.  But that means the <running> or <candidate> is not valid (but would pass your <should-succesfully-apply> check) , but the <intended> is valid.







At the moment, IETF, and other SDOs are busy defining standard YANG models, but for those models to end up being truly generic they also need to have consistency about which bits of configuration are always expected to implicitly exist on the device.  E.g. considering the example above of configuration referencing loopback0: if some systems automatically create a loopback0 interface and others do not, then a generic configuration needs to handle both scenarios.

If IETF standardizes YANG configuration templates, then perhaps it would be good to investigate whether some of these "useful default system properties" could instead be embodied into one or more standard device templates?  These templates could then be explicitly referenced in the <running> configuration.  This may allow <running> to be small, but still allow it to be "complete" and able to be validated off the box.
[>>JTS] I'm not clear on this approach.
OK, so this is just an idea:

1) Assume a YANG extension is defined to allow templates to be defined.
2) Further, assume that there is a way to store, and uniquely name some of those templates in a standard place.

So, perhaps IETF could define a template like this, which is stored as a well defined place:

<template>
  <name>ietf-basic-router-v1</name>
  <config>
    <interfaces>
      <interface>
        <name>Loopback0</name>
        <type>ianaift:loopback</type>
      </interface>
      <interface>
        <name>Null0</name>
        <type>ianaift:null</type>
      </interface>
    </interfaces>
  </config>
</template>

Now, when it comes to the configuration file for your device, it could look like this:
  <config>
    <use-remote-template>http://yang-templates.ietf.org/ietf-basic-router-v1</use-remote-template>

    ... rest of config as normal.
  </config>

So, the expanded configuration would include the explicit configuration for Loopback0 and Null0 interfaces, but that would be pulled via use of a remote template (the contents of which is probably already cached on the device).
[>>JTS] Sorry for harping on this point -> but this still means the candidate/running are not valid.  It is only post-expansion that a datastore becomes valid against the YANG model.   I'm not saying this "use-remote-template" idea doesn't have merit, but it still leaves us with the problem of invalid running/candidate.

It is also causing me to think that perhaps the transition from running->intended could also include the addition of the system provisioned objects (if they haven't been explicitly configured by the client as discussed above) as part of that 'expansion'.



How would the templates be referenced in the <running> ?  Can you give me an example ? (is this different than a direct reference to a system created entry that I am talking about ?)
The above is just an idea.  Normally I would expect configuration templates to be defined in the running configuration.  But here I was considering the idea that a template is predefined in some way, and then referenced, so that it doesn't have to be provided inline in the running configuration.



How would this allow <running> to be small ? Do the templates contain the full definition of the system created entries ?
Yes, I was assuming that the template could contain what might normally be represented as system created entries today.  Standard templates could either be defined by SDOs, vendors, or operators, as long as there is a way to reference them.

Thanks,
Rob





Thanks,
Rob





Rgds,
Jason






_______________________________________________

netmod mailing list

[e-mail]<mailto:[e-mail]>

https://www.ietf.org/mailman/listinfo/netmod