Re: [netmod] [Netconf] Alternative YANG library structure for 7895bis

[Juergen Schoenwaelder <[e-mail]>]

On Tue, Dec 12, 2017 at 11:37:58AM +0100, Vladimir Vassilev wrote:
> 
> 
> On 12/11/2017 05:15 PM, Juergen Schoenwaelder wrote:
> > I do not fully understand why you think it is worth to preserve the
> > old YANG module library structure. Clearly, it won't be backwards
> > compatible since an NMDA implementation may list modules that are not
> > implemented in all datastores and an old client talking to a new
> > server will thus misunderstand what is exposed via the old YANG module
> > library structure. Your proposal "looks" backwards compatible but it
> > is not if one takes a closer look.
> > 
> > One can start making changes to say the conformance-type but then this
> > does not solve the problem since an old client does not know what a
> > new conformance type value means. The design team did look into the
> > option of keeping the old structure unchanged some weeks ago and we
> > finally arrived at the conclusion that it gets (i) ugly and (ii) does
> > not really provide backwards compatibility for a client written agains
> > the old YANG module library structure.
> > 
> > Note that with the new proposals on the table, it should be possible
> > to provide a backwards compatible view on YANG library for systems
> > that implement the exact same module set (with the exact same set of
> > features and deviations) on all datastores they support. But for
> > systems where this is not true, it seems better to use new
> > definitions instead of tweaking semantics.
> > 
> > Perhaps it helps if you can clearly phrase the objective of keeping
> > the old structure. What is the goal you want to achieve with that?
> I prefer to call it the current structure for now. I will try to explain. My
> goal is to efficiently build transactional systems based on YANG models and
> open standards.
> 
> For this I need some reusable bricks. Such a brick is the rfc7895
> ietf-yang-libraray module. Even in a system with multiple YANG model
> contexts eventually processing breaks down to a single model context e.g.
> for example a YANG data validation task performed in a command line
> application comes down to:
>  $ yang-data-validator yang-library-data.xml data.xml

I think you can implement

$ yang-data-validator yang-library-data-bis.xml data.xml

and it will work for any datastore data in data.xml with and without
schema mount.

> The command can be called multiple times for different data and different
> contexts but it is the same very well tested simple tool.

Looks like you do not want to make changes but then you have to make
changes. I am not sure how to address such a requirement.

> So my point is why
> can we not use one of the 2 newly introduced methods for instantiating this
> very well designed model for the needs of NMDA.
> 
> 1. Use datastore for instantiation.
> 2. Use schema mount to mount rfc 7895 ietf-yang-library in the
> ietf-datastores list. (an alternative I assume is especially applicable to
> modules like ietf-yang-library that are self-contained)
> 
> In general having a solution that is flexible can easily be trimmed down.
> Adding special cases that are less flexible e.g. the (not-implemented-in
> leaf-list or none at all for the case of fully transactional systems with a
> single model context (no extra work for these)). The advantage is that even
> with the not-implemented-in leaf-list solution or other diff based the data
> processing tasks will eventually convert the differential information to a
> data structure implementing the rfc7895 ietf-yang-library data tree and the
> yang-data-validator application will be backward compatible and reusable.
> With changes that introduce a whole new level of abstraction in place of the
> simple list with module name and revision as keys one can't even reuse the
> groupings defined in that module and that is a problem I try to avoid.

I think you can easily express the ietf-yang-library model in
ietf-yang-library-bis so moving to ietf-yang-library-bis will
give you a clean solution.
 
> Vladimir
> 
> 
> > /js
> > 
> > On Mon, Dec 11, 2017 at 04:53:30PM +0100, Vladimir Vassilev wrote:
> > > On 12/11/2017 12:16 PM, Robert Wilton wrote:
> > > 
> > > > Hi Vladimir,
> > > > 
> > > > 
> > > > On 09/12/2017 11:49, Vladimir Vassilev wrote:
> > > > > On 12/08/2017 07:01 PM, Andy Bierman wrote:
> > > > > 
> > > > > > Hi,
> > > > > > 
> > > > > > A library per datastore sounds too complicated.
> > > > > I am not proposing that.
> > > > I'm slightly lost, because I thought that was exactly what you were
> > > > proposing! ;-)
> > > I propose a solution that keeps ietf-yang-library a data model of a single
> > > YANG context specification doing the datastore specific modeling in
> > > ietf-datastores model instead. The solution can be both flexible
> > > (independent yang-library data instances per datastore as my example was
> > > focused on) or constrained ('not-implemented-in' modules leaf-list). Here is
> > > everything that is needed:
> > > 
> > > module: ietf-datastores
> > >      +--ro datastores-state
> > >         +--ro datastore* [name]
> > >         +--ro (model)?
> > >            +--:(same-as-operational)
> > >            +--:(constrained-to-operational)
> > >            |  +--ro not-implemented-in*       ->
> > > /yanglib:module-state/module/name
> > >            +--:(unconstrained)
> > >               +--ro yang-library-datastore?   identityref
> > > 
> > > YANG:
> > > ...
> > > container datastores-state {
> > >      config false;
> > >      list datastore {
> > >        key name;
> > >      }
> > >      choice model {
> > >          case same-as-operational {
> > >          }
> > >          case constrained-to-operational {
> > >            leaf-list not-implemented-in {
> > >              type leafref {
> > >                path "/yanglib:module-state/yanglib:module/yanglib:name";
> > >              }
> > >            }
> > >          }
> > >          case unconstrained {
> > >            leaf yang-library-datastore {
> > >              type identityref {
> > >                base ds:datastore;
> > >            }
> > >          }
> > >        }
> > >      }
> > >    }
> > > ...
> > > 
> > > IMO ietf-yang-library as defined in rfc7895 is modular and reusable. I do
> > > not see why this has to be compromised.
> > > > > The fundamental point proposed is that the datastore relevant bits
> > > > > are kept in the ietf-datastores module instead of merging everything
> > > > > in a new ietf-yang-library entangled monster module. If needed
> > > > > ietf-datastores can augment ietf-yang-library but ietf-yang-library
> > > > > should be usable on its own without ietf-datastores. The solution is
> > > > > coherent and modular and addresses the problem statement.
> > > > The issue with this is that datastore augmentations to YANG library
> > > > would end up changing the meaning of the existing YANG library nodes.
> > > > E.g. an old client that ignores the datastore augmentations is going to
> > > > get a nasty surprise when the server does not behave how it expects.
> > > > E.g. because the configuration node that it thinks should be there isn't
> > > > there because it only supported in <operational>.
> > > > 
> > > > This was one of the reasons for changing YANG library.
> > > A well written client that finds out unsupported newer versions of
> > > ietf-yang-library (which is reported in the capabilities) or any of the NMDA
> > > modules is deployed should not do any damage. How exactly did you solve the
> > > problem for the bad clients by changing the structure of yang-library in a
> > > incompatible way is something I do not understand.
> > > > In terms of the idea of just re-using YANG library but export a separate
> > > > copy for each datastore I think that this has its own problems:
> > > > - I don't like the idea of returning meta-data along with configuration
> > > > for a <get-data> on any of the configuration datastores.
> > > There is no meta data. The datastores contain only the config false;
> > > /modules-state tree. I think I explained that very clearly.
> > > > - How does a client know whether the YANG library for <running> applies
> > > > to the whole server (as it does today) vs just applies to the <running>
> > > > datastore (as it would for an NMDA server)?
> > > All datastore models are "case same-as-operational" for such systems.
> > > > - It requires more handshaking between the client and server to get the
> > > > schema, since a separate request would be required for each datastore
> > > > that is supported.
> > > Correct. Flexibility and modularity come at a price. IMO modularity is more
> > > important in this case. And there are solutions for the problem e.g. send
> > > all <get-data> requests before waiting for replies. NETCONF supports this.
> > > > So, for me, I think that the only way that this solution works, would be
> > > > to define a new <get-server-metadata> RPC, but even then I think that it
> > > > would make sense to combine the data together into a new YANG library
> > > > structure.
> > > As I said my focus is on keeping ietf-yang-library modular (single YANG
> > > model context). <get-server-metadata> or just adding datastore identities
> > > allowing the client to use <get-data> to achieve the retrieval of the data
> > > is of little importance to me.
> > > 
> > > > At the end of the day, I don't think that a new YANG library is going to
> > > > be were the real cost for supporting NMDA comes from.  I think that the
> > > > real work is supporting <operational> independently from <running> both
> > > > in the client and servers. But I also think that once servers start
> > > > implementing this properly that it will simplify automation, because
> > > > rather than a client having to guess what state a server is in, it can
> > > > actually querey, or be notified of it, without having to write a lot of
> > > > model specific code.
> > > +1
> > > 
> > > Vladimir
> > > > Thanks,
> > > > Rob
> > > > 
> > > > 
> > > > > > I prefer the proposal that was made at the IETF meeting that had
> > > > > > a 'not-implemented-in' leaf-list and a single module list.
> > > > > This constraint is already specified in the text of the revised
> > > > > datastores draft. Clients conforming to the draft can expect servers
> > > > > to comply with the MUST requirement even if there is a separate
> > > > > yang-library data tree for each datastore the constraint of
> > > > > configuration stores mapping to 'operational' should be enforced
> > > > > according to the draft. There is no contradiction here.
> > > > > 
> > > > > That said I would be also be OK with ietf-datastores augmenting
> > > > > ietf-yang-library with such a leaf-list ('not-implemented-in'
> > > > > leaf-list) as a more constrained flavor of the same approach instead
> > > > > of going for independent copies of yang-library data. For any of
> > > > > that to happen change in ietf-datastores.yang is needed and change
> > > > > in the original rfc7895 ietf-yang-library is not needed at all.
> > > > > 
> > > > > Vladimir
> > > > > 
> > > > > > Why is it interesting to have a separate module list for regular
> > > > > > modules and imported modules?
> > > > > > I prefer to keep the conformance leaf and not change the module list.
> > > > > > 
> > > > > > NMDA needs to be possible to implement with a single schema tree
> > > > > > such that a module
> > > > > > is implemented in all datastores, or a subset of all
> > > > > > datastores.  Otherwise it probably won't
> > > > > > get supported in clients.
> > > > > > 
> > > > > > 
> > > > > > Andy
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > On Fri, Dec 8, 2017 at 9:21 AM, Kent Watsen <[e-mail]
> > > > > > <mailto:[e-mail]>> wrote:
> > > > > > 
> > > > > >      CC-ing NETCONF, where the draft is being worked on.
> > > > > > 
> > > > > >      Kent
> > > > > > 
> > > > > > 
> > > > > >      On Fri, 2017-12-08 at 16:34 +0100, Juergen Schoenwaelder wrote:
> > > > > >      > On Fri, Dec 08, 2017 at 04:19:28PM +0100, Vladimir
> > > > > > Vassilev wrote:
> > > > > >      > >
> > > > > >      > > Yes. The default value for yang-library-datastore leaf is
> > > > > >      ds:operational
> > > > > >      > > (the only possible one for the ds:operational datastore). This
> > > > > >      is backward
> > > > > >      > > compatible. If one needs different model for 'running', etc.
> > > > > >      then a new
> > > > > >      > > datastore identity has to be defined  and set in place of the
> > > > > >      default value.
> > > > > >      > > Then this identity can be used to read the yang-library
> > > > > > data with
> > > > > >      > > <get-data>.
> > > > > >      > >
> > > > > >      >
> > > > > >      > Sorry, but I have to ask this: How do I obtain the schema for the
> > > > > >      > datastore (lets call it <running-library>) that reports the
> > > > > >      schema for
> > > > > >      > <running>? Is there another <running-library-library> datastore?
> > > > > >      Will
> > > > > >      > the recursion end? Perhaps it does since
> > > > > > <running-library-library>
> > > > > >      > might have itself listed as the schema defining datastore.
> > > > > > I guess
> > > > > >      > Lada will like these kind of meta and meta-meta datastores.
> > > > > > 
> > > > > >      Not really. Metadata needn't be in datastores.
> > > > > > 
> > > > > >      Lada
> > > > > > 
> > > > > >      >
> > > > > >      > /js
> > > > > >      >
> > > > > >      --
> > > > > >      Ladislav Lhotka
> > > > > >      Head, CZ.NIC Labs
> > > > > >      PGP Key ID: 0xB8F92B08A9F76C67
> > > > > > 
> > > > > >      _______________________________________________
> > > > > >      netmod mailing list
> > > > > >      [e-mail]  <mailto:[e-mail]>
> > > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=5qj6BQUSwqYmkAVeKz5axFV8k3gxYEPSJ5Cp0RSnxrE&s=I7fR1GY5lN2hVMkDuvryrhDeRypike3wPeFRrvQI5l8&e=
> > > > > > 
> > > > > > <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=5qj6BQUSwqYmkAVeKz5axFV8k3gxYEPSJ5Cp0RSnxrE&s=I7fR1GY5lN2hVMkDuvryrhDeRypike3wPeFRrvQI5l8&e=>
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > >      _______________________________________________
> > > > > >      netmod mailing list
> > > > > >      [e-mail]  <mailto:[e-mail]>
> > > > > >      https://www.ietf.org/mailman/listinfo/netmod
> > > > > >      <https://www.ietf.org/mailman/listinfo/netmod>
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > _______________________________________________
> > > > > > netmod mailing list
> > > > > > [e-mail]
> > > > > > https://www.ietf.org/mailman/listinfo/netmod
> > > > > _______________________________________________
> > > > > Netconf mailing list
> > > > > [e-mail]
> > > > > https://www.ietf.org/mailman/listinfo/netconf
> > > _______________________________________________
> > > netmod mailing list
> > > [e-mail]
> > > https://www.ietf.org/mailman/listinfo/netmod
> 

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: [phone]         Campus Ring 1 | 28759 Bremen | Germany
Fax:   [phone]         <http://www.jacobs-university.de/>