Re: [netmod] initial comments on draft-rwilton-netmod-yang-packages

[Robert Wilton <[e-mail]>]

Hi Andy,

Thanks for the comments.

On 30/01/2019 01:22, Andy Bierman wrote:
> Hi,
>
> I originally brought up this issue in July 2015
> https://datatracker.ietf.org/doc/draft-bierman-netmod-yang-package/

Yes.

The solution I propose is different in the sense that it uses YANG 
instance data to define YANG packages rather than new YANG keywords.   I 
believe that this should make it a lower cost solution to define and 
implement.


>
> I don't think the WG ever agreed on the problem that needs to be solved,
> and that is still the case.

That wasn't quite my impression.  I also think that folks were busy 
focusing on other WG activity and didn't necessarily have the time to 
concentrate on this.

My draft was aiming on solving two broad problems:

    The main goals of YANG package definitions include, but are not
    restricted to:

    o  To act as a simplified YANG conformance mechanism.  Rather than
       conformance being performed against a set of individual YANG
       module revisions, conformance could also be more simply stated in
       terms of YANG packages, with a set modifications (e.g. additional
       modules, deviations, or features).

    o  To allow YANG datastore schema to be specified in a more concise
       way rather than having to list all modules and revisions.  YANG
       package definitions can be defined in documents that can be
       referenced by a URL rather than requiring explicit lists of
       modules to be shared between client and server.  Hence, a YANG
       package must contain sufficient information to allow a client or
       server to precisely construct the schema associated with the
       package.



>
> In reality each server has 1 package -- its entire library.

This doesn't apply to all servers.  For a long time, as a vendor, we 
have had separate packages that can be independently installed, and 
which extend the management model to cover the new functionality.  E.g. 
BNG functionality could be in a separate, independently installable, 
package on top of the base router functionality.

For a Linux server, the manageability interface will depend on what 
applications have been installed.


> The SEMVER work shows
> that vendors are treating platforms as independent release trains, and 
> not really
> developing loadable packages.

This depends on the vendor.  The YANG versioning work is trying to find 
a solution that works across the industry.  I believe that the 
versioning requirements are different for standards developed modules, 
vs industry developed modules, vs vendor modules.


>
> I think YANG 1.2 improvements for conformance (e.g., YANG-redirects, 
> SEMVER import)
> and  the YANG Catalog can solve the module compatibility issues. It is 
> more of a documentation
> problem than a standards problem.

Having a standard YANG module that can be used to define packages is 
something this is useful and should be standardized.  I believe that 
this is better than each vendor coming up with their own solution for 
this problem.

Thanks,
Rob


>
>
> Andy
>
>
>
>
> On Tue, Jan 29, 2019 at 4:55 PM Sterne, Jason (Nokia - CA/Ottawa) 
> <[e-mail] <mailto:[e-mail]>> wrote:
>
>     Thanks Rob. Please see inline.
>
>     Jason
>
>     *From:*Robert Wilton <[e-mail] <mailto:[e-mail]>>
>     *Sent:* Thursday, January 24, 2019 1:16 PM
>     *To:* Sterne, Jason (Nokia - CA/Ottawa) <[e-mail]
>     <mailto:[e-mail]>>; [e-mail]
>     <mailto:[e-mail]>
>     *Subject:* Re: initial comments on draft-rwilton-netmod-yang-packages
>
>     Hi Jason,
>
>     Thanks for the review and comments.
>
>     I've put some responses inline ...
>
>     On 24/01/2019 14:56, Sterne, Jason (Nokia - CA/Ottawa) wrote:
>
>         Hi guys,
>
>         I've gotten most of the way through the draft and have some
>         initial comments. I haven't digested the section 10 open
>         issues yet or the examples.
>
>         Section 5 mentions the following:
>
>         YANG library is augmented to allow servers to report the packages
>
>         that they implement and to associate those packages back to
>
>         particular datastore schema.
>
>         Does the combination of this draft and rfc7895bis somehow
>         allow the same package to be advertised in 2 different
>         datastores, but with different deviations in each datastore?
>         I'm thinking of a case, for example, where a package is fully
>         supported in the running but the package minus a few modules
>         (or parts of modules) is supported in the operational
>         datastore. There seems to be a 1:1 relationship between
>         package and rfc7895bis schema.
>
>     So, the intention is no, not directly.
>
>     My aim here is that <running> would implement package "foo", and
>     <operational> would implement package "modified-foo".  Package
>     "modified-foo" would import package "foo" and also specify the set
>     of modules that contain the deviations "foo".
>
>     I didn't want a server to be able to see that I implement package
>     "foo", but then I have all these deviations that change its
>     behavior.  Instead, it is really implementing a different package
>     that is based on "foo".
>
>         The packages draft doesn't include any specific leaf-list for
>         deviations. Section 7.2 mentions that deviations could be
>         expressed by including modules that happen to contain
>         deviations. That seems a bit inconsistent with rfc7895bis that
>         has a specific leaf-list of deviations (and NETCONF hello that
>         specifically explicitly labels deviation modules).
>
>     I'm conflicted on this one.  I don't really like the deviation
>     list in YANG library because I regard it as a duplicate source of
>     information, and then there is a question of which source of data
>     do you trust.  E.g. do you process a deviation in a module that is
>     not listed in the deviations module list?
>
>     */[>>JTS: ] Good point. I suppose this issue applies today
>     already. i.e. what if one of the modules advertised in the <hello>
>     is a module of deviations (without having been referenced by
>     another module as a deviation module)./*
>
>         Section 5.1 says the package must be referentially complete. I
>         can see the advantages of that although wondering if that
>         might limit flexibility of partitioning modules into packages.
>         I could imagine use cases for dividing a large set of modules
>         into a few packages that might rev independently but can still
>         all work together (especially if they rev in a bc manner). But
>         maybe that just starts to introduce too much complexity?
>
>     Yes, having partial packages may be useful.  Perhaps just adding a
>     leaf to indicate whether a package is referentially complete could
>     be the answer here.
>
>         I didn't understand this part of section 5.1. Can you maybe
>         illustrate with an example?
>
>         The version/revision of a module listed in the package module list
>
>         supercedes any version/revision of the module listed in a imported
>
>         package module list.  This allows a package to resolve any
>
>         conflicting implemented module versions/revisions in imported
>
>         packages.
>
>     Probably best to see example B.3. in the appendix because it
>     exactly illustrates this point.
>
>     Basically:
>     1) Packages must explicitly list all versions of all modules they
>     define/import.
>     2) If two imported packages define different versions of modules,
>     then the package that is importing them needs a way to define
>     which version to use.
>     3) A package needs a way to override the version of module
>     specified in an imported package.
>
>     */[>>JTS: ] Thx. That example does help. I suppose the designer of
>     the package needs to carefully check that the version they select
>     can be successfully used by all the modules in the package. /*
>
>     */I think there is a minor typo in example B.3.  The example-3-pkg
>     is importing "/* */example-import-1" but I believe you meant "/*
>     */example-import-1-pkg" (and some for import-2)./*
>
>         It might be a good idea to add a parent-version to the package
>         module (to allow tracking lineage of packages).
>
>     Agreed, or maybe allowing a revision history like modules.  Not
>     sure which is better here.  Packages could get a lot of updates,
>     and a long revision history would not be helpful at all.
>
>     */[>>JTS: ] I think a minimum of just specifying the direct parent
>     is enough to build the full tree of lineage. We don't need a long
>     history of N revisions./*
>
>         I like the use of groupings. That allows a manager to use this
>         as a building block to compose a model that has a list of
>         packages.
>
>     OK.
>
>         Having a global list of mandatory features (vs having the
>         mandatory feature a per-module list) means inventing the new
>         <module-name>:<feature> format. Should we instead somehow put
>         the mandatory features against each module of the package?
>
>     Perhaps.  My thinking here was to have the list of features high
>     up and very easy to find/parse.
>
>         The location leaf is a uri but then the description says it
>         must be a url (where the model can be retrieved). I do like
>         that the namespace is separate from the location, but maybe we
>         should make location a url type?
>
>     Yes, I was thinking that is should be a URL.
>
>         Do we need a namespace for package names in the model?
>
>     I had them in an earlier version, but I took them out, because I
>     wasn't sure that they are really useful/required.
>
>     Defining a format to make package names themselves globally unique
>     might be sufficient.
>
>     */[>>JTS: ] I'm OK with that. It is similar to how we're finding
>     that it is useful that YANG module names are globally unique (i.e.
>     by naming with ietf-xxxx or companyabc-xxx)./*
>
>         In 7.3 we only reference module-sets and not modules. So the
>         grouping of modules into sets and packages must be the same?
>
>     Not necessarily.
>
>     I am trying to reuse the module-set definitions as much as
>     possible (to avoid duplication).  One issue here is that
>     module-sets are combined then all the modules must not overlap,
>     which doesn't make the mapping to module-sets quite so clean.
>
>         A schema can only have a single package. I think that works
>         but it means a server would advertise multiple schemas if it
>         wants to support multiple packages. I'm not sure if there are
>         some downsides to that (it just surprised me).
>
>     My aim here was:
>      - multiple packages are advertised in yang-library/packages
>      - datastores only report that they "implement" one [top level]
>     package version.  [The package itself might import other packages.]
>
>     If we do package selection, then for a given YANG client session,
>     and the version of YANG library available/reported by that
>     session, it would appear as if the server only implements one top
>     level package for a datastore.  Different clients choosing
>     different versions would see slightly different output depending
>     on which package version they had selected to use.
>
>     Thanks again for the review and the comments!
>
>     Rob
>
>         Jason
>
>         *From:*netmod <[e-mail]>
>         <mailto:[e-mail]> *On Behalf Of *Robert Wilton
>         *Sent:* Thursday, December 20, 2018 12:45 PM
>         *To:* [e-mail] <mailto:[e-mail]>
>         *Subject:* [netmod] YANG Packages
>
>         Hi,
>
>         I've written up an ID for a potential solution for YANG
>         packages using instance data:
>
>         Abstract
>
>            This document defines YANG packages, an organizational
>         structure
>
>            holding a set of related YANG modules, that can be used to
>         simplify
>
>            the conformance and sharing of YANG schema.  It describes
>         how YANG
>
>            instance data documents are used to define YANG packages,
>         and how the
>
>            YANG library information published by a server can be
>         augmented with
>
>            additional packaging related information.
>
>         https://datatracker.ietf.org/doc/draft-rwilton-netmod-yang-packages/
>
>         Potentially this work may be of use as part of the YANG
>         versioning design team work.  In addition, if the WG likes
>         this approach of defining YANG packages, then it might also be
>         useful to bind a schema to a YANG instance data document.
>
>         Some questions for members of the WG:
>
>         1) Do members of the WG agree that YANG packages is something
>         that needs to be solved?
>
>         2) Is the approach in this draft of defining these as instance
>         data documents a good starting point?
>
>         3) This approach augments YANG library-bis, reusing
>         module-sets, but not replacing the way that modules are
>         reported in YANG library-bis.  Is this the right approach? 
>         This approach tries to allow module-sets to be reused for both
>         schema and packages, but the YANG library-bis rules for
>         combining module-sets (i.e. no conflicts) may make this harder
>         to really reuse the module-sets for both purposes.
>
>         Of course, any other comments or feedback is welcome and
>         appreciated.
>
>         Thanks,
>         Rob
>
>     _______________________________________________
>     netmod mailing list
>     [e-mail] <mailto:[e-mail]>
>     https://www.ietf.org/mailman/listinfo/netmod
>