Re: [netmod] initial comments on draft-rwilton-netmod-yang-packages

On Wed, Jan 30, 2019 at 4:19 AM Robert Wilton <rwilton@cisco.com> wrote:

> Hi Andy,
>
> Thanks for the comments.
> On 30/01/2019 01:22, Andy Bierman wrote:
>
> Hi,
>
> I originally brought up this issue in July 2015
> https://datatracker.ietf.org/doc/draft-bierman-netmod-yang-package/
>
> Yes.
>
> The solution I propose is different in the sense that it uses YANG
> instance data to define YANG packages rather than new YANG keywords.   I
> believe that this should make it a lower cost solution to define and
> implement.
>
>
>
I think yangvalidator.org has a better solution that does not change YANG
conformance.

Andy

>
> I don't think the WG ever agreed on the problem that needs to be solved,
> and that is still the case.
>
> That wasn't quite my impression.  I also think that folks were busy
> focusing on other WG activity and didn't necessarily have the time to
> concentrate on this.
>
> My draft was aiming on solving two broad problems:
>
>    The main goals of YANG package definitions include, but are not
>    restricted to:
>
>    o  To act as a simplified YANG conformance mechanism.  Rather than
>       conformance being performed against a set of individual YANG
>       module revisions, conformance could also be more simply stated in
>       terms of YANG packages, with a set modifications (e.g. additional
>       modules, deviations, or features).
>
>    o  To allow YANG datastore schema to be specified in a more concise
>       way rather than having to list all modules and revisions.  YANG
>       package definitions can be defined in documents that can be
>       referenced by a URL rather than requiring explicit lists of
>       modules to be shared between client and server.  Hence, a YANG
>       package must contain sufficient information to allow a client or
>       server to precisely construct the schema associated with the
>       package.
>
>
>
>
>
> In reality each server has 1 package -- its entire library.
>
> This doesn't apply to all servers.  For a long time, as a vendor, we have
> had separate packages that can be independently installed, and which extend
> the management model to cover the new functionality.  E.g. BNG
> functionality could be in a separate, independently installable, package on
> top of the base router functionality.
>
> For a Linux server, the manageability interface will depend on what
> applications have been installed.
>
>
> The SEMVER work shows
> that vendors are treating platforms as independent release trains, and not
> really
> developing loadable packages.
>
> This depends on the vendor.  The YANG versioning work is trying to find a
> solution that works across the industry.  I believe that the versioning
> requirements are different for standards developed modules, vs industry
> developed modules, vs vendor modules.
>
>
>
> I think YANG 1.2 improvements for conformance (e.g., YANG-redirects,
> SEMVER import)
> and  the YANG Catalog can solve the module compatibility issues. It is
> more of a documentation
> problem than a standards problem.
>
> Having a standard YANG module that can be used to define packages is
> something this is useful and should be standardized.  I believe that this
> is better than each vendor coming up with their own solution for this
> problem.
>
> Thanks,
> Rob
>
>
>
>
> Andy
>
>
>
>
> On Tue, Jan 29, 2019 at 4:55 PM Sterne, Jason (Nokia - CA/Ottawa) <
> jason.sterne@nokia.com> wrote:
>
>> Thanks Rob. Please see inline.
>>
>> Jason
>>
>>
>>
>> *From:* Robert Wilton <rwilton@cisco.com>
>> *Sent:* Thursday, January 24, 2019 1:16 PM
>> *To:* Sterne, Jason (Nokia - CA/Ottawa) <jason.sterne@nokia.com>;
>> netmod@ietf.org
>> *Subject:* Re: initial comments on draft-rwilton-netmod-yang-packages
>>
>>
>>
>> Hi Jason,
>>
>> Thanks for the review and comments.
>>
>> I've put some responses inline ...
>>
>> On 24/01/2019 14:56, Sterne, Jason (Nokia - CA/Ottawa) wrote:
>>
>> Hi guys,
>>
>>
>>
>> I've gotten most of the way through the draft and have some initial
>> comments. I haven't digested the section 10 open issues yet or the examples.
>>
>>
>>
>> Section 5 mentions the following:
>>
>>    YANG library is augmented to allow servers to report the packages
>>
>>    that they implement and to associate those packages back to
>>
>>    particular datastore schema.
>>
>>
>>
>> Does the combination of this draft and rfc7895bis somehow allow the same
>> package to be advertised in 2 different datastores, but with different
>> deviations in each datastore? I'm thinking of a case, for example, where a
>> package is fully supported in the running but the package minus a few
>> modules (or parts of modules) is supported in the operational datastore.
>> There seems to be a 1:1 relationship between package and rfc7895bis schema.
>>
>> So, the intention is no, not directly.
>>
>> My aim here is that <running> would implement package "foo", and
>> <operational> would implement package "modified-foo".  Package
>> "modified-foo" would import package "foo" and also specify the set of
>> modules that contain the deviations "foo".
>>
>> I didn't want a server to be able to see that I implement package "foo",
>> but then I have all these deviations that change its behavior.  Instead, it
>> is really implementing a different package that is based on "foo".
>>
>>
>>
>>
>>
>> The packages draft doesn't include any specific leaf-list for deviations.
>> Section 7.2 mentions that deviations could be expressed by including
>> modules that happen to contain deviations. That seems a bit inconsistent
>> with rfc7895bis that has a specific leaf-list of deviations (and NETCONF
>> hello that specifically explicitly labels deviation modules).
>>
>> I'm conflicted on this one.  I don't really like the deviation list in
>> YANG library because I regard it as a duplicate source of information, and
>> then there is a question of which source of data do you trust.  E.g. do you
>> process a deviation in a module that is not listed in the deviations module
>> list?
>>
>> *[>>JTS: ] Good point. I suppose this issue applies today already. i.e.
>> what if one of the modules advertised in the <hello> is a module of
>> deviations (without having been referenced by another module as a deviation
>> module).*
>>
>>
>>
>>
>>
>> Section 5.1 says the package must be referentially complete. I can see
>> the advantages of that although wondering if that might limit flexibility
>> of partitioning modules into packages. I could imagine use cases for
>> dividing a large set of modules into a few packages that might rev
>> independently but can still all work together (especially if they rev in a
>> bc manner). But maybe that just starts to introduce too much complexity?
>>
>> Yes, having partial packages may be useful.  Perhaps just adding a leaf
>> to indicate whether a package is referentially complete could be the answer
>> here.
>>
>>
>>
>>
>>
>> I didn't understand this part of section 5.1. Can you maybe illustrate
>> with an example?
>>
>>       The version/revision of a module listed in the package module list
>>
>>       supercedes any version/revision of the module listed in a imported
>>
>>       package module list.  This allows a package to resolve any
>>
>>       conflicting implemented module versions/revisions in imported
>>
>>       packages.
>>
>> Probably best to see example B.3. in the appendix because it exactly
>> illustrates this point.
>>
>> Basically:
>> 1) Packages must explicitly list all versions of all modules they
>> define/import.
>> 2) If two imported packages define different versions of modules, then
>> the package that is importing them needs a way to define which version to
>> use.
>> 3) A package needs a way to override the version of module specified in
>> an imported package.
>>
>> *[>>JTS: ] Thx. That example does help. I suppose the designer of the
>> package needs to carefully check that the version they select can be
>> successfully used by all the modules in the package. *
>>
>> *I think there is a minor typo in example B.3.  The example-3-pkg is
>> importing "* *example-import-1" but I believe you meant "* *example-import-1-pkg"
>> (and some for import-2).*
>>
>>
>>
>> It might be a good idea to add a parent-version to the package module (to
>> allow tracking lineage of packages).
>>
>> Agreed, or maybe allowing a revision history like modules.  Not sure
>> which is better here.  Packages could get a lot of updates, and a long
>> revision history would not be helpful at all.
>>
>> *[>>JTS: ] I think a minimum of just specifying the direct parent is
>> enough to build the full tree of lineage. We don't need a long history of N
>> revisions.*
>>
>>
>>
>>
>>
>> I like the use of groupings. That allows a manager to use this as a
>> building block to compose a model that has a list of packages.
>>
>> OK.
>>
>>
>>
>>
>>
>> Having a global list of mandatory features (vs having the mandatory
>> feature a per-module list) means inventing the new <module-name>:<feature>
>> format. Should we instead somehow put the mandatory features against each
>> module of the package?
>>
>> Perhaps.  My thinking here was to have the list of features high up and
>> very easy to find/parse.
>>
>>
>>
>>
>>
>> The location leaf is a uri but then the description says it must be a url
>> (where the model can be retrieved). I do like that the namespace is
>> separate from the location, but maybe we should make location a url type?
>>
>> Yes, I was thinking that is should be a URL.
>>
>>
>>
>>
>>
>> Do we need a namespace for package names in the model?
>>
>> I had them in an earlier version, but I took them out, because I wasn't
>> sure that they are really useful/required.
>>
>> Defining a format to make package names themselves globally unique might
>> be sufficient.
>>
>> *[>>JTS: ] I'm OK with that. It is similar to how we're finding that it
>> is useful that YANG module names are globally unique (i.e. by naming with
>> ietf-xxxx or companyabc-xxx).*
>>
>>
>>
>>
>>
>> In 7.3 we only reference module-sets and not modules. So the grouping of
>> modules into sets and packages must be the same?
>>
>> Not necessarily.
>>
>> I am trying to reuse the module-set definitions as much as possible (to
>> avoid duplication).  One issue here is that module-sets are combined then
>> all the modules must not overlap, which doesn't make the mapping to
>> module-sets quite so clean.
>>
>>
>>
>>
>>
>> A schema can only have a single package. I think that works but it means
>> a server would advertise multiple schemas if it wants to support multiple
>> packages. I'm not sure if there are some downsides to that (it just
>> surprised me).
>>
>> My aim here was:
>>  - multiple packages are advertised in yang-library/packages
>>  - datastores only report that they "implement" one [top level] package
>> version.  [The package itself might import other packages.]
>>
>> If we do package selection, then for a given YANG client session, and the
>> version of YANG library available/reported by that session, it would appear
>> as if the server only implements one top level package for a datastore.
>> Different clients choosing different versions would see slightly different
>> output depending on which package version they had selected to use.
>>
>> Thanks again for the review and the comments!
>>
>> Rob
>>
>>
>>
>>
>>
>>
>>
>> Jason
>>
>>
>>
>>
>>
>>
>>
>> *From:* netmod <netmod-bounces@ietf.org> <netmod-bounces@ietf.org> *On
>> Behalf Of *Robert Wilton
>> *Sent:* Thursday, December 20, 2018 12:45 PM
>> *To:* netmod@ietf.org
>> *Subject:* [netmod] YANG Packages
>>
>>
>>
>> Hi,
>>
>> I've written up an ID for a potential solution for YANG packages using
>> instance data:
>>
>> Abstract
>>
>>
>>
>>    This document defines YANG packages, an organizational structure
>>
>>    holding a set of related YANG modules, that can be used to simplify
>>
>>    the conformance and sharing of YANG schema.  It describes how YANG
>>
>>    instance data documents are used to define YANG packages, and how the
>>
>>    YANG library information published by a server can be augmented with
>>
>>    additional packaging related information.
>>
>> https://datatracker.ietf.org/doc/draft-rwilton-netmod-yang-packages/
>>
>> Potentially this work may be of use as part of the YANG versioning design
>> team work.  In addition, if the WG likes this approach of defining YANG
>> packages, then it might also be useful to bind a schema to a YANG instance
>> data document.
>>
>> Some questions for members of the WG:
>>
>> 1) Do members of the WG agree that YANG packages is something that needs
>> to be solved?
>>
>> 2) Is the approach in this draft of defining these as instance data
>> documents a good starting point?
>>
>> 3) This approach augments YANG library-bis, reusing module-sets, but not
>> replacing the way that modules are reported in YANG library-bis.  Is this
>> the right approach?  This approach tries to allow module-sets to be reused
>> for both schema and packages, but the YANG library-bis rules for combining
>> module-sets (i.e. no conflicts) may make this harder to really reuse the
>> module-sets for both purposes.
>>
>> Of course, any other comments or feedback is welcome and appreciated.
>>
>> Thanks,
>> Rob
>>
>> _______________________________________________
>> netmod mailing list
>> netmod@ietf.org
>> https://www.ietf.org/mailman/listinfo/netmod
>>
>