Re: [netmod] [Netconf] Alternative YANG library structure for 7895bis

[Vladimir Vassilev <[e-mail]>]

On 12/11/2017 05:15 PM, Juergen Schoenwaelder wrote:
> I do not fully understand why you think it is worth to preserve the
> old YANG module library structure. Clearly, it won't be backwards
> compatible since an NMDA implementation may list modules that are not
> implemented in all datastores and an old client talking to a new
> server will thus misunderstand what is exposed via the old YANG module
> library structure. Your proposal "looks" backwards compatible but it
> is not if one takes a closer look.
>
> One can start making changes to say the conformance-type but then this
> does not solve the problem since an old client does not know what a
> new conformance type value means. The design team did look into the
> option of keeping the old structure unchanged some weeks ago and we
> finally arrived at the conclusion that it gets (i) ugly and (ii) does
> not really provide backwards compatibility for a client written agains
> the old YANG module library structure.
>
> Note that with the new proposals on the table, it should be possible
> to provide a backwards compatible view on YANG library for systems
> that implement the exact same module set (with the exact same set of
> features and deviations) on all datastores they support. But for
> systems where this is not true, it seems better to use new
> definitions instead of tweaking semantics.
>
> Perhaps it helps if you can clearly phrase the objective of keeping
> the old structure. What is the goal you want to achieve with that?
I prefer to call it the current structure for now. I will try to 
explain. My goal is to efficiently build transactional systems based on 
YANG models and open standards.

For this I need some reusable bricks. Such a brick is the rfc7895 
ietf-yang-libraray module. Even in a system with multiple YANG model 
contexts eventually processing breaks down to a single model context 
e.g. for example a YANG data validation task performed in a command line 
application comes down to:
  $ yang-data-validator yang-library-data.xml data.xml

The command can be called multiple times for different data and 
different contexts but it is the same very well tested simple tool. So 
my point is why can we not use one of the 2 newly introduced methods for 
instantiating this very well designed model for the needs of NMDA.

1. Use datastore for instantiation.
2. Use schema mount to mount rfc 7895 ietf-yang-library in the 
ietf-datastores list. (an alternative I assume is especially applicable 
to modules like ietf-yang-library that are self-contained)

In general having a solution that is flexible can easily be trimmed 
down. Adding special cases that are less flexible e.g. the 
(not-implemented-in leaf-list or none at all for the case of fully 
transactional systems with a single model context (no extra work for 
these)). The advantage is that even with the not-implemented-in 
leaf-list solution or other diff based the data processing tasks will 
eventually convert the differential information to a data structure 
implementing the rfc7895 ietf-yang-library data tree and the 
yang-data-validator application will be backward compatible and 
reusable. With changes that introduce a whole new level of abstraction 
in place of the simple list with module name and revision as keys one 
can't even reuse the groupings defined in that module and that is a 
problem I try to avoid.

Vladimir


> /js
>
> On Mon, Dec 11, 2017 at 04:53:30PM +0100, Vladimir Vassilev wrote:
>> On 12/11/2017 12:16 PM, Robert Wilton wrote:
>>
>>> Hi Vladimir,
>>>
>>>
>>> On 09/12/2017 11:49, Vladimir Vassilev wrote:
>>>> On 12/08/2017 07:01 PM, Andy Bierman wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> A library per datastore sounds too complicated.
>>>> I am not proposing that.
>>> I'm slightly lost, because I thought that was exactly what you were
>>> proposing! ;-)
>> I propose a solution that keeps ietf-yang-library a data model of a single
>> YANG context specification doing the datastore specific modeling in
>> ietf-datastores model instead. The solution can be both flexible
>> (independent yang-library data instances per datastore as my example was
>> focused on) or constrained ('not-implemented-in' modules leaf-list). Here is
>> everything that is needed:
>>
>> module: ietf-datastores
>>      +--ro datastores-state
>>         +--ro datastore* [name]
>>         +--ro (model)?
>>            +--:(same-as-operational)
>>            +--:(constrained-to-operational)
>>            |  +--ro not-implemented-in*       ->
>> /yanglib:module-state/module/name
>>            +--:(unconstrained)
>>               +--ro yang-library-datastore?   identityref
>>
>> YANG:
>> ...
>> container datastores-state {
>>      config false;
>>      list datastore {
>>        key name;
>>      }
>>      choice model {
>>          case same-as-operational {
>>          }
>>          case constrained-to-operational {
>>            leaf-list not-implemented-in {
>>              type leafref {
>>                path "/yanglib:module-state/yanglib:module/yanglib:name";
>>              }
>>            }
>>          }
>>          case unconstrained {
>>            leaf yang-library-datastore {
>>              type identityref {
>>                base ds:datastore;
>>            }
>>          }
>>        }
>>      }
>>    }
>> ...
>>
>> IMO ietf-yang-library as defined in rfc7895 is modular and reusable. I do
>> not see why this has to be compromised.
>>>> The fundamental point proposed is that the datastore relevant bits
>>>> are kept in the ietf-datastores module instead of merging everything
>>>> in a new ietf-yang-library entangled monster module. If needed
>>>> ietf-datastores can augment ietf-yang-library but ietf-yang-library
>>>> should be usable on its own without ietf-datastores. The solution is
>>>> coherent and modular and addresses the problem statement.
>>> The issue with this is that datastore augmentations to YANG library
>>> would end up changing the meaning of the existing YANG library nodes.
>>> E.g. an old client that ignores the datastore augmentations is going to
>>> get a nasty surprise when the server does not behave how it expects.
>>> E.g. because the configuration node that it thinks should be there isn't
>>> there because it only supported in <operational>.
>>>
>>> This was one of the reasons for changing YANG library.
>> A well written client that finds out unsupported newer versions of
>> ietf-yang-library (which is reported in the capabilities) or any of the NMDA
>> modules is deployed should not do any damage. How exactly did you solve the
>> problem for the bad clients by changing the structure of yang-library in a
>> incompatible way is something I do not understand.
>>> In terms of the idea of just re-using YANG library but export a separate
>>> copy for each datastore I think that this has its own problems:
>>> - I don't like the idea of returning meta-data along with configuration
>>> for a <get-data> on any of the configuration datastores.
>> There is no meta data. The datastores contain only the config false;
>> /modules-state tree. I think I explained that very clearly.
>>> - How does a client know whether the YANG library for <running> applies
>>> to the whole server (as it does today) vs just applies to the <running>
>>> datastore (as it would for an NMDA server)?
>> All datastore models are "case same-as-operational" for such systems.
>>> - It requires more handshaking between the client and server to get the
>>> schema, since a separate request would be required for each datastore
>>> that is supported.
>> Correct. Flexibility and modularity come at a price. IMO modularity is more
>> important in this case. And there are solutions for the problem e.g. send
>> all <get-data> requests before waiting for replies. NETCONF supports this.
>>> So, for me, I think that the only way that this solution works, would be
>>> to define a new <get-server-metadata> RPC, but even then I think that it
>>> would make sense to combine the data together into a new YANG library
>>> structure.
>> As I said my focus is on keeping ietf-yang-library modular (single YANG
>> model context). <get-server-metadata> or just adding datastore identities
>> allowing the client to use <get-data> to achieve the retrieval of the data
>> is of little importance to me.
>>
>>> At the end of the day, I don't think that a new YANG library is going to
>>> be were the real cost for supporting NMDA comes from.  I think that the
>>> real work is supporting <operational> independently from <running> both
>>> in the client and servers. But I also think that once servers start
>>> implementing this properly that it will simplify automation, because
>>> rather than a client having to guess what state a server is in, it can
>>> actually querey, or be notified of it, without having to write a lot of
>>> model specific code.
>> +1
>>
>> Vladimir
>>> Thanks,
>>> Rob
>>>
>>>
>>>>> I prefer the proposal that was made at the IETF meeting that had
>>>>> a 'not-implemented-in' leaf-list and a single module list.
>>>> This constraint is already specified in the text of the revised
>>>> datastores draft. Clients conforming to the draft can expect servers
>>>> to comply with the MUST requirement even if there is a separate
>>>> yang-library data tree for each datastore the constraint of
>>>> configuration stores mapping to 'operational' should be enforced
>>>> according to the draft. There is no contradiction here.
>>>>
>>>> That said I would be also be OK with ietf-datastores augmenting
>>>> ietf-yang-library with such a leaf-list ('not-implemented-in'
>>>> leaf-list) as a more constrained flavor of the same approach instead
>>>> of going for independent copies of yang-library data. For any of
>>>> that to happen change in ietf-datastores.yang is needed and change
>>>> in the original rfc7895 ietf-yang-library is not needed at all.
>>>>
>>>> Vladimir
>>>>
>>>>> Why is it interesting to have a separate module list for regular
>>>>> modules and imported modules?
>>>>> I prefer to keep the conformance leaf and not change the module list.
>>>>>
>>>>> NMDA needs to be possible to implement with a single schema tree
>>>>> such that a module
>>>>> is implemented in all datastores, or a subset of all
>>>>> datastores.  Otherwise it probably won't
>>>>> get supported in clients.
>>>>>
>>>>>
>>>>> Andy
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Dec 8, 2017 at 9:21 AM, Kent Watsen <[e-mail]
>>>>> <mailto:[e-mail]>> wrote:
>>>>>
>>>>>      CC-ing NETCONF, where the draft is being worked on.
>>>>>
>>>>>      Kent
>>>>>
>>>>>
>>>>>      On Fri, 2017-12-08 at 16:34 +0100, Juergen Schoenwaelder wrote:
>>>>>      > On Fri, Dec 08, 2017 at 04:19:28PM +0100, Vladimir
>>>>> Vassilev wrote:
>>>>>      > >
>>>>>      > > Yes. The default value for yang-library-datastore leaf is
>>>>>      ds:operational
>>>>>      > > (the only possible one for the ds:operational datastore). This
>>>>>      is backward
>>>>>      > > compatible. If one needs different model for 'running', etc.
>>>>>      then a new
>>>>>      > > datastore identity has to be defined  and set in place of the
>>>>>      default value.
>>>>>      > > Then this identity can be used to read the yang-library
>>>>> data with
>>>>>      > > <get-data>.
>>>>>      > >
>>>>>      >
>>>>>      > Sorry, but I have to ask this: How do I obtain the schema for the
>>>>>      > datastore (lets call it <running-library>) that reports the
>>>>>      schema for
>>>>>      > <running>? Is there another <running-library-library> datastore?
>>>>>      Will
>>>>>      > the recursion end? Perhaps it does since
>>>>> <running-library-library>
>>>>>      > might have itself listed as the schema defining datastore.
>>>>> I guess
>>>>>      > Lada will like these kind of meta and meta-meta datastores.
>>>>>
>>>>>      Not really. Metadata needn't be in datastores.
>>>>>
>>>>>      Lada
>>>>>
>>>>>      >
>>>>>      > /js
>>>>>      >
>>>>>      --
>>>>>      Ladislav Lhotka
>>>>>      Head, CZ.NIC Labs
>>>>>      PGP Key ID: 0xB8F92B08A9F76C67
>>>>>
>>>>>      _______________________________________________
>>>>>      netmod mailing list
>>>>>      [e-mail]  <mailto:[e-mail]>
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=5qj6BQUSwqYmkAVeKz5axFV8k3gxYEPSJ5Cp0RSnxrE&s=I7fR1GY5lN2hVMkDuvryrhDeRypike3wPeFRrvQI5l8&e=
>>>>>
>>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=5qj6BQUSwqYmkAVeKz5axFV8k3gxYEPSJ5Cp0RSnxrE&s=I7fR1GY5lN2hVMkDuvryrhDeRypike3wPeFRrvQI5l8&e=>
>>>>>
>>>>>
>>>>>
>>>>>      _______________________________________________
>>>>>      netmod mailing list
>>>>>      [e-mail]  <mailto:[e-mail]>
>>>>>      https://www.ietf.org/mailman/listinfo/netmod
>>>>>      <https://www.ietf.org/mailman/listinfo/netmod>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> netmod mailing list
>>>>> [e-mail]
>>>>> https://www.ietf.org/mailman/listinfo/netmod
>>>> _______________________________________________
>>>> Netconf mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/netconf
>> _______________________________________________
>> netmod mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/netmod