Re: [core] YANG Packages for CoMI

[Andy Bierman <[e-mail]>]

On Tue, Jul 28, 2015 at 12:04 AM, Alexander Pelov <[e-mail]> wrote:

>  Hi Andy,
>
> Le 27/07/2015 20:01, Andy Bierman a écrit :
>
>
>
> On Mon, Jul 27, 2015 at 3:25 AM, Alexander Pelov < <[e-mail]>
> [e-mail]> wrote:
>
>>  Hi Andy!
>>
>> Thanks for the draft! Seems a good start to get the things moving in the
>> direction of having a good, structured introspection mechanism for module
>> discovery!
>>
>> I'm not completely sure that it will be a static document. You still need
>> to think of the possibility of a firmware upgrade, in which case you have
>> to consider the clients which have obtained your previous package
>> descriptions.
>>
>>
>  Each package has a name and a revision-date.
> The expected use for CoMI would be a new
> package revision for each firmware release.
>
> I meant that there should be a mechanism for the client to be notified
> when the YANG package definition changes. This may be a rare event, but
> imagine a new client arrives and discovers the server 1 minute before the
> server gets a new firmware release (or some other event, making it change
> its package definition).
>


I agree.
There can be a YANG Package Library URI

  /mg/pkg.uri

URI of the YANG Package Library resource to GET or Observe

The package data tree could be much smaller than the module library tree.

  container packages {
     leaf pkg-set-id { type uint32; }
     list package {
        key name;
        leaf name { type string; }
        leaf rev { type yang:revision-date; }
     }
   }


  { "packages": {
     "pkg-set-id":42,
     "package": [
       {
          "name":"example-sensor314",
          "rev":"2015-06-14"
       }
     ]
   }
 }



> I would think that if a client is expected to keep long-term association
> to the server, it should use Observe or NOTIFY. The latter will work if the
> package is defined as module.
>
>
>
>
>>  Why don't you want it to be a YANG module? I would suppose that for a
>> structured numbering scheme (such as CoOL) we could have a nice fixed entry
>> point for such introspection resource, e.g. moduleID = 1. Alternatively, I
>> would think that it should be discoverable through .well-known/core
>>
>>
>  Not sure what you mean.
> This is static data used to identify a set of YANG modules.
> Do you mean why not have the contents of a package be YANG objects
> to read from the server?  This would be even more data than the
> current YANG module library.
>
>  The point of a package list is to avoid listing all the modules,
> features,
> and revision dates.  There is a YANG Package Identifier in sec. 4
>
>    urn:ietf:params:xml:ns:yang:pkg?name=example-sensor314&rev=2015-07-01
>
>
>  This would be the entire response needed to discover the entire YANG
> implementation by the server. The size remains the same no matter how many
> modules and features are added over time.
>
>
> Ah, OK! Now I understand! It was not obvious to me that you were aiming at
> having the package definition be detached from the server. I like that!
> This, however makes me think about something else - you should have a way
> to obtain the package from the sensor. However, this goes a little bit in
> the sense of having YANG packages being mandatory for CoMI. I like the
> "uses-package" statement, as this could be used to have full definition of
> the modules on a server.
>
> I would suppose that you could represent the information of a YANG package
> as a YANG module. Then, a client could query the server with a
> select/fields statement to obtain only the package identifier. IF the
> client wants, it could obtain the whole package content from the server:
>
> E.g. for a YANG module definition such as:
> module yang-package {
>   ...
>   leaf yang-package-uri {
>     type string;
>   }
>
>   container package {
>     leaf organization {type string;}
>     leaf contact {type string;}
>     leaf description {type string;}
>     leaf revision {type string;}
>     container imports-module {
>        key module-name;
>        leaf module-name {type string;}
>        leaf module-cool-id {type int32;}
>     }
>     container uses-module {
>        ...
>     }
>     ...
>   }
> }
>
>

The point of the YANG package is that it serves as an offline handle
for lots of YANG details related to a set of modules.

NETCONF and RESTCONF just advertise the conformance details
for each module (name, revision, features, deviations). There is
a <get-schema> operation in both protocols to allow the client to retrieve
the actual YANG files.  There is no YANG module that modules
YANG module contents.

The same approach <get-pkg-schema> should be used in CoMI,
except I would expect a centralized server to have that data,
not any constrained servers.




module ID for this module would be fixed in CoOL to 1, so yang-package-uri
> will have CoOL ID 2049.
>
> In CoOL this would be as getting :
> GET /cdat?fields(2049)
>
> Response:
> {
>
> 2049:"urn:ietf:params:xml:ns:yang:pkg?name=example-sensor314&rev=2015-07-01"
> }
>
>
Maybe this could be even more optimized.
>


I don't know if package numbers are really needed.
The entire package library above would fit in 128 bytes.



Andy



>
> Now, if the client has no idea how to find this URN, it would simply  GET
> /cdat?fields(2048) and retrieve the YANG package information from the
> server.
>
>
>
>
>
>
>>  A point to make is that we can have alias mappings through this
>> introspection mechanism. Defining such an introspection mechanism for the
>> aliases was actually one of the work I was considering for CoOL.
>>
>>
>  RESTCONF and NETCONF provide the ability for the client to retrieve
> the schema, and get all the meta-data, for every module.
> IMO this should be done by a centralized server in CoMI.
>
>  But I agree the package data could include alias mappings.
> This might be a way to convert YANG Hash to COOL for example,
> to optimize frequently used identifiers..
>
>
> Maybe this should also be the way to handle alias attribution, e.g. add a
> "alias" statement, which indicates the mapping alias-id -> CoOL id / CoMI
> hash / full identifier.
>
> module yang-package {
> ....
>   container alias {
>     key cool-id;
>     leaf cool-id {type int32};
>     leaf alias-id {type int32};
>   }
> }
>
> So you can GET /cdat?fields(2058) and have
> {
>   2058: [
>     {11: 29834394, 12:1},
>     {11: 99999999, 12:2}
>     {11: 88888384, 12:3}
>   ]
> }
>
> assuming the following CoOL id automatic attribution:
> alias -> node ID=10
> cool-id -> node Id=11
> alias-id -> node ID=12
> and some random IDs to be mapped (29834394, 99999999, 88888384).
>
> Ideally, of course, you would be fetching this information from a server
> somewhere on the Internet, so that you don't fetch from your constrained
> server.
>
> Best,
> Alexander
>
>
>
>
>
>
>>  Best,
>> Alexander
>>
>
>  Andy
>
>
>>
>> Le 27/07/2015 08:29, Andy Bierman a écrit :
>>
>>
>>
>> On Sun, Jul 26, 2015 at 11:26 PM, peter van der Stok <
>> <[e-mail]>[e-mail]> wrote:
>>
>>> HI Andy,
>>>
>>> Is the proposal to make YANG packages compulsory over yang-library in
>>> Comi servers and clients?
>>>
>>>
>>  No, it would be optional.
>> Another solution approach would be to make the module-set-id values
>> global instead of per-server. That might be less work perhaps.
>>
>>
>>  Peter
>>>
>>
>>  Andy
>>
>>
>>>
>>> Andy Bierman schreef op 2015-07-26 19:44:
>>>
>>>> Hi,
>>>>
>>>> Michel raised some issues with the YANG library draft
>>>> http://datatracker.ietf.org/doc/draft-ietf-netconf-yang-library/
>>>>
>>>> Maybe CoMI needs its own optimized module library.
>>>> I am concerned that data sizes of the YANG library will be
>>>> too big in some really constrained environments.
>>>>
>>>> I wrote a draft that defines a way to specify the contents of the
>>>> YANG library in a single static document, called a YANG package.
>>>> http://datatracker.ietf.org/doc/draft-bierman-netmod-yang-package/
>>>>
>>>> If there was a "shorthand" resource that listed YANG packages, instead
>>>> of YANG modules, then the CoMI client that supported YANG packages
>>>> could read that instead of the module library.
>>>>
>>>> The YANG package message response can identify a single package
>>>> (e.g. match the firmware) so the size of the response will remain very
>>>> small, and not depend on the entire list of modules, features,
>>>> and deviations. The data savings could 1000X for 100 modules.
>>>>
>>>> The client needs to retrieve the library first-time and anytime it
>>>> changes.
>>>> The module-set-id values are per-server, not global. This could be a
>>>> significant bottleneck when a CoMI client starts or restarts, and
>>>> manages lots of servers.
>>>>
>>>> Andy
>>>>
>>>>
>>>> _______________________________________________
>>>> core mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/core
>>>>
>>>
>>
>>
>> _______________________________________________
>> core mailing [e-mail]://www.ietf.org/mailman/listinfo/core
>>
>>
>>
>
>