Re: [netmod] instance file parsing

[Juergen Schoenwaelder <[e-mail]>]

On Mon, Dec 03, 2018 at 09:56:48AM -0800, Andy Bierman wrote:
> On Mon, Dec 3, 2018 at 3:36 AM Juergen Schoenwaelder <
> [e-mail]> wrote:
> 
> > On Mon, Dec 03, 2018 at 10:21:20AM +0000, Robert Wilton wrote:
> > >
> > > On 30/11/2018 19:28, Juergen Schoenwaelder wrote:
> > > > I doubt it makes sense to carry an entire yang library schema with the
> > > > instance data. The modules are actually known via the namespaces.
> > >
> > > Knowing the module name or namespace isn't sufficient to be able to parse
> > > the data:
> > >  - E.g. if the contents are in the format of RFC7895bis, but the client
> > > tries to parse it using the schema from RFC7895, then it will fail.
> > >  - Once NBC changes are allowed via YANG versioning it will become more
> > > necessary to know the revision or version of the modules to be able to
> > parse
> > > the data.  E.g. not even just the latest module revision will necessarily
> > > work.   It is entirely plausible that different server revisions may be
> > > generating instance data using slightly different schema.
> >
> > Perhaps this is a problem of the versioning solution then. ;-)
> >
> > > If the server has deviations then the client may also need to know those
> > > deviations to correctly parse the file.
> > >
> > > So, I think that it is pretty clear that knowing the right schema is
> > > required to be able to correctly parse and interpret the instance data.
> > >
> > > I think that there are 4 ways that this can be achieved:
> > >  1) embedded the necessary schema information into the YANG instance
> > data.
> > >  2) put the necessary schema information online somewhere and have a URI
> > > reference to it.
> > >  3) some combination of (1) and (2), e.g. packages defined centrally,
> > with
> > > deviations listed in the file.
> > >  4) the schema is determined using some out of bound mechanism, or
> > possibly
> > > it is hard coded.
> >
> > Perhaps we need to define first what "parse" means. I am not sure
> > parsing requires to know all schema details. Anyway, if all schema
> > details are needed, then the simplest solution seems to be to read the
> > schema from another instance file. This then only requires to
> > bootstrap the schema model version.
> >
> >
> 
> I agree with Rob.
> The solution has to support accurate parsing of the instance data.
> This means that the parser has the correct schema tree to use for
> validating an instance document against the schema.
> 
> Since the new yang-library can have a different schema tree for each
> datastore,
> clearly the option of specifying the datastore is needed.

We need to get the terminology straight. I can parse XML and JSON
files without the need to have schema information available. So I
think the word 'parsing' is quite misleading here. And I can extract
data out of XML and JSON files that I find interesting without schema
information as well. So its a certain type of tools that may take
advantage of being schema aware but not all tools need to be schema
aware.

> > > I don't think that there is a one size fits all answer here.  I think
> > that
> > > that it depends on the use case.  Certainly, I think that facilitating 1
> > -3
> > > is useful, but they should be optional rather than mandatory.  I.e.
> > defining
> > > nodes for these doesn't seem to cost much if a server isn't obliged to
> > > populate them.
> > >
> > > I do think that YANG packages (themselves defined in instance data
> > > documents) could be very helpful here.  I.e. rather than listing all the
> > > modules, instead list the packages + any deviations from that.  I'm
> > > presuming here that the definitions of the packages are available via a
> > URI.
> >
> >
> 
> Yes -- this has been my goal for YANG Packages since the start.
> By using nested packages and offline caching, the entire YANG library for a
> device
> could be recognized in a few URIs.

An instance file storing the schema tree is an offline cache. It falls
out of the instance file format for free. Yes, packages are yet
something entirely different but so far no work in this direction is
chartered so we should get instance file formats defined without
solving another bigger problem first.


> > > >   If
> > > > you want to capture the schema, dump the relevant yang library into
> > > > another instance file.
> > >
> > > That just means another file to carry around and manage.
> >
> > Yes, but compared to solutions that require new and/or much more
> > elaborate data formats, this is very cheap and efficient, in
> > particular for systems where the schema itself is relatively
> > static. Also in terms of engineering time this is a rather cheap
> > solution since you do not need to invent a new way to document and
> > communicate a schema.
> >
> >
> I agree the cost of an extra instance document is not a problem, especially
> since
> it would be optional and only used if the defaults are not sufficient:
>    - latest revision of module will be OK
>    - assume all features present will be OK
>    - assume no deviations will be OK
> 
> If the defaults are not OK, then the parser will incorrectly accept or
> reject certain instance data,
> unless the correct schema tree is obtained somehow.

Or the tool processing instance file content just does not care about
the schema and it simply assumes that a certain (module,path)
combination has fixed semantics. I am big fan of being able to use
generic tools to filter and process XML or JSON encoded data.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: [phone]         Campus Ring 1 | 28759 Bremen | Germany
Fax:   [phone]         <https://www.jacobs-university.de/>