Re: [netmod] instance file parsing

Andy Bierman <andy@yumaworks.com> Mon, 03 December 2018 17:57 UTC

Return-Path: <andy@yumaworks.com>
X-Original-To: netmod@ietfa.amsl.com
Delivered-To: netmod@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4D9B91271FF for <netmod@ietfa.amsl.com>; Mon, 3 Dec 2018 09:57:05 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.359
X-Spam-Level:
X-Spam-Status: No, score=-3.359 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-1.459, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=yumaworks-com.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 06TIJ6fCnMZv for <netmod@ietfa.amsl.com>; Mon, 3 Dec 2018 09:57:02 -0800 (PST)
Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com [IPv6:2a00:1450:4864:20::12d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3CAEB124BE5 for <netmod@ietf.org>; Mon, 3 Dec 2018 09:57:02 -0800 (PST)
Received: by mail-lf1-x12d.google.com with SMTP id e26so9867344lfc.2 for <netmod@ietf.org>; Mon, 03 Dec 2018 09:57:02 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yumaworks-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=P8kh8MTedmZdX/RWT9EiXGq0WuRF8IJlPSnM9E1eQqA=; b=IW+Q0oP7v1FcFS1RnjTBQ5sy7BK7Du44iaulDgtv31jXMhzu6F/eeq3bLSZ+fhBsGK 8mFrr+UvqJNMy4X2cLIzs9cnkObSFyWmvDlCWdl00BcAwNaAMuaB53rB9f4pH2GgMks5 Lg85t9rn9iSc4iVFcmsztMQ9lhsb1nISIDbMGcT/afLVZM/z/hXb9OYosa4Qqnn9t3Uf /BsmSa8pyBNYdVUki/fHBpbIIeNrlP+Tsebbwh1MXxThqhzdUio/5WUFoLFvs7Rj+FOs dE8NrBNq3/6ZCVI4Uem/U6Ib+lFDERvucA0XB3Npd1Ip16ajU0I52VQXBkA1Jou1PyWG jXVA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=P8kh8MTedmZdX/RWT9EiXGq0WuRF8IJlPSnM9E1eQqA=; b=AXdg8iwYz8ZQij3d29GANmLa9/yy51Tk+8MyWCdH3nFdTj3DVpLBxifEaWh+eQP5ms h3LDuOE01eiHAqS0XsCWkCkDerr1GA7n7tRkQewq9pD8GyXO2oLgFbzb+tW0iZVODTkL hoLHbRNt2V65/M5OyVjCdoJs9yqjIZzDXNf04d19qbmCeGmsk/Xl1ya3za10rJjglxqp 8uzcul0+y9V0yRTUSRt7XO9R8VlYaq2x7EHA4wd7WyoZme3a57REzzUpl01Mjm7kP0r3 HbGC03a+uAIEyfupxDzdq+SyUvUuku07phuIHbcrrN7Rs1fFyoX5qFY4w2MlesSKwRpM 1hkA==
X-Gm-Message-State: AA+aEWZpsgfhJWK+muisgLQL6wBg0v7GKt29e2d40TKgafE6BP1oCnqt O0jaHv0AMVs5Kk5JDTOQqIj6IjseGNc8ivYUCuhm2A==
X-Google-Smtp-Source: AFSGD/Vcc+45vAOZNIY3IRkShRzfUUNszhHUqRj1TRMxmhi2nnlWM+1qIQpNC/crtlZ3ayya8NIi7GlIc8huBYNUc3w=
X-Received: by 2002:a19:d58e:: with SMTP id m136mr10470335lfg.70.1543859820025; Mon, 03 Dec 2018 09:57:00 -0800 (PST)
MIME-Version: 1.0
References: <CABCOCHQYR_iY7Lp=0m8D-GQ8+Rzuijaa8_41bJw6ZvWPc+Cw_w@mail.gmail.com> <b2ee0537-465a-fbc0-1b94-000da5993b05@cisco.com> <20181130192830.g2622izshwn4f55z@anna.jacobs.jacobs-university.de> <6281fd79-2284-46c0-350c-770dcdade29d@cisco.com> <20181203113633.v3nfuen66ngjjp6l@anna.jacobs.jacobs-university.de>
In-Reply-To: <20181203113633.v3nfuen66ngjjp6l@anna.jacobs.jacobs-university.de>
From: Andy Bierman <andy@yumaworks.com>
Date: Mon, 3 Dec 2018 09:56:48 -0800
Message-ID: <CABCOCHRC_Z7QJhV-sMrUyX_d49UM8C=x9rgOtTc9iP2Wz52+VQ@mail.gmail.com>
To: Juergen Schoenwaelder <j.schoenwaelder@jacobs-university.de>, Robert Wilton <rwilton@cisco.com>, NetMod WG <netmod@ietf.org>
Content-Type: multipart/alternative; boundary="00000000000027fffb057c21e0da"
Archived-At: <https://mailarchive.ietf.org/arch/msg/netmod/pYE25ZHik-5KDrjvkdP8RBZYSKI>
Subject: Re: [netmod] instance file parsing
X-BeenThere: netmod@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETMOD WG list <netmod.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netmod>, <mailto:netmod-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netmod/>
List-Post: <mailto:netmod@ietf.org>
List-Help: <mailto:netmod-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netmod>, <mailto:netmod-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 03 Dec 2018 17:57:05 -0000

On Mon, Dec 3, 2018 at 3:36 AM Juergen Schoenwaelder <
j.schoenwaelder@jacobs-university.de> wrote:

> On Mon, Dec 03, 2018 at 10:21:20AM +0000, Robert Wilton wrote:
> >
> > On 30/11/2018 19:28, Juergen Schoenwaelder wrote:
> > > I doubt it makes sense to carry an entire yang library schema with the
> > > instance data. The modules are actually known via the namespaces.
> >
> > Knowing the module name or namespace isn't sufficient to be able to parse
> > the data:
> >  - E.g. if the contents are in the format of RFC7895bis, but the client
> > tries to parse it using the schema from RFC7895, then it will fail.
> >  - Once NBC changes are allowed via YANG versioning it will become more
> > necessary to know the revision or version of the modules to be able to
> parse
> > the data.  E.g. not even just the latest module revision will necessarily
> > work.   It is entirely plausible that different server revisions may be
> > generating instance data using slightly different schema.
>
> Perhaps this is a problem of the versioning solution then. ;-)
>
> > If the server has deviations then the client may also need to know those
> > deviations to correctly parse the file.
> >
> > So, I think that it is pretty clear that knowing the right schema is
> > required to be able to correctly parse and interpret the instance data.
> >
> > I think that there are 4 ways that this can be achieved:
> >  1) embedded the necessary schema information into the YANG instance
> data.
> >  2) put the necessary schema information online somewhere and have a URI
> > reference to it.
> >  3) some combination of (1) and (2), e.g. packages defined centrally,
> with
> > deviations listed in the file.
> >  4) the schema is determined using some out of bound mechanism, or
> possibly
> > it is hard coded.
>
> Perhaps we need to define first what "parse" means. I am not sure
> parsing requires to know all schema details. Anyway, if all schema
> details are needed, then the simplest solution seems to be to read the
> schema from another instance file. This then only requires to
> bootstrap the schema model version.
>
>

I agree with Rob.
The solution has to support accurate parsing of the instance data.
This means that the parser has the correct schema tree to use for
validating an instance document against the schema.

Since the new yang-library can have a different schema tree for each
datastore,
clearly the option of specifying the datastore is needed.



> > I don't think that there is a one size fits all answer here.  I think
> that
> > that it depends on the use case.  Certainly, I think that facilitating 1
> -3
> > is useful, but they should be optional rather than mandatory.  I.e.
> defining
> > nodes for these doesn't seem to cost much if a server isn't obliged to
> > populate them.
> >
> > I do think that YANG packages (themselves defined in instance data
> > documents) could be very helpful here.  I.e. rather than listing all the
> > modules, instead list the packages + any deviations from that.  I'm
> > presuming here that the definitions of the packages are available via a
> URI.
>
>

Yes -- this has been my goal for YANG Packages since the start.
By using nested packages and offline caching, the entire YANG library for a
device
could be recognized in a few URIs.



> > >   If
> > > you want to capture the schema, dump the relevant yang library into
> > > another instance file.
> >
> > That just means another file to carry around and manage.
>
> Yes, but compared to solutions that require new and/or much more
> elaborate data formats, this is very cheap and efficient, in
> particular for systems where the schema itself is relatively
> static. Also in terms of engineering time this is a rather cheap
> solution since you do not need to invent a new way to document and
> communicate a schema.
>
>
I agree the cost of an extra instance document is not a problem, especially
since
it would be optional and only used if the defaults are not sufficient:
   - latest revision of module will be OK
   - assume all features present will be OK
   - assume no deviations will be OK

If the defaults are not OK, then the parser will incorrectly accept or
reject certain instance data,
unless the correct schema tree is obtained somehow.



/js
>


Andy


>
> --
> Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
> Fax:   +49 421 200 3103         <https://www.jacobs-university.de/>
>