Re: [netmod] validating instance data against YANG schema including 'must' statements

"Sterne, Jason (Nokia - CA/Ottawa)" <jason.sterne@nokia.com> Fri, 31 January 2020 16:21 UTC

Return-Path: <jason.sterne@nokia.com>
X-Original-To: netmod@ietfa.amsl.com
Delivered-To: netmod@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DDDC9120120 for <netmod@ietfa.amsl.com>; Fri, 31 Jan 2020 08:21:48 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.901
X-Spam-Level:
X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=nokia.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IFTdd1BJMCpO for <netmod@ietfa.amsl.com>; Fri, 31 Jan 2020 08:21:46 -0800 (PST)
Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-eopbgr150121.outbound.protection.outlook.com [40.107.15.121]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 466DB120103 for <netmod@ietf.org>; Fri, 31 Jan 2020 08:21:46 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EsiYxcMxKQ/V5yYK3BvK6EwqLKUEuBUJrDtWgVYJPRM09Ozh2MDOG/es+57b3iMgb/Bt25wkHvWPlYNNR/xRM46tuAPaMXSuBETAyzugMKjOq13lp3c1m1Er7/oX7ZJXPDzl2jLlVQiKqYJ/aJCrbyDasVjLqEntVowa9B1rWIBZ9m5WsWqcCn7ZQNy6iSl7ujyristi4nihDAkRTM9iB+L8a6JsXY66C8JsOuiCCmS8ztryRVFF//RP+XknqOvV/ki7O5ggf8OKb7OzRTFIaggEUHHIMB/+8+oP5eqqKr7WOUPa1jl4PxAUveijhZ3gtCrMlZ1tXIozBtoNUpcJ+g==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dGYKTYXmuTdvEvJobVhiv0n8kmxBYPXr39Lhu+bH8jQ=; b=THF8yW+BglCC7w3iwsNOsPzQChb+PXmP0CUvVWn71gw65BWijlUM1WcHGIPN299eFm5w9Bq4ps3pcvH+B5wb2TXDsj5yOfuiA3pf3Ja1JNgPSRvmfbrLdDCzO2903mpdXyMr6Bnh8K3JAYk1e0wnOe/U2T74VnNaaSxFp61jKmRurR8KV9ECNS+iZHTaM8NSKrc3TtHaBPgZRJfXjjqY7FmUN1PweRJubyA7MJw5FpZW+iJJQMKmphFdV5G53SzJf86d76FB31+DKT3ooSQdcEKW57BzoAXhVzS/iLzyDNO7Moeht7HLLYuswnYctgS2pZs07xziM/g5+JIuy0YC+A==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nokia.com; dmarc=pass action=none header.from=nokia.com; dkim=pass header.d=nokia.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nokia.onmicrosoft.com; s=selector1-nokia-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dGYKTYXmuTdvEvJobVhiv0n8kmxBYPXr39Lhu+bH8jQ=; b=JhDxdKHnUXLsycTjUPfD8zWvejUGaUXXFnEn6TYLXxagpSGYPtLzUaBZPKxBkbYVrDDbNG/T3n9C2iY0J3UKqYdGDC+rXnNSLAYI40Szp4EwOKQg95CjPEGDptwX3xG/598i0kviDHC/ZXTXkGgOyZ2ozbpe1o0jPr2SJzB8gDc=
Received: from VI1PR07MB3981.eurprd07.prod.outlook.com (52.134.29.24) by VI1PR07MB6381.eurprd07.prod.outlook.com (10.186.163.216) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2686.14; Fri, 31 Jan 2020 16:21:44 +0000
Received: from VI1PR07MB3981.eurprd07.prod.outlook.com ([fe80::fcbf:bc9d:6366:e93e]) by VI1PR07MB3981.eurprd07.prod.outlook.com ([fe80::fcbf:bc9d:6366:e93e%5]) with mapi id 15.20.2707.011; Fri, 31 Jan 2020 16:21:44 +0000
From: "Sterne, Jason (Nokia - CA/Ottawa)" <jason.sterne@nokia.com>
To: Ladislav Lhotka <lhotka@nic.cz>, "netmod@ietf.org" <netmod@ietf.org>
Thread-Topic: [netmod] validating instance data against YANG schema including 'must' statements
Thread-Index: AdWxCsORBSgRrCTES6eC/F2G+DspvgAgGU0ACbGh+UA=
Date: Fri, 31 Jan 2020 16:21:44 +0000
Message-ID: <VI1PR07MB398195F5BD06B9DB54F0B0229B070@VI1PR07MB3981.eurprd07.prod.outlook.com>
References: <VI1PR07MB3981BBE54D2E235648758C209B550@VI1PR07MB3981.eurprd07.prod.outlook.com> <ca9cdf86c50f0980caa479bf2621ff0f9bfc0941.camel@nic.cz>
In-Reply-To: <ca9cdf86c50f0980caa479bf2621ff0f9bfc0941.camel@nic.cz>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is ) smtp.mailfrom=jason.sterne@nokia.com;
x-originating-ip: [135.245.20.4]
x-ms-publictraffictype: Email
x-ms-office365-filtering-ht: Tenant
x-ms-office365-filtering-correlation-id: 636ffdba-8f20-4bf0-be40-08d7a669a979
x-ms-traffictypediagnostic: VI1PR07MB6381:
x-microsoft-antispam-prvs: <VI1PR07MB6381E31457722436E7DD68EF9B070@VI1PR07MB6381.eurprd07.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:8882;
x-forefront-prvs: 029976C540
x-forefront-antispam-report: SFV:NSPM; SFS:(10001)(10019020)(4636009)(346002)(376002)(396003)(366004)(136003)(39860400002)(199004)(189003)(86362001)(66446008)(64756008)(76116006)(71200400001)(8676002)(66946007)(66476007)(966005)(33656002)(8936002)(478600001)(316002)(81166006)(5660300002)(66556008)(81156014)(19273905006)(186003)(52536014)(26005)(6506007)(53546011)(2906002)(55016002)(7696005)(9686003)(4001150100001)(110136005)(562404015)(563064011); DIR:OUT; SFP:1102; SCL:1; SRVR:VI1PR07MB6381; H:VI1PR07MB3981.eurprd07.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1;
received-spf: None (protection.outlook.com: nokia.com does not designate permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: dkn4WUC8jDytMw9pH7n9U2llfKDlU3EjtlqLRd0e58AvWL1nFnZCYM24jg2kYAtkiTCkEW2VJUAFIzIIKSsX4Rzn4thEMdQEJGNexihJ5zztEkv2QZgOwgCfW8/YtMMsjE4F+vl7zUsxoA5VmWeX89xZneLZYhLeqoqStta0ifxEVmDIC9gW3opLuWzdhcYCk4aFOpU8Kt+7lZ06NRNpxrFtLM3EjkFEptLVGme69DEJIqOuJgzy3TacfiwacxI1LqWdBpnakz5YfdP5s6EsXbVtLQlVM4K1wMxpAKBPVg/Jf9YCY/Kxv+QfaCL0jjLKMeVp79v4UmWZvxQyMyKpjMx3U9BQwF+tBRvd+HbPExGJ+jqSnD2zMFc/ZLPWQyuvz1ExXAt9sNq48VhG/zaWg+t52/KzrBuPL84JTDJwv7okufmg3Lz4IgqiKfEIR7GWBnZiOVw7q/EQZ7T0J2UdCs8tAvbVwkI1uDUczLpLNJuC7qGP8aOw6Q5vBT+XZgMlirZGKtjztN5LYEtQA4f+AOvvBOc0FqwPZSIrWbqbp2xgwe+w8L84AL/JSLz32+jYiyn0dgpZMA/0VEaStt8Zmsr2PXC1rYCrVTZu/isTYL3VBCQ6619K6/FeqKRFq1ieVWgDRKkaeT6rZBggrcMghuqDTFVWOAtT3KcxlH9jmfM=
x-ms-exchange-antispam-messagedata: VMPm+4xS6a0wIegVOJN6NSXojYsCoq5W1UkSGAVMSXbglRgU8QSMcDVvr+zB8NRP8JVG+EcwGzEd1/iUyNDbbbenrUF1agaIBUD7XZeJ2PK+TT0JG/LjC3ewRKd8om0uric9/HGjqFvbMs7hUapK+w==
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: nokia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 636ffdba-8f20-4bf0-be40-08d7a669a979
X-MS-Exchange-CrossTenant-originalarrivaltime: 31 Jan 2020 16:21:44.0390 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 5d471751-9675-428d-917b-70f44f9630b0
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 9VgXvWq1x23Gyex9+XYZ7DPrQG3drFntPM/sNQRnatRJrphtUJd7vMPlYrr1pe3KuZx7zSyCaXLNSaCMhvikCQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR07MB6381
Archived-At: <https://mailarchive.ietf.org/arch/msg/netmod/w8bHcq3AzkgAdlofL-SLRMFGMio>
Subject: Re: [netmod] validating instance data against YANG schema including 'must' statements
X-BeenThere: netmod@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETMOD WG list <netmod.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netmod>, <mailto:netmod-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netmod/>
List-Post: <mailto:netmod@ietf.org>
List-Help: <mailto:netmod-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netmod>, <mailto:netmod-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 31 Jan 2020 16:21:49 -0000

Thx Lada.

If I only have XML (e.g. from a NETCONF interface) I suppose I could use yanglint, e.g. this type of usage:
yanglint    --format=json    --type=config   --output=data.json   ./all-my-modules/*.yang ./data.xml

Is that correct?

I believe yanglint can also validate instance data against a YANG schema. Can anyone confirm that yang2dsdl and yanglint do *not* validate against the 'must' statements?

Jason

> -----Original Message-----
> From: netmod <netmod-bounces@ietf.org> On Behalf Of Ladislav Lhotka
> Sent: Friday, December 13, 2019 2:59 AM
> To: netmod@ietf.org
> Subject: Re: [netmod] validating instance data against YANG schema including
> 'must' statements
> 
> On Thu, 2019-12-12 at 16:42 +0000, Sterne, Jason (Nokia - CA/Ottawa) wrote:
> > Hi all,
> >
> > A few years ago there were a few discussions on the list about tools to
> > validate instance data (e.g. the data returned by a <get-config>) against a
> > YANG model.
> >
> > yang2dsdl is one option but I'm pretty sure it doesn't actually check the data
> > against 'must' statements.
> >
> > Are there some tools that check against 'must' (and 'when') statements?
> Do
> > those tools also work with YANG 1.1 modules?
> 
> Yangson does a complete validation, and supports YANG 1.1, but only JSON
> representation of instance data. The GitHub link is below, a PyPI package is
> also available:
> 
> https://pypi.org/project/yangson/
> 
> Lada
> 
> >
> > Thx,
> > Jason
> >
> >
> ###############################################################
> ###############
> > ########################
> > Re: [netmod] Toolchain upgraded to yangdump-pro 16.10-5 => 16.10-5..1
> > Ladislav Lhotka <lhotka@nic.cz> Tue, 07 March 2017 12:42 UTCShow
> header
> >
> > Kent Watsen <kwatsen@juniper.net>; writes:
> >
> > > Hi Benoit,
> > >
> > > You seem to know the ins and outs of many tools these days, maybe you
> > > can point me in the right direction...which tool is able to validate
> > > instance documents against YANG 1.1 modules?
> >
> > Yangson can validate JSON documents:
> >
> > https://github.com/CZ-NIC/yangson
> >
> > >
> > > I've always used `yang2dsdl`, but currently it outputs "DSDL plugin
> > > supports only YANG version 1".
> >
> > I considered updating the DSDL plugin to 1.1 but it turned up to be
> > immensely difficult - it would basically require a complete rewrite. And
> > even then, the Schematron implementation that is included in pyang
> > distribution won't support the new XPath functions.
> >
> > Lada
> >
> >
> ###############################################################
> ###############
> > ########################
> >
> > Re: [netmod] DSDL plugin in pyang
> > Ladislav Lhotka <lhotka@nic.cz> Tue, 29 November 2016 13:39 UTCShow
> header
> >
> > Hi William,
> >
> > apart from yang2dsdl, I have personal experience with these two instance
> > validation tools:
> >
> > * yanglint - written in C, supports both XML and JSON instance encoding
> >
> >   https://github.com/CESNET/libyang
> >
> > * yangson - written in Python, supports only JSON
> >
> >   https://github.com/CZ-NIC/yangson
> >   installation: pip install yangson
> >   manual page: http://yangson.readthedocs.io/en/latest/cmdline.html
> >
> > Lada
> >
> > William Lupton <wlupton@broadband-forum.org>; writes:
> >
> > > Are you able to provide a list (either privately or via the NETMOD list) of
> > other instance data validators that are available and cover YANG 1.1
> features?
> > Tx, W.
> > >
> > >> On 25 Nov 2016, at 14:33, Ladislav Lhotka <lhotka@nic.cz>; wrote:
> > >>
> > >> Hi,
> > >>
> > >> for users of $subj: I modified the plugin so that it now immediately
> > refuses to process modules of yang-version greater than 1. Supporting some
> of
> > the YANG 1.1 features (new XPath functions, leafref handling) would require
> > massive changes and I cannot do them now - I am not even sure it is worth
> the
> > effort given that other instance data validators are available.
> > >>
> > >> Lada
> >
> > --
> >
> >
> ###############################################################
> ###############
> > ########################
> >
> > _______________________________________________
> > netmod mailing list
> > netmod@ietf.org
> > https://www.ietf.org/mailman/listinfo/netmod
> --
> Ladislav Lhotka
> Head, CZ.NIC Labs
> PGP Key ID: 0xB8F92B08A9F76C67
> 
> _______________________________________________
> netmod mailing list
> netmod@ietf.org
> https://www.ietf.org/mailman/listinfo/netmod