Re: [netmod] validating instance data against YANG schema including 'must' statements

Mahesh Jethanandani <mjethanandani@gmail.com> Fri, 31 January 2020 17:45 UTC

Return-Path: <mjethanandani@gmail.com>
X-Original-To: netmod@ietfa.amsl.com
Delivered-To: netmod@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 52483120A96 for <netmod@ietfa.amsl.com>; Fri, 31 Jan 2020 09:45:33 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.997
X-Spam-Level:
X-Spam-Status: No, score=-1.997 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lUm-QRL646xT for <netmod@ietfa.amsl.com>; Fri, 31 Jan 2020 09:45:29 -0800 (PST)
Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B615912021C for <netmod@ietf.org>; Fri, 31 Jan 2020 09:45:29 -0800 (PST)
Received: by mail-pl1-x632.google.com with SMTP id p11so3012608plq.10 for <netmod@ietf.org>; Fri, 31 Jan 2020 09:45:29 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:message-id:mime-version:subject:date:in-reply-to:cc:to :references; bh=1AfsioIr/OXJizQ/sqH9pFfPkp9UZ/l2S+4oLS9XvJU=; b=gW43xcm6BuE/nMutFzqL9llP/CKAg8fHHp5Mj2UehRrufeJjCHYqVQYwEaL3fp7L+F zrI3c+ktpoUdc3Vm8dFMpmQ++MTnxkn9Gp9DedTqpWVhKMaiMuq40LV/86ymXKfGLqi4 zQC4CIlug8SWIboDrFpY2uuJ2wg0Ncn4hmVymGqQS2Y2ZPKJWuVzLVMDY58tT/lRpYcW U2ehrC3ckmmjMnw47d8UJsJchubHr/QOpyc4hR+TCubmwouhF5VaE8V3uN27vMZus+so ZH4ec8zMAuKlJEA1nVO7xM/KP/wmp+Pe8sRf97A0YlTrxPjev8KE9teSbGdUVvGHrG4M H2ug==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:message-id:mime-version:subject:date :in-reply-to:cc:to:references; bh=1AfsioIr/OXJizQ/sqH9pFfPkp9UZ/l2S+4oLS9XvJU=; b=Yz8wqxScSg75TYiHScHSUxzkxcdHYIIUTcac2qIOSDLqrpJsU7mjzj0z0NQH1hyUnL SiTlo+s+BZRIOeLU622qOKZ1T8meT6W7GIj7+AxuLtL5U6GvZ063w2a7B/t5qGB1sNQf aIrkClr9a21auC0JtGOyiFLFPlTdhNWn6ckRUCHVNt4jkIXg4WznYkBbSVavrbw3Aqlo 4YWZYMN4I0hpR3sS8W80Ycdm1Xr9SrosNkuN21cLZOPvGuMrWEtfTn9pilBSnGS7yJ/V VPuoNhEo/GxUGk6ZQAD+BOj7oDLZpr9xhtPiwoe6UUzj6tXlnGWvo2mjfbxEKbeFjGdq dlYQ==
X-Gm-Message-State: APjAAAWJTASLSQ3qXafguYhemorHtIhc8139j6RhDOmrb8vCz3sw4JiI ul0ZZCDe1eKBtbFTY9D1PcY=
X-Google-Smtp-Source: APXvYqxcBFUgQ9wKiemvnH34T+G8xTr+RE8oN4BqyFr+2W5Ir7aVEs0NtH/8Jn/w3W4ooypP180Gcw==
X-Received: by 2002:a17:902:59c9:: with SMTP id d9mr11132842plj.184.1580492729115; Fri, 31 Jan 2020 09:45:29 -0800 (PST)
Received: from ?IPv6:2601:647:5600:5020:558b:dc24:5bc9:2f9? ([2601:647:5600:5020:558b:dc24:5bc9:2f9]) by smtp.gmail.com with ESMTPSA id g18sm11028989pfi.80.2020.01.31.09.45.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 31 Jan 2020 09:45:28 -0800 (PST)
From: Mahesh Jethanandani <mjethanandani@gmail.com>
Message-Id: <650ADE92-EAC1-4C8E-ACB3-433B95645AFE@gmail.com>
Content-Type: multipart/alternative; boundary="Apple-Mail=_E31155D5-6D97-43D7-8397-774A71563361"
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
Date: Fri, 31 Jan 2020 09:45:26 -0800
In-Reply-To: <VI1PR07MB398195F5BD06B9DB54F0B0229B070@VI1PR07MB3981.eurprd07.prod.outlook.com>
Cc: Ladislav Lhotka <lhotka@nic.cz>, "netmod@ietf.org" <netmod@ietf.org>
To: Jason Sterne <jason.sterne@nokia.com>
References: <VI1PR07MB3981BBE54D2E235648758C209B550@VI1PR07MB3981.eurprd07.prod.outlook.com> <ca9cdf86c50f0980caa479bf2621ff0f9bfc0941.camel@nic.cz> <VI1PR07MB398195F5BD06B9DB54F0B0229B070@VI1PR07MB3981.eurprd07.prod.outlook.com>
X-Mailer: Apple Mail (2.3445.9.1)
Archived-At: <https://mailarchive.ietf.org/arch/msg/netmod/sMGxF8HqcFm6IJqpzxYalHpGZX8>
Subject: Re: [netmod] validating instance data against YANG schema including 'must' statements
X-BeenThere: netmod@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: NETMOD WG list <netmod.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/netmod>, <mailto:netmod-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/netmod/>
List-Post: <mailto:netmod@ietf.org>
List-Help: <mailto:netmod-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/netmod>, <mailto:netmod-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 31 Jan 2020 17:45:33 -0000

Hi Jason,

yanglint does validate XML instance data for must statements:

MAHESH-M-M8D1:/Volumes/External/git/my-YANG-public/src/model/draft *833 > yanglint -t auto -s -i -p common ../../../build/mef-legato-services@2018-07-17.yang MEF6.2-bwp-per-uni-mef-interface-configuration.xml 
err : Must condition ". >= 1 and . <= count(../../bwp-flow/rank)" not satisfied. (/mef-legato-interfaces:mef-interfaces/carrier-ethernet/subscriber-interfaces/uni[uni-id='ciscoD21:GigabitEthernet0/0/0/1']/ingress-envelopes/envelope[id='ciscoD21-per-cos-env2']/bwp-flows/bwp-flow[rank='3']/rank)
err : The rank of a Bandwidth Profile Flow must be
between 1 and n, where n is the number of flows
in the Envelope (/mef-legato-interfaces:mef-interfaces/carrier-ethernet/subscriber-interfaces/uni[uni-id='ciscoD21:GigabitEthernet0/0/0/1']/ingress-envelopes/envelope[id='ciscoD21-per-cos-env2']/bwp-flows/bwp-flow[rank='3']/rank)

> On Jan 31, 2020, at 8:21 AM, Sterne, Jason (Nokia - CA/Ottawa) <jason.sterne@nokia.com> wrote:
> 
> Thx Lada.
> 
> If I only have XML (e.g. from a NETCONF interface) I suppose I could use yanglint, e.g. this type of usage:
> yanglint    --format=json    --type=config   --output=data.json   ./all-my-modules/*.yang ./data.xml
> 
> Is that correct?
> 
> I believe yanglint can also validate instance data against a YANG schema. Can anyone confirm that yang2dsdl and yanglint do *not* validate against the 'must' statements?
> 
> Jason
> 
>> -----Original Message-----
>> From: netmod <netmod-bounces@ietf.org> On Behalf Of Ladislav Lhotka
>> Sent: Friday, December 13, 2019 2:59 AM
>> To: netmod@ietf.org
>> Subject: Re: [netmod] validating instance data against YANG schema including
>> 'must' statements
>> 
>> On Thu, 2019-12-12 at 16:42 +0000, Sterne, Jason (Nokia - CA/Ottawa) wrote:
>>> Hi all,
>>> 
>>> A few years ago there were a few discussions on the list about tools to
>>> validate instance data (e.g. the data returned by a <get-config>) against a
>>> YANG model.
>>> 
>>> yang2dsdl is one option but I'm pretty sure it doesn't actually check the data
>>> against 'must' statements.
>>> 
>>> Are there some tools that check against 'must' (and 'when') statements?
>> Do
>>> those tools also work with YANG 1.1 modules?
>> 
>> Yangson does a complete validation, and supports YANG 1.1, but only JSON
>> representation of instance data. The GitHub link is below, a PyPI package is
>> also available:
>> 
>> https://pypi.org/project/yangson/
>> 
>> Lada
>> 
>>> 
>>> Thx,
>>> Jason
>>> 
>>> 
>> ###############################################################
>> ###############
>>> ########################
>>> Re: [netmod] Toolchain upgraded to yangdump-pro 16.10-5 => 16.10-5..1
>>> Ladislav Lhotka <lhotka@nic.cz> Tue, 07 March 2017 12:42 UTCShow
>> header
>>> 
>>> Kent Watsen <kwatsen@juniper.net>; writes:
>>> 
>>>> Hi Benoit,
>>>> 
>>>> You seem to know the ins and outs of many tools these days, maybe you
>>>> can point me in the right direction...which tool is able to validate
>>>> instance documents against YANG 1.1 modules?
>>> 
>>> Yangson can validate JSON documents:
>>> 
>>> https://github.com/CZ-NIC/yangson
>>> 
>>>> 
>>>> I've always used `yang2dsdl`, but currently it outputs "DSDL plugin
>>>> supports only YANG version 1".
>>> 
>>> I considered updating the DSDL plugin to 1.1 but it turned up to be
>>> immensely difficult - it would basically require a complete rewrite. And
>>> even then, the Schematron implementation that is included in pyang
>>> distribution won't support the new XPath functions.
>>> 
>>> Lada
>>> 
>>> 
>> ###############################################################
>> ###############
>>> ########################
>>> 
>>> Re: [netmod] DSDL plugin in pyang
>>> Ladislav Lhotka <lhotka@nic.cz> Tue, 29 November 2016 13:39 UTCShow
>> header
>>> 
>>> Hi William,
>>> 
>>> apart from yang2dsdl, I have personal experience with these two instance
>>> validation tools:
>>> 
>>> * yanglint - written in C, supports both XML and JSON instance encoding
>>> 
>>>  https://github.com/CESNET/libyang
>>> 
>>> * yangson - written in Python, supports only JSON
>>> 
>>>  https://github.com/CZ-NIC/yangson
>>>  installation: pip install yangson
>>>  manual page: http://yangson.readthedocs.io/en/latest/cmdline.html
>>> 
>>> Lada
>>> 
>>> William Lupton <wlupton@broadband-forum.org>; writes:
>>> 
>>>> Are you able to provide a list (either privately or via the NETMOD list) of
>>> other instance data validators that are available and cover YANG 1.1
>> features?
>>> Tx, W.
>>>> 
>>>>> On 25 Nov 2016, at 14:33, Ladislav Lhotka <lhotka@nic.cz>; wrote:
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> for users of $subj: I modified the plugin so that it now immediately
>>> refuses to process modules of yang-version greater than 1. Supporting some
>> of
>>> the YANG 1.1 features (new XPath functions, leafref handling) would require
>>> massive changes and I cannot do them now - I am not even sure it is worth
>> the
>>> effort given that other instance data validators are available.
>>>>> 
>>>>> Lada
>>> 
>>> --
>>> 
>>> 
>> ###############################################################
>> ###############
>>> ########################
>>> 
>>> _______________________________________________
>>> netmod mailing list
>>> netmod@ietf.org
>>> https://www.ietf.org/mailman/listinfo/netmod
>> --
>> Ladislav Lhotka
>> Head, CZ.NIC Labs
>> PGP Key ID: 0xB8F92B08A9F76C67
>> 
>> _______________________________________________
>> netmod mailing list
>> netmod@ietf.org
>> https://www.ietf.org/mailman/listinfo/netmod
> 
> _______________________________________________
> netmod mailing list
> netmod@ietf.org
> https://www.ietf.org/mailman/listinfo/netmod