Re: [yang-doctors] XPath validation during development

Kent Watsen <kent@watsen.net> Mon, 02 March 2020 20:14 UTC

Return-Path: <010001709ce294c6-d6393ee5-9c3c-4d0f-80c6-3ffff2523742-000000@amazonses.watsen.net>
X-Original-To: yang-doctors@ietfa.amsl.com
Delivered-To: yang-doctors@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5C1753A108C for <yang-doctors@ietfa.amsl.com>; Mon, 2 Mar 2020 12:14:24 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.896
X-Spam-Level:
X-Spam-Status: No, score=-1.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=amazonses.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qn2HHqGQ5fJU for <yang-doctors@ietfa.amsl.com>; Mon, 2 Mar 2020 12:14:22 -0800 (PST)
Received: from a8-96.smtp-out.amazonses.com (a8-96.smtp-out.amazonses.com [54.240.8.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7D1B63A108A for <yang-doctors@ietf.org>; Mon, 2 Mar 2020 12:14:22 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=6gbrjpgwjskckoa6a5zn6fwqkn67xbtw; d=amazonses.com; t=1583180061; h=From:Message-Id:Content-Type:Mime-Version:Subject:Date:In-Reply-To:Cc:To:References:Feedback-ID; bh=klUQjpJUoDlt8tPwZ3Gb/LWJ0HwcdkJ5z5BkD8qYM4g=; b=jGql3jW/ZJvB8kuoT76Jd2t2gaWdt1WhAvcQyiSyf7bzkTIy6bzX9si+1urScf6n K1cO80pY6sMdbJ+xKtVz5UwSDLgqgZABBTM1ImJjKJJGhRvq7WDS86kzN0RrHIL02O9 8Ktsqj5+yIli5DW8y7zXLx5WM0kxO7ACKlCjFVFo=
From: Kent Watsen <kent@watsen.net>
Message-ID: <010001709ce294c6-d6393ee5-9c3c-4d0f-80c6-3ffff2523742-000000@email.amazonses.com>
Content-Type: multipart/alternative; boundary="Apple-Mail=_008DAF47-6600-4B87-8CDF-D8058B587F3E"
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Date: Mon, 02 Mar 2020 20:14:20 +0000
In-Reply-To: <DM5PR15MB1772033DBA980333DF41A5348BE70@DM5PR15MB1772.namprd15.prod.outlook.com>
Cc: "yang-doctors@ietf.org" <yang-doctors@ietf.org>
To: Scott Mansfield <scott.mansfield=40ericsson.com@dmarc.ietf.org>
References: <DM5PR15MB1772033DBA980333DF41A5348BE70@DM5PR15MB1772.namprd15.prod.outlook.com>
X-Mailer: Apple Mail (2.3445.104.11)
X-SES-Outgoing: 2020.03.02-54.240.8.96
Feedback-ID: 1.us-east-1.DKmIRZFhhsBhtmFMNikgwZUWVrODEw9qVcPhqJEI2DA=:AmazonSES
Archived-At: <https://mailarchive.ietf.org/arch/msg/yang-doctors/WyRkF-Jr407YgjjeA72N7RATivk>
Subject: Re: [yang-doctors] XPath validation during development
X-BeenThere: yang-doctors@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Email list of the yang-doctors directorate <yang-doctors.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/yang-doctors>, <mailto:yang-doctors-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/yang-doctors/>
List-Post: <mailto:yang-doctors@ietf.org>
List-Help: <mailto:yang-doctors-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/yang-doctors>, <mailto:yang-doctors-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 02 Mar 2020 20:14:25 -0000

“Best” is hard to qualify, but perhaps best would be to 1) validate the YANG module(s) and 2) validate as many instance documents as it takes to have near 100% coverage; and doing both is these things with as many validation tools as possible.

I do this myself for all my models, though I only ave access to `yanglint` and `yangson`.  For drafts, my Makefiles do this for each build (pro-tip: the time setting up the automation pays for itself in the long run).

A couple additional comments:

1) `yangson` has an ability to output test-coverage statistics per https://github.com/CZ-NIC/yangson/issues/13 <https://github.com/CZ-NIC/yangson/issues/13>.

2) testing against multiple implementation has two benefits: a) some implementation catch things others don’t and b) sometimes inconsistencies arise that resolve by either i) one implementation being improved or ii) discovery of a clarification-issue with YANG (RFC 7950).  Example open clarification issues include how NP-containers and implicit-case statements are handled.  My modules are now constructed in such a way to be behaviorally-correct across a greater set of tools (e.g., avoid implicit-case statements, always use a presence-container if a descendent contains a “must” statement, etc.)

Kent


> On Mar 2, 2020, at 2:24 PM, Scott Mansfield <scott.mansfield=40ericsson.com@dmarc.ietf.org> wrote:
> 
> Hello YANG Doctors!
>  
> I searched through the yang-doctors mailing list and didn’t see an answer to my question, so apologies if this has been covered before.
>  
> While development a YANG module, how is the best way to ensure that my XPath (must etc) statements are syntactically and semantically correct?  Is the best way to use YUMA123 or confd and build a set of test cases that can be run on an instantiated model?  What do you do to test your XPath during development of YANG?
>  
> Many thanks,
> -scott.
> _______________________________________________
> yang-doctors mailing list
> yang-doctors@ietf.org <mailto:yang-doctors@ietf.org>
> https://www.ietf.org/mailman/listinfo/yang-doctors <https://www.ietf.org/mailman/listinfo/yang-doctors>