Re: [yang-doctors] automating yang doctor reviews

Kent Watsen <> Tue, 24 April 2018 19:35 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 4088412D7EC for <>; Tue, 24 Apr 2018 12:35:45 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.7
X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 3MPM2LJc8xqp for <>; Tue, 24 Apr 2018 12:35:42 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id F2ABF120727 for <>; Tue, 24 Apr 2018 12:35:41 -0700 (PDT)
Received: from pps.filterd ( []) by ( with SMTP id w3OJZPHj019878; Tue, 24 Apr 2018 12:35:41 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=PPS1017; bh=IwpU1uz8lrVITaDYeiSW9cQPo5fPW9WaNK1bs9d7MRk=; b=wbyydDGphLsLDx3HF5zjD+wId+HVJh15CQFpiy2RjDZgjFvuGoHkNOiWzU8+OsWqXeio ZTb30GnMSSKNNWmYEkzjTqkOWC8I4Wkpnsmf4jyjIvFn7HBbx2sht7BAppzzsA0Y4atP Ld4tqwJcAQLpPi9XbFAODctHExMRp8uk96/2BXq5RkGX4x4I2dWKIT4p/ykyOSnMFzMR UKdWuOwwPVlEtLanZ89+PLjHBHHKoitOcp1klZ23jyQIkzD0cUVE+rAq0gMnS20aoqLe dsDtkzcQBMaUuxIb4pepJZoyYd5lncO5vroY4SwPVOqnye9bYabzEnDxOybI7VKMYBP+ Fg==
Received: from ( []) by with ESMTP id 2hj650rmsd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Tue, 24 Apr 2018 12:35:41 -0700
Received: from ( by ( with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.715.14; Tue, 24 Apr 2018 19:35:38 +0000
Received: from ([fe80::173:36cf:42b7:5965]) by ([fe80::173:36cf:42b7:5965%6]) with mapi id 15.20.0715.015; Tue, 24 Apr 2018 19:35:38 +0000
From: Kent Watsen <>
To: Juergen Schoenwaelder <>
CC: "" <>
Thread-Topic: [yang-doctors] automating yang doctor reviews
Thread-Index: AQHT2zKO/z8cMyy250C9E9+CDPU0IaQOy/aAgAFBbgA=
Date: Tue, 24 Apr 2018 19:35:38 +0000
Message-ID: <>
References: <> <20180423202511.omhs22xnios3itjw@elstar.local>
In-Reply-To: <20180423202511.omhs22xnios3itjw@elstar.local>
Accept-Language: en-US
Content-Language: en-US
user-agent: Microsoft-MacOutlook/f.20.0.170309
x-originating-ip: []
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; DM5PR05MB3212; 7:7HkhH5U0cK1kwm4Xnabl7gYt/jQOa8WLj3U/viy0OEhBnDVzWbzOu128Emk0G/JmJsisge8mQbZGs1z78TkgZjjIY6gEC22zwAhQfV2sWNZKn01O8ZyKxKVzrbgu1FBk/jcbX5W9Fk9PBl0hNawfSz5KrqAO9fUSYsmIgwkOSp5rpgMrm5kozhm6snoHdlOtbbLwCEPWLNCxW6ffOPioDcc8qok1G7BHoaM0/QkgJpBCU62CA4L8DHC5gWONbFq5
x-ms-exchange-antispam-srfa-diagnostics: SOS;
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(48565401081)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:DM5PR05MB3212;
x-ms-traffictypediagnostic: DM5PR05MB3212:
x-microsoft-antispam-prvs: <>
x-exchange-antispam-report-test: UriScan:(10436049006162)(131327999870524);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(10201501046)(3002001)(3231232)(944501410)(52105095)(93006095)(93001095)(6055026)(6041310)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123558120)(6072148)(201708071742011); SRVR:DM5PR05MB3212; BCL:0; PCL:0; RULEID:; SRVR:DM5PR05MB3212;
x-forefront-prvs: 0652EA5565
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(39860400002)(396003)(366004)(376002)(346002)(39380400002)(199004)(189003)(81166006)(81156014)(8936002)(8676002)(102836004)(66066001)(25786009)(561944003)(76176011)(33656002)(59450400001)(6506007)(229853002)(2900100001)(6246003)(36756003)(6306002)(53936002)(6512007)(99286004)(6486002)(106356001)(82746002)(4326008)(105586002)(2616005)(446003)(11346002)(476003)(6436002)(83716003)(5660300001)(68736007)(486006)(6916009)(5250100002)(316002)(478600001)(3280700002)(58126008)(26005)(3846002)(7736002)(6116002)(86362001)(305945005)(14454004)(2906002)(3660700001)(97736004)(186003); DIR:OUT; SFP:1102; SCL:1; SRVR:DM5PR05MB3212;; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1;
received-spf: None ( does not designate permitted sender hosts)
x-microsoft-antispam-message-info: EgTcTbLnEDLUjfRxrYTt7izIotnItosW9IMnomwciqJc5KH2YZo5YV6NWRup16kJR0QihWrCOlXlnOfFhg4nVPrkLgE/L5HmvuTUqBtpOEaWYh7FNYTVVhI0pkidPItL1hcEoZZJIlKZCQCcAfRYkXo24beQK7DeJCBI2ThRuHpQjnPVtwaXEKkIcWVan+y6
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="utf-8"
Content-ID: <>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Office365-Filtering-Correlation-Id: bef2781d-116e-4f77-1757-08d5aa1a8f0c
X-MS-Exchange-CrossTenant-Network-Message-Id: bef2781d-116e-4f77-1757-08d5aa1a8f0c
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Apr 2018 19:35:38.7948 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bea78b3c-4cdb-4130-854a-1d193232e5f4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR05MB3212
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-04-24_05:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_spam_notspam policy=outbound_spam score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1804240186
Archived-At: <>
Subject: Re: [yang-doctors] automating yang doctor reviews
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Email list of the yang-doctors directorate <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Tue, 24 Apr 2018 19:35:45 -0000

Hi Juergen,

Yes, the gist of the draft is to put everything in a single 
XML-based draft-submission document.   The thinking for this
was that it would be easier for the ietf-tools team to act on.

Turning it around and introducing multiple files to be 
submitted as a collection (e.g., a TGZ?) is an interesting 
idea, not for its elimination of the extraction-step (which 
is trivial), but for its ability to include extras (that 
would otherwise not be in the draft itself) including:

 - additional YANG modules needed to validate import statements
   (though the is a danger here that they might be out of date)

 - additional instance documents (xml/json files) needed to when 
   validating examples.  E.g. some tools need to RPC file when
   validating an RPC-reply; also, some tools need to have an
   instance document to resolve leafrefs used in examples.

 - scripts that can be used to perform the validation, rather
   than embedding a command-line into an XML attribute.  As a
   standalone file, the scripts could be arbitrarily complex.
   E.g., for the zerotouch draft, I use scripts to convert the
   RESTCONF-based examples into NETCONF-based examples that get
   validated, since no tool I'm aware of knows how to validate
   RESTCONF-based examples.

So, if I understand your proposal, the idea is:

 1) to enable the submission of:
     - a pile of files
     - a script that can validate each file
     - a script that can build a resulting XML2RFC-based doc, 
       including the generation of any tree-diagrams

 2) this raw submission structure would be available throughout
    the document publication process (I-D --> AUTH48)

 3) the generation of the XML2RFC-based doc would occur
    whenever an update is made, so that all the existing
    web UI would remain intact.

I know that you're not a fan of bloating the XML-doc into an
all-encompassing container, but I don't see it as that big of
a deal, and would even consider adding some new XML2RFC elements
(e.g., <file>, <script>, etc.) to address some of the gaps
identified above so the one file can include everything needed.


===== original message =====


it seems that many of us manage several artefacts while producing an
I-D (that is the .xml that is then rendered as an I-D).

a) YANG modules
b) derived artefacts such as tree diagrams
c) examples snippets (XML or JSON)
d) source of textual information (growing variety of formats used here)
e) tooling to validate examples and YANG modules
f) tooling to produce .xml out of a)-d)

What you are proposing, if I understand correctly, is to extend the
xml2rfc vocabulary such that we can reverse step f), at least
partially, to extract a) and c) out of the xml2rfc format and perhaps
generate b) automatically from a) and then do e) within the IETF
infrastructure. I am not sure I am excited about turning a document
markup format (xml2rfc) into a general container format and perhaps
also into a (scriptable) data transformation format. (But yes, it
sound much better than what we do today, extracting artefacts out of
rendered and paginated text.)

Perhaps things could be simpler if authors would be able to submit all
pieces that make up an I-D as separate artefacts instead of first
inlining all of the artefacts in some xml2rfc container format and
then extracting them again out of this format to feed tool chains.

The whole idea that a document is a single file using a single format
really feels somewhat odd these days. (Well, it already felt odd when
people were writing MIB modules and the RFC editor used nroff or plain
text to start their editing. The pain isn't new, just coming along in
a different flavor.)

Perhaps all we need is a proper sourcecode include mechanism and a
submission tool that allows to submit multiple artefacts so we do not
need to first inline everything into xml2rfc and then extract things
out of it again.


On Mon, Apr 23, 2018 at 06:40:30PM +0000, Kent Watsen wrote:
> anyone interested in this draft?  (draft attached in my previous message)
> /kw
> ===== original message =====
> Doctors,
> Here's a stab at how we might automate the basic parts of a YANG Doctor review, something I've mentioned wanting at the YD-lunch meeting at the last two IETF meetings.
> I'll be the first to say that this proposal has issues, but hopefully it's in the ballpark, and we can finish it off together, assuming there is interest in bringing it forward at all...
> Note, I assume that this document will be AD-sponsored, just like RFC 7991 was, hence why the draft name is what it is.
> Kent
> _______________________________________________
> yang-doctors mailing list

Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
Fax:   +49 421 200 3103         <>