Re: [yang-doctors] automating yang doctor reviews

Andy Bierman <andy@yumaworks.com> Tue, 24 April 2018 20:40 UTC

Return-Path: <andy@yumaworks.com>
X-Original-To: yang-doctors@ietfa.amsl.com
Delivered-To: yang-doctors@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 44B8512D87C for <yang-doctors@ietfa.amsl.com>; Tue, 24 Apr 2018 13:40:57 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.609
X-Spam-Level:
X-Spam-Status: No, score=-2.609 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=yumaworks-com.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bZGfZcZBq6gP for <yang-doctors@ietfa.amsl.com>; Tue, 24 Apr 2018 13:40:54 -0700 (PDT)
Received: from mail-lf0-x22b.google.com (mail-lf0-x22b.google.com [IPv6:2a00:1450:4010:c07::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AC08812D96D for <yang-doctors@ietf.org>; Tue, 24 Apr 2018 13:40:53 -0700 (PDT)
Received: by mail-lf0-x22b.google.com with SMTP id q5-v6so22267683lff.12 for <yang-doctors@ietf.org>; Tue, 24 Apr 2018 13:40:53 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yumaworks-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Qn4jlq93jE26hyB8bP3RkoafB2DwPYpJ6INrTBrBi/E=; b=ujR7i+FwQkWAob+pxlXxyltIEYDbO0y8VOf+9KtpNsvFL9kcBAAonq5PYKGQswl1IH bX+5bDScgT6RPQSNRBKM76Ny2C6KLHm7YPBXzuBQT7qPLgs6ugDfWTq/nJDYU1wVF20F UmN6/gEfTdkEm2tcmUUlc1T54J+Am/X/wKWbEirx/sKl1xO6rFQHW4zIEKrUsPewP5D6 AAJGIOLmh4oc5zefMFAvGY1LK2UMi78UYGs4mqxYhHNlVFazsGUdWte7UotB+QWM1EPv xpEFMRL12nsp5J4kyAN3SkJDbxdlwCXWbeM52TA3nx0U+ow3oEZpTbMBag0mCp86hRZr 56iQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Qn4jlq93jE26hyB8bP3RkoafB2DwPYpJ6INrTBrBi/E=; b=MC88rrVs7nvJ/G907nnqn1aPwQ8V/frXru6PXs4fhRBI+W+++jBNhRMzGa2sOBbwGG YkSzJVudu2pKF4NUOB5cDa3fkqEeWWFqP/6v7vgSRlugbl9LdHF1nAvT8WxA9DCjG8ll PTa9nSPni0OWQLEv3KSRQewu/oEaI3V70nf1TS5mvhFoStWPnPpz89VVIvkv1spdko5A TIfOAclJuggEvuGANp7pl50NBj8eTDIBLQMMu+gyxB5pvmDW1+4AM5SypkSNjJcJpTqv Ugr47QrgbFkM2Bege5bXQtr5GwC66gYgFUVIjmEIQXdiNs99xe5PzIikgWQY7ECtHdg8 /1ng==
X-Gm-Message-State: ALQs6tAYKp3ScSN+3FcJCCzDO0C9kjtfUWAiJqIqBZ9QL7j6HahOOAAC +TWuhnkdUb+lWF5F9ACY8uXoIigfRMb1QuOxeT/A6Q==
X-Google-Smtp-Source: AIpwx4+kd40OG+/Fc2Frc2MyivoM1DkCGH0jYdiQGFmiIMxjCtz29DZT+eYDKZyEr1HR6GPk/FmbsdxjebjFd5I3dzA=
X-Received: by 10.46.62.12 with SMTP id l12mr17218522lja.66.1524602451927; Tue, 24 Apr 2018 13:40:51 -0700 (PDT)
MIME-Version: 1.0
Received: by 2002:a19:d8c6:0:0:0:0:0 with HTTP; Tue, 24 Apr 2018 13:40:51 -0700 (PDT)
In-Reply-To: <469B6948-794C-4D97-90C8-4957A5D2C6BC@juniper.net>
References: <51E10A3A-FF6F-4A14-AAD6-BBD12041EF2F@juniper.net> <20180423202511.omhs22xnios3itjw@elstar.local> <469B6948-794C-4D97-90C8-4957A5D2C6BC@juniper.net>
From: Andy Bierman <andy@yumaworks.com>
Date: Tue, 24 Apr 2018 13:40:51 -0700
Message-ID: <CABCOCHQq58BJp3ZQKN6aCRKEkHhTTNCfT-vEd89YZk6PwHYwGw@mail.gmail.com>
To: Kent Watsen <kwatsen@juniper.net>
Cc: Juergen Schoenwaelder <j.schoenwaelder@jacobs-university.de>, "yang-doctors@ietf.org" <yang-doctors@ietf.org>
Content-Type: multipart/alternative; boundary="f4f5e8075c4092378a056a9e2b74"
Archived-At: <https://mailarchive.ietf.org/arch/msg/yang-doctors/d_mYwaxaVXVxklf2C6yKg7s7yI0>
Subject: Re: [yang-doctors] automating yang doctor reviews
X-BeenThere: yang-doctors@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Email list of the yang-doctors directorate <yang-doctors.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/yang-doctors>, <mailto:yang-doctors-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/yang-doctors/>
List-Post: <mailto:yang-doctors@ietf.org>
List-Help: <mailto:yang-doctors-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/yang-doctors>, <mailto:yang-doctors-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 24 Apr 2018 20:40:57 -0000

Hi,

This seems like an elaborate project, but a valuable goal.

The first assumption is that it is YANG Doctors' responsibility to validate
every example and every tree diagram in a draft.  If this is the case,
then we all agree that tools should do the work, not people.

Since people rely on the tree diagrams and examples (maybe too much)
they have to be correct. This means (1) adding them early and (2) updating
them as
things change.

I suggest getting input from the IETF Tools Team as to how idnits can solve
this problem.
What tools need to be developed. What is the scope of the work (just
NETCONF, RESTCONF, and YANG?).


Andy






On Tue, Apr 24, 2018 at 12:35 PM, Kent Watsen <kwatsen@juniper.net>; wrote:

> Hi Juergen,
>
> Yes, the gist of the draft is to put everything in a single
> XML-based draft-submission document.   The thinking for this
> was that it would be easier for the ietf-tools team to act on.
>
> Turning it around and introducing multiple files to be
> submitted as a collection (e.g., a TGZ?) is an interesting
> idea, not for its elimination of the extraction-step (which
> is trivial), but for its ability to include extras (that
> would otherwise not be in the draft itself) including:
>
>  - additional YANG modules needed to validate import statements
>    (though the is a danger here that they might be out of date)
>
>  - additional instance documents (xml/json files) needed to when
>    validating examples.  E.g. some tools need to RPC file when
>    validating an RPC-reply; also, some tools need to have an
>    instance document to resolve leafrefs used in examples.
>
>  - scripts that can be used to perform the validation, rather
>    than embedding a command-line into an XML attribute.  As a
>    standalone file, the scripts could be arbitrarily complex.
>    E.g., for the zerotouch draft, I use scripts to convert the
>    RESTCONF-based examples into NETCONF-based examples that get
>    validated, since no tool I'm aware of knows how to validate
>    RESTCONF-based examples.
>
> So, if I understand your proposal, the idea is:
>
>  1) to enable the submission of:
>      - a pile of files
>      - a script that can validate each file
>      - a script that can build a resulting XML2RFC-based doc,
>        including the generation of any tree-diagrams
>
>  2) this raw submission structure would be available throughout
>     the document publication process (I-D --> AUTH48)
>
>  3) the generation of the XML2RFC-based doc would occur
>     whenever an update is made, so that all the existing
>     web UI would remain intact.
>
>
> I know that you're not a fan of bloating the XML-doc into an
> all-encompassing container, but I don't see it as that big of
> a deal, and would even consider adding some new XML2RFC elements
> (e.g., <file>, <script>, etc.) to address some of the gaps
> identified above so the one file can include everything needed.
>
> Kent
>
>
> ===== original message =====
>
> Kent,
>
> it seems that many of us manage several artefacts while producing an
> I-D (that is the .xml that is then rendered as an I-D).
>
> a) YANG modules
> b) derived artefacts such as tree diagrams
> c) examples snippets (XML or JSON)
> d) source of textual information (growing variety of formats used here)
> e) tooling to validate examples and YANG modules
> f) tooling to produce .xml out of a)-d)
>
> What you are proposing, if I understand correctly, is to extend the
> xml2rfc vocabulary such that we can reverse step f), at least
> partially, to extract a) and c) out of the xml2rfc format and perhaps
> generate b) automatically from a) and then do e) within the IETF
> infrastructure. I am not sure I am excited about turning a document
> markup format (xml2rfc) into a general container format and perhaps
> also into a (scriptable) data transformation format. (But yes, it
> sound much better than what we do today, extracting artefacts out of
> rendered and paginated text.)
>
> Perhaps things could be simpler if authors would be able to submit all
> pieces that make up an I-D as separate artefacts instead of first
> inlining all of the artefacts in some xml2rfc container format and
> then extracting them again out of this format to feed tool chains.
>
> The whole idea that a document is a single file using a single format
> really feels somewhat odd these days. (Well, it already felt odd when
> people were writing MIB modules and the RFC editor used nroff or plain
> text to start their editing. The pain isn't new, just coming along in
> a different flavor.)
>
> Perhaps all we need is a proper sourcecode include mechanism and a
> submission tool that allows to submit multiple artefacts so we do not
> need to first inline everything into xml2rfc and then extract things
> out of it again.
>
> /js
>
> On Mon, Apr 23, 2018 at 06:40:30PM +0000, Kent Watsen wrote:
> >
> > anyone interested in this draft?  (draft attached in my previous message)
> >
> > /kw
> >
> >
> > ===== original message =====
> >
> > Doctors,
> >
> > Here's a stab at how we might automate the basic parts of a YANG Doctor
> review, something I've mentioned wanting at the YD-lunch meeting at the
> last two IETF meetings.
> >
> > I'll be the first to say that this proposal has issues, but hopefully
> it's in the ballpark, and we can finish it off together, assuming there is
> interest in bringing it forward at all...
> >
> > Note, I assume that this document will be AD-sponsored, just like RFC
> 7991 was, hence why the draft name is what it is.
> >
> > Kent
> >
> >
> >
> >
> > _______________________________________________
> > yang-doctors mailing list
> > yang-doctors@ietf.org
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.
> ietf.org_mailman_listinfo_yang-2Ddoctors&d=DwIBAg&c=
> HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=
> 9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=UrziQP3s4fITo_
> wUsi2cdmXRqEtk_zMyn2Ut8_jri7E&s=7QA7pbsnofbw_
> zkgaZMhCEAJ1s4XokxRbhuihtUhibo&e=
>
> --
> Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> Phone: +49 421 200 3587         Campus Ring 1 | 28759 Bremen | Germany
> Fax:   +49 421 200 3103         <https://urldefense.
> proofpoint.com/v2/url?u=https-3A__www.jacobs-2Duniversity.de_&d=DwIBAg&c=
> HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=
> 9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=UrziQP3s4fITo_
> wUsi2cdmXRqEtk_zMyn2Ut8_jri7E&s=GPgpXZD_hh6MbuLEDrj2KHARk2aPtKWbTjJwNl
> s_kvE&e=>
>
>
> _______________________________________________
> yang-doctors mailing list
> yang-doctors@ietf.org
> https://www.ietf.org/mailman/listinfo/yang-doctors
>