Re: [yang-doctors] YD review and yang-push and friends

Martin Bjorklund <mbj@tail-f.com> Thu, 15 March 2018 07:33 UTC

Return-Path: <mbj@tail-f.com>
X-Original-To: yang-doctors@ietfa.amsl.com
Delivered-To: yang-doctors@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 895B4127735 for <yang-doctors@ietfa.amsl.com>; Thu, 15 Mar 2018 00:33:44 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.91
X-Spam-Level:
X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HnT-AbL3C76T for <yang-doctors@ietfa.amsl.com>; Thu, 15 Mar 2018 00:33:42 -0700 (PDT)
Received: from mail.tail-f.com (mail.tail-f.com [46.21.102.45]) by ietfa.amsl.com (Postfix) with ESMTP id 9C1BD124234 for <yang-doctors@ietf.org>; Thu, 15 Mar 2018 00:33:41 -0700 (PDT)
Received: from localhost (h-80-27.A165.priv.bahnhof.se [212.85.80.27]) by mail.tail-f.com (Postfix) with ESMTPSA id A2A4A1AE02EF; Thu, 15 Mar 2018 08:33:39 +0100 (CET)
Date: Thu, 15 Mar 2018 08:33:39 +0100
Message-Id: <20180315.083339.570580949668320282.mbj@tail-f.com>
To: andy@yumaworks.com
Cc: kwatsen@juniper.net, mersue@gmail.com, bclaise@cisco.com, yang-doctors@ietf.org
From: Martin Bjorklund <mbj@tail-f.com>
In-Reply-To: <CABCOCHTvsHtXrKUaea9OsnMgLK_xBFoDZ9QSVr_hDorcNP8dXA@mail.gmail.com>
References: <007301d3badd$7b300640$719012c0$@gmail.com> <A98F0B76-B46F-4FFD-8543-2AE619CB3812@juniper.net> <CABCOCHTvsHtXrKUaea9OsnMgLK_xBFoDZ9QSVr_hDorcNP8dXA@mail.gmail.com>
X-Mailer: Mew version 6.7 on Emacs 24.5 / Mule 6.0 (HANACHIRUSATO)
Mime-Version: 1.0
Content-Type: Text/Plain; charset="utf-8"
Content-Transfer-Encoding: base64
Archived-At: <https://mailarchive.ietf.org/arch/msg/yang-doctors/yTnxmVWY1lt8fhgmdVPuZZsmnKM>
Subject: Re: [yang-doctors] YD review and yang-push and friends
X-BeenThere: yang-doctors@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Email list of the yang-doctors directorate <yang-doctors.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/yang-doctors>, <mailto:yang-doctors-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/yang-doctors/>
List-Post: <mailto:yang-doctors@ietf.org>
List-Help: <mailto:yang-doctors-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/yang-doctors>, <mailto:yang-doctors-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 15 Mar 2018 07:33:44 -0000

Andy Bierman <andy@yumaworks.com> wrote:
> On Tue, Mar 13, 2018 at 3:29 PM, Kent Watsen <kwatsen@juniper.net> wrote:
> 
> >
> > Fine, but what about the examples that are contained within a draft that
> > defines the YANG module.  Do we expect YANG Doctors to review the examples
> > or not?   What I'm looking for is a definition of what all a YANG Doctor
> > looks at, if anything less than the entire draft.  Can the YANG Doctor
> > function be automated, or is function more than validators could ever hope
> > to do?
> >
> > K.
> >
> > =====
> >
> > As YANG secretary I have an issue with reviewing draft which do not
> > include YANG modules.
> >
> > > A YANG module has its review criteria defined in YANG RFCs.
> > > However examples may be manifold and imperfect.
> >
> > If the group decides to review such documents the review criteria needs to
> > be defined first.
> >
> >
> 
> IMO the scope needs to be fairly tight, and needs to focus on the aspects
> that
> no tool could ever hope to automate:
> 
>   Review YANG module(s) from 3 POVs
>       A) standard POV -- consistent with all related standards; reusing
> existing YANG correctly?
>       B) server POV -- it is clear to server implementors what to do
>       C) client POV -- it is clear to client developers what to do, and
> what a server is expected to do
> 
> Since normative text is spread all over the place, determining if (2) is
> correct
> can be a massive undertaking.
> 
> So I will focus only only these aspects during my reviews.
> If you want to check if the indentation is exactly right or
> every single reference is fully named, then an automated tool should do
> that.

I agree.  But there are two issues right now: (i) who checks that
these tools have been run or who runs the tools; and (ii) such tools
currently do not do a 100% good job.

Until we have these tools 100% automated, I think it is part of the YD
review to check these things.

For example, to check indentation of YANG itself you can do:

   $ pyang -f yang --keep-comments <IN> > <OUT>
   $ diff <IN> <OUT>

... and manually check the result (it won't be empty).

The reference checker "should" be built into idnits, or similar.



/martin



> 
> 
> Cheers,
> > Mehmet
> >
> 
> Andy
> 
> 
> >
> > > -----Original Message-----
> > > From: Martin Bjorklund <mbj@tail-f.com>
> > > Sent: Tuesday, March 13, 2018 9:22 AM
> > > To: bclaise@cisco.com
> > > Cc: kwatsen@juniper.net; mersue@gmail.com; yang-doctors@ietf.org
> > > Subject: Re: [yang-doctors] YD review and yang-push and friends
> > >
> > > Benoit Claise <bclaise@cisco.com> wrote:
> > > > Why not review the document, even if there is no YANG module, and see
> > > > if there is something to pay attention to? The examples, for example,
> > > > are important to review and validate.
> > >
> > > Yes, but is this something for the YANG doctors in general?
> > >
> > > In this particular case, it doesn't really matter, since most likely
> > several YDs
> > > will review the document anyway.
> > >
> > >
> > > /martin
> > >
> > > >
> > > > Regards, B.
> > > >
> > > >
> > > > > Now that the YD page has been restored, here's what it says:
> > > > >
> > > > > """
> > > > >
> > > > > What to look for during a review
> > > > >
> > > > > The most important item is to give the AD a sense of how important
> > > > > it is that they pay attention to the document.
> > > > > For YANG reviews the YANG Doctors will apply the RFC6087bis document
> > > > > on the Guidelines for Authors and Reviewers of YANG Data Model
> > > > > Documents
> > > > > ​https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__datatracker.ietf.org_doc_draft-2Dietf-2Dnetmod-
> > 2Drfc6087bis_&d=DwIFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=
> > 9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=9NbPiPD-CVHGtQCZXZJQf-
> > eRBVyomnwn2DvqhWonaBc&s=a93P1wWzQy2YYDl9KQTai1EbHdIRYwH_EwYv-TcUYjU&e=.
> > The
> > > > > YANG language syntax and semantics should be analyzed. The
> > > > > compliance with ​Network Management Datastore Architecture should to
> > > > > be ensured (see also ​NMDA guidelines).
> > > > >
> > > > > Review Information
> > > > >
> > > > > Under some circumstances, the YANG doctors might discover open
> > > > > issues or provide feedback worth documenting for the larger
> > > > > community. While the NETMOD WG still work on RFC6087bis, updating
> > > > > this document is preferred. If the topic is not appropriate for the
> > > > > RFC6087bis or if RFC6087bis has already been published, then this
> > > > > must be documented on the YANG questions/answers WIKI
> > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__trac.
> > ietf.org_trac_ops_wiki_YANGDoctorsFAQ&d=DwIFaQ&c=
> > HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=
> > 9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=9NbPiPD-CVHGtQCZXZJQf-
> > eRBVyomnwn2DvqhWonaBc&s=s7Yh35PaMXizxNvkc0_aLaDV1FDcJgoj2_IuPiEoRlc&e=.
> > > > >
> > > > > """
> > > > >
> > > > > The scope of the YD's review is unclear.
> > > > >
> > > > > K.
> > > > >
> > > > >
> > > > > ===== original message =====
> > > > >
> > > > > One question coming up in my mind is against which criteria should
> > > > > such drafts be reviewed.
> > > > > A YANG module has its review criteria defined in YANG RFCs.
> > > > > However examples may be manifold and imperfect.
> > > > >
> > > > > Cheers,
> > > > > Mehmet
> > > > >
> > > > >> -----Original Message-----
> > > > >> From: Kent Watsen <kwatsen@juniper.net>
> > > > >> Sent: Thursday, March 8, 2018 6:09 PM
> > > > >> To: Mehmet Ersue <mersue@gmail.com>; 'Martin Bjorklund'
> > > <mbj@tail-
> > > > >> f.com>
> > > > >> Cc: yang-doctors@ietf.org
> > > > >> Subject: Re: [yang-doctors] YD review and yang-push and friends
> > > > >>
> > > > >>
> > > > >>
> > > > >>> I did not start review for netconf-event-notifications-08.
> > > > >>>
> > > > >>> Netconf co-chairs: Please clarify whether a review is required.
> > > > >>
> > > > >> What's in a YANG Doctor review?  Is it just syntax, or semantics
> > too?
> > > > >> If it includes semantics, then does that then entail needing to
> > > > >> read the draft text as well, to determine if the YANG module
> > > > >> expresses the correct semantics or find that the draft text is
> > > > >> wrong?  Would it also extend to reviewing the examples in the
> > > > >> draft, to further ensure that the semantics are understood
> > > > >> correctly or, possibly, that there is an error in the example?
> > > > >>
> > > > >> Yes, I am aware that netconf-event-notifications does not define a
> > > > >> YANG module, but it does have examples that for the YANG modules in
> > > > >> the
> > > > >> yang-
> > > > >> push and subscriber-notifications drafts.  In that sense, I'm
> > > > >> wondering if they need to be reviewed, or do we expect the YD
> > > > >> reviewers of those other two drafts to look at this draft already?
> > > > >>
> > > > >> FWIW, I not talking about what might be found via validation.  I've
> > > > >> already asked the authors to post a script that validates the 14
> > > > >> examples in this draft...
> > > > >>
> > > > >>
> > > > >> K.
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > yang-doctors mailing list
> > > > > yang-doctors@ietf.org
> > > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.
> > ietf.org_mailman_listinfo_yang-2Ddoctors&d=DwIFaQ&c=
> > HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=
> > 9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=9NbPiPD-CVHGtQCZXZJQf-
> > eRBVyomnwn2DvqhWonaBc&s=3pky8v7zSfdi3HNvorvvT3Y60l7ZxfBUm6K8ulTV3r8&e=
> > > >
> >
> >
> >
> > _______________________________________________
> > yang-doctors mailing list
> > yang-doctors@ietf.org
> > https://www.ietf.org/mailman/listinfo/yang-doctors
> >