Re: [yang-doctors] YD review and yang-push and friends

Andy Bierman <andy@yumaworks.com> Wed, 14 March 2018 19:52 UTC

Return-Path: <andy@yumaworks.com>
X-Original-To: yang-doctors@ietfa.amsl.com
Delivered-To: yang-doctors@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0AAB21200F1 for <yang-doctors@ietfa.amsl.com>; Wed, 14 Mar 2018 12:52:57 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.6
X-Spam-Level:
X-Spam-Status: No, score=-2.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=yumaworks-com.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Vj4j89O8Kida for <yang-doctors@ietfa.amsl.com>; Wed, 14 Mar 2018 12:52:53 -0700 (PDT)
Received: from mail-lf0-x234.google.com (mail-lf0-x234.google.com [IPv6:2a00:1450:4010:c07::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 065C0126DED for <yang-doctors@ietf.org>; Wed, 14 Mar 2018 12:52:53 -0700 (PDT)
Received: by mail-lf0-x234.google.com with SMTP id w16-v6so6603665lfc.13 for <yang-doctors@ietf.org>; Wed, 14 Mar 2018 12:52:52 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yumaworks-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=fsl2kjF+wrNU0c5MSBEYFymCzwXXWFWBkY0TZ/uiHRM=; b=XZ0ak2oFrIqI3zClqKX2Mstd8mu9BuX6iqnca4ZRibTPt0ewfVlxFXmdC4Q+vlB+AM FSTZP2naTSwy9qWZfgIEZd0uLMkTKPUCjDahO4du4EcJ5eJdAmFqhm4U7hdpFk2QBBdV 1OzZQbSN5Moqbk1TMhl6Ugy6Z3ptnKFCY2QWs/M8UdbS+FSKz6ZOKNZuEzjMGDL5CQX+ LaaEwprdAqNiXMzQPzetNnA1ckZTI7IxHOP2/QPTsmUx9cmVdLHpNhmp6R/wbMWXOhM4 ETJ/ae078Kme/juwIWFloVSktNwBHvmw+7xTWKV9eymG3w/PGdMHgf3RyMzkWB1W/17G ENjA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=fsl2kjF+wrNU0c5MSBEYFymCzwXXWFWBkY0TZ/uiHRM=; b=F/R+O/YBcYHw2BGhvjKyqqp32RNCYPcDGt2cDHR13afPwUCN/mSGDRSqfx5VpZCMM6 inLmuUqJL3Ek3W4MyZSJf0TnIsNMgGykPhf367Wy6vJP45rT4S+OfyCRBbWyucO5Oowp KStCriv3j0TP8rmKaSFTXAUAXUI65eqIm9gCe0ta1nQmrNjPELzZ4xWJzZ7uaxB9LOzC ZDsvi+kkd1kZ7h1GbrSUnyO6lfRvT/KH4ks4yKpoaqcjhmzCoPcL1gCb6dRs2vE2yddB hKrNdWpIeIX1vM1PTWtDhjeBaggYUyDkHJ+oQHFlXKSFUhUQYzfempsz13ilpAIR0liZ 2j3A==
X-Gm-Message-State: AElRT7FOaDe1lmLb1g5llaOCiuyNYMGlsVbdq9KBb1ewLXcoiR0w1XoS 7ovwE4+pdWPnLlOLaXhPPmcUh2B1tAWk1Ggmdc2WxQ==
X-Google-Smtp-Source: AG47ELu1WupYALy2sL5aZgvBvHK8yEg/AE3CKJ6BQbAaDaE26dZnifWjFV/FuqAZeRsIhjPqomUCZh/KGK+x0aiVXYs=
X-Received: by 10.46.86.1 with SMTP id k1mr4119894ljb.28.1521057171225; Wed, 14 Mar 2018 12:52:51 -0700 (PDT)
MIME-Version: 1.0
Received: by 2002:a19:1a95:0:0:0:0:0 with HTTP; Wed, 14 Mar 2018 12:52:50 -0700 (PDT)
In-Reply-To: <A98F0B76-B46F-4FFD-8543-2AE619CB3812@juniper.net>
References: <045201d3b7d7$62c2c6a0$284853e0$@gmail.com> <25C4AC06-F4CB-4303-B19C-CA7BBFEBACD8@juniper.net> <81dbd2c7-2962-12f0-0f58-81a5fbac648f@cisco.com> <20180313.092218.1630545026391500372.mbj@tail-f.com> <007301d3badd$7b300640$719012c0$@gmail.com> <A98F0B76-B46F-4FFD-8543-2AE619CB3812@juniper.net>
From: Andy Bierman <andy@yumaworks.com>
Date: Wed, 14 Mar 2018 12:52:50 -0700
Message-ID: <CABCOCHTvsHtXrKUaea9OsnMgLK_xBFoDZ9QSVr_hDorcNP8dXA@mail.gmail.com>
To: Kent Watsen <kwatsen@juniper.net>
Cc: Mehmet Ersue <mersue@gmail.com>, Martin Bjorklund <mbj@tail-f.com>, "bclaise@cisco.com" <bclaise@cisco.com>, "yang-doctors@ietf.org" <yang-doctors@ietf.org>
Content-Type: multipart/alternative; boundary="94eb2c1cde8c60035e056764b8ff"
Archived-At: <https://mailarchive.ietf.org/arch/msg/yang-doctors/GQc0M5ur9FqPoS_hRbkEqsCBLpA>
Subject: Re: [yang-doctors] YD review and yang-push and friends
X-BeenThere: yang-doctors@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Email list of the yang-doctors directorate <yang-doctors.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/yang-doctors>, <mailto:yang-doctors-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/yang-doctors/>
List-Post: <mailto:yang-doctors@ietf.org>
List-Help: <mailto:yang-doctors-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/yang-doctors>, <mailto:yang-doctors-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 14 Mar 2018 19:52:57 -0000

On Tue, Mar 13, 2018 at 3:29 PM, Kent Watsen <kwatsen@juniper.net> wrote:

>
> Fine, but what about the examples that are contained within a draft that
> defines the YANG module.  Do we expect YANG Doctors to review the examples
> or not?   What I'm looking for is a definition of what all a YANG Doctor
> looks at, if anything less than the entire draft.  Can the YANG Doctor
> function be automated, or is function more than validators could ever hope
> to do?
>
> K.
>
> =====
>
> As YANG secretary I have an issue with reviewing draft which do not
> include YANG modules.
>
> > A YANG module has its review criteria defined in YANG RFCs.
> > However examples may be manifold and imperfect.
>
> If the group decides to review such documents the review criteria needs to
> be defined first.
>
>

IMO the scope needs to be fairly tight, and needs to focus on the aspects
that
no tool could ever hope to automate:

  Review YANG module(s) from 3 POVs
      A) standard POV -- consistent with all related standards; reusing
existing YANG correctly?
      B) server POV -- it is clear to server implementors what to do
      C) client POV -- it is clear to client developers what to do, and
what a server is expected to do

Since normative text is spread all over the place, determining if (2) is
correct
can be a massive undertaking.

So I will focus only only these aspects during my reviews.
If you want to check if the indentation is exactly right or
every single reference is fully named, then an automated tool should do
that.


Cheers,
> Mehmet
>

Andy


>
> > -----Original Message-----
> > From: Martin Bjorklund <mbj@tail-f.com>
> > Sent: Tuesday, March 13, 2018 9:22 AM
> > To: bclaise@cisco.com
> > Cc: kwatsen@juniper.net; mersue@gmail.com; yang-doctors@ietf.org
> > Subject: Re: [yang-doctors] YD review and yang-push and friends
> >
> > Benoit Claise <bclaise@cisco.com> wrote:
> > > Why not review the document, even if there is no YANG module, and see
> > > if there is something to pay attention to? The examples, for example,
> > > are important to review and validate.
> >
> > Yes, but is this something for the YANG doctors in general?
> >
> > In this particular case, it doesn't really matter, since most likely
> several YDs
> > will review the document anyway.
> >
> >
> > /martin
> >
> > >
> > > Regards, B.
> > >
> > >
> > > > Now that the YD page has been restored, here's what it says:
> > > >
> > > > """
> > > >
> > > > What to look for during a review
> > > >
> > > > The most important item is to give the AD a sense of how important
> > > > it is that they pay attention to the document.
> > > > For YANG reviews the YANG Doctors will apply the RFC6087bis document
> > > > on the Guidelines for Authors and Reviewers of YANG Data Model
> > > > Documents
> > > > ​https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__datatracker.ietf.org_doc_draft-2Dietf-2Dnetmod-
> 2Drfc6087bis_&d=DwIFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=
> 9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=9NbPiPD-CVHGtQCZXZJQf-
> eRBVyomnwn2DvqhWonaBc&s=a93P1wWzQy2YYDl9KQTai1EbHdIRYwH_EwYv-TcUYjU&e=.
> The
> > > > YANG language syntax and semantics should be analyzed. The
> > > > compliance with ​Network Management Datastore Architecture should to
> > > > be ensured (see also ​NMDA guidelines).
> > > >
> > > > Review Information
> > > >
> > > > Under some circumstances, the YANG doctors might discover open
> > > > issues or provide feedback worth documenting for the larger
> > > > community. While the NETMOD WG still work on RFC6087bis, updating
> > > > this document is preferred. If the topic is not appropriate for the
> > > > RFC6087bis or if RFC6087bis has already been published, then this
> > > > must be documented on the YANG questions/answers WIKI
> > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__trac.
> ietf.org_trac_ops_wiki_YANGDoctorsFAQ&d=DwIFaQ&c=
> HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=
> 9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=9NbPiPD-CVHGtQCZXZJQf-
> eRBVyomnwn2DvqhWonaBc&s=s7Yh35PaMXizxNvkc0_aLaDV1FDcJgoj2_IuPiEoRlc&e=.
> > > >
> > > > """
> > > >
> > > > The scope of the YD's review is unclear.
> > > >
> > > > K.
> > > >
> > > >
> > > > ===== original message =====
> > > >
> > > > One question coming up in my mind is against which criteria should
> > > > such drafts be reviewed.
> > > > A YANG module has its review criteria defined in YANG RFCs.
> > > > However examples may be manifold and imperfect.
> > > >
> > > > Cheers,
> > > > Mehmet
> > > >
> > > >> -----Original Message-----
> > > >> From: Kent Watsen <kwatsen@juniper.net>
> > > >> Sent: Thursday, March 8, 2018 6:09 PM
> > > >> To: Mehmet Ersue <mersue@gmail.com>om>; 'Martin Bjorklund'
> > <mbj@tail-
> > > >> f.com>
> > > >> Cc: yang-doctors@ietf.org
> > > >> Subject: Re: [yang-doctors] YD review and yang-push and friends
> > > >>
> > > >>
> > > >>
> > > >>> I did not start review for netconf-event-notifications-08.
> > > >>>
> > > >>> Netconf co-chairs: Please clarify whether a review is required.
> > > >>
> > > >> What's in a YANG Doctor review?  Is it just syntax, or semantics
> too?
> > > >> If it includes semantics, then does that then entail needing to
> > > >> read the draft text as well, to determine if the YANG module
> > > >> expresses the correct semantics or find that the draft text is
> > > >> wrong?  Would it also extend to reviewing the examples in the
> > > >> draft, to further ensure that the semantics are understood
> > > >> correctly or, possibly, that there is an error in the example?
> > > >>
> > > >> Yes, I am aware that netconf-event-notifications does not define a
> > > >> YANG module, but it does have examples that for the YANG modules in
> > > >> the
> > > >> yang-
> > > >> push and subscriber-notifications drafts.  In that sense, I'm
> > > >> wondering if they need to be reviewed, or do we expect the YD
> > > >> reviewers of those other two drafts to look at this draft already?
> > > >>
> > > >> FWIW, I not talking about what might be found via validation.  I've
> > > >> already asked the authors to post a script that validates the 14
> > > >> examples in this draft...
> > > >>
> > > >>
> > > >> K.
> > > >>
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > yang-doctors mailing list
> > > > yang-doctors@ietf.org
> > > > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.
> ietf.org_mailman_listinfo_yang-2Ddoctors&d=DwIFaQ&c=
> HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=
> 9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=9NbPiPD-CVHGtQCZXZJQf-
> eRBVyomnwn2DvqhWonaBc&s=3pky8v7zSfdi3HNvorvvT3Y60l7ZxfBUm6K8ulTV3r8&e=
> > >
>
>
>
> _______________________________________________
> yang-doctors mailing list
> yang-doctors@ietf.org
> https://www.ietf.org/mailman/listinfo/yang-doctors
>