Re: [yang-doctors] YD review and yang-push and friends

[Benoit Claise <[e-mail]>]

Dear all,
> Andy Bierman <[e-mail]> wrote:
>> On Tue, Mar 13, 2018 at 3:29 PM, Kent Watsen <[e-mail]> wrote:
>>
>>> Fine, but what about the examples that are contained within a draft that
>>> defines the YANG module.  Do we expect YANG Doctors to review the examples
>>> or not?   What I'm looking for is a definition of what all a YANG Doctor
>>> looks at, if anything less than the entire draft.  Can the YANG Doctor
>>> function be automated, or is function more than validators could ever hope
>>> to do?
>>>
>>> K.
>>>
>>> =====
>>>
>>> As YANG secretary I have an issue with reviewing draft which do not
>>> include YANG modules.
>>>
>>>> A YANG module has its review criteria defined in YANG RFCs.
>>>> However examples may be manifold and imperfect.
>>> If the group decides to review such documents the review criteria needs to
>>> be defined first.
>>>
>>>
>> IMO the scope needs to be fairly tight, and needs to focus on the aspects
>> that
>> no tool could ever hope to automate:
>>
>>    Review YANG module(s) from 3 POVs
>>        A) standard POV -- consistent with all related standards; reusing
>> existing YANG correctly?
>>        B) server POV -- it is clear to server implementors what to do
>>        C) client POV -- it is clear to client developers what to do, and
>> what a server is expected to do
>>
>> Since normative text is spread all over the place, determining if (2) is
>> correct
>> can be a massive undertaking.
>>
>> So I will focus only only these aspects during my reviews.
>> If you want to check if the indentation is exactly right or
>> every single reference is fully named, then an automated tool should do
>> that.
> I agree.  But there are two issues right now: (i) who checks that
> these tools have been run or who runs the tools; and (ii) such tools
> currently do not do a 100% good job.
>
> Until we have these tools 100% automated, I think it is part of the YD
> review to check these things.
>
> For example, to check indentation of YANG itself you can do:
>
>     $ pyang -f yang --keep-comments <IN> > <OUT>
>     $ diff <IN> <OUT>
>
> ... and manually check the result (it won't be empty).
>
> The reference checker "should" be built into idnits, or similar.
We all agree that everything that could be automated should be automated.
In the end, it boils down to IETF hackathon and community members 
investing their time. Looking at the IETF hackathon participants this 
coming week-end ... I guess that Joe Clarke, Vladimir Vassilev, and I 
will be lonely at our YANG tools development table.

I was able to get some Cisco funding for one developer/9 months for the 
yangcatalog.org and related integration in the datatracker. That money 
stream dried out.

Next step: I hope for some ISOC money for the next phase of development 
on the yangcatalog.org and related tools. I've been trying for 3 months 
but the process is very slow.
In the mean time, see you guys at the hackathon?

Regards, Benoit
>
>
>
> /martin
>
>
>
>>
>> Cheers,
>>> Mehmet
>>>
>> Andy
>>
>>
>>>> -----Original Message-----
>>>> From: Martin Bjorklund <[e-mail]>
>>>> Sent: Tuesday, March 13, 2018 9:22 AM
>>>> To: [e-mail]
>>>> Cc: [e-mail]; [e-mail]; [e-mail]
>>>> Subject: Re: [yang-doctors] YD review and yang-push and friends
>>>>
>>>> Benoit Claise <[e-mail]> wrote:
>>>>> Why not review the document, even if there is no YANG module, and see
>>>>> if there is something to pay attention to? The examples, for example,
>>>>> are important to review and validate.
>>>> Yes, but is this something for the YANG doctors in general?
>>>>
>>>> In this particular case, it doesn't really matter, since most likely
>>> several YDs
>>>> will review the document anyway.
>>>>
>>>>
>>>> /martin
>>>>
>>>>> Regards, B.
>>>>>
>>>>>
>>>>>> Now that the YD page has been restored, here's what it says:
>>>>>>
>>>>>> """
>>>>>>
>>>>>> What to look for during a review
>>>>>>
>>>>>> The most important item is to give the AD a sense of how important
>>>>>> it is that they pay attention to the document.
>>>>>> For YANG reviews the YANG Doctors will apply the RFC6087bis document
>>>>>> on the Guidelines for Authors and Reviewers of YANG Data Model
>>>>>> Documents
>>>>>> ​https://urldefense.proofpoint.com/v2/url?u=https-
>>> 3A__datatracker.ietf.org_doc_draft-2Dietf-2Dnetmod-
>>> 2Drfc6087bis_&d=DwIFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=
>>> 9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=9NbPiPD-CVHGtQCZXZJQf-
>>> eRBVyomnwn2DvqhWonaBc&s=a93P1wWzQy2YYDl9KQTai1EbHdIRYwH_EwYv-TcUYjU&e=.
>>> The
>>>>>> YANG language syntax and semantics should be analyzed. The
>>>>>> compliance with ​Network Management Datastore Architecture should to
>>>>>> be ensured (see also ​NMDA guidelines).
>>>>>>
>>>>>> Review Information
>>>>>>
>>>>>> Under some circumstances, the YANG doctors might discover open
>>>>>> issues or provide feedback worth documenting for the larger
>>>>>> community. While the NETMOD WG still work on RFC6087bis, updating
>>>>>> this document is preferred. If the topic is not appropriate for the
>>>>>> RFC6087bis or if RFC6087bis has already been published, then this
>>>>>> must be documented on the YANG questions/answers WIKI
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__trac.
>>> ietf.org_trac_ops_wiki_YANGDoctorsFAQ&d=DwIFaQ&c=
>>> HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=
>>> 9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=9NbPiPD-CVHGtQCZXZJQf-
>>> eRBVyomnwn2DvqhWonaBc&s=s7Yh35PaMXizxNvkc0_aLaDV1FDcJgoj2_IuPiEoRlc&e=.
>>>>>> """
>>>>>>
>>>>>> The scope of the YD's review is unclear.
>>>>>>
>>>>>> K.
>>>>>>
>>>>>>
>>>>>> ===== original message =====
>>>>>>
>>>>>> One question coming up in my mind is against which criteria should
>>>>>> such drafts be reviewed.
>>>>>> A YANG module has its review criteria defined in YANG RFCs.
>>>>>> However examples may be manifold and imperfect.
>>>>>>
>>>>>> Cheers,
>>>>>> Mehmet
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Kent Watsen <[e-mail]>
>>>>>>> Sent: Thursday, March 8, 2018 6:09 PM
>>>>>>> To: Mehmet Ersue <[e-mail]>; 'Martin Bjorklund'
>>>> <mbj@tail-
>>>>>>> f.com>
>>>>>>> Cc: [e-mail]
>>>>>>> Subject: Re: [yang-doctors] YD review and yang-push and friends
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> I did not start review for netconf-event-notifications-08.
>>>>>>>>
>>>>>>>> Netconf co-chairs: Please clarify whether a review is required.
>>>>>>> What's in a YANG Doctor review?  Is it just syntax, or semantics
>>> too?
>>>>>>> If it includes semantics, then does that then entail needing to
>>>>>>> read the draft text as well, to determine if the YANG module
>>>>>>> expresses the correct semantics or find that the draft text is
>>>>>>> wrong?  Would it also extend to reviewing the examples in the
>>>>>>> draft, to further ensure that the semantics are understood
>>>>>>> correctly or, possibly, that there is an error in the example?
>>>>>>>
>>>>>>> Yes, I am aware that netconf-event-notifications does not define a
>>>>>>> YANG module, but it does have examples that for the YANG modules in
>>>>>>> the
>>>>>>> yang-
>>>>>>> push and subscriber-notifications drafts.  In that sense, I'm
>>>>>>> wondering if they need to be reviewed, or do we expect the YD
>>>>>>> reviewers of those other two drafts to look at this draft already?
>>>>>>>
>>>>>>> FWIW, I not talking about what might be found via validation.  I've
>>>>>>> already asked the authors to post a script that validates the 14
>>>>>>> examples in this draft...
>>>>>>>
>>>>>>>
>>>>>>> K.
>>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> yang-doctors mailing list
>>>>>> [e-mail]
>>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.
>>> ietf.org_mailman_listinfo_yang-2Ddoctors&d=DwIFaQ&c=
>>> HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=
>>> 9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=9NbPiPD-CVHGtQCZXZJQf-
>>> eRBVyomnwn2DvqhWonaBc&s=3pky8v7zSfdi3HNvorvvT3Y60l7ZxfBUm6K8ulTV3r8&e=
>>>
>>>
>>> _______________________________________________
>>> yang-doctors mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/yang-doctors
>>>