Re: [yang-doctors] automating yang doctor reviews

"Reshad Rahman (rrahman)" <rrahman@cisco.com> Thu, 26 April 2018 15:07 UTC

Return-Path: <rrahman@cisco.com>
X-Original-To: yang-doctors@ietfa.amsl.com
Delivered-To: yang-doctors@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1D4EA1275F4 for <yang-doctors@ietfa.amsl.com>; Thu, 26 Apr 2018 08:07:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -14.51
X-Spam-Level:
X-Spam-Status: No, score=-14.51 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_HI=-5, SPF_PASS=-0.001, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cisco.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id P30p6ov3fgUg for <yang-doctors@ietfa.amsl.com>; Thu, 26 Apr 2018 08:07:40 -0700 (PDT)
Received: from rcdn-iport-9.cisco.com (rcdn-iport-9.cisco.com [173.37.86.80]) (using TLSv1.2 with cipher DHE-RSA-SEED-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 50B93127241 for <yang-doctors@ietf.org>; Thu, 26 Apr 2018 08:07:40 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=6032; q=dns/txt; s=iport; t=1524755260; x=1525964860; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-id:content-transfer-encoding: mime-version; bh=LWOvScmaPu4jwWHSt0+KITS6ZbrQxVlS5/+OM6aKwo0=; b=OTl4Vu6wgMz0GBIRSMI86hbohqT6xryjeZmisag9pkpEvSaTAS8Cy2Zp uP8vXPfYX4fOR8AzBRlG8BTGhhDKiuoT+vb1nfroDQ6aUKRKnFauGVoiS wsoFWNnSG6gNSFyXXozZ7r/6EYEKI8OeGtnOotocXwwQ0qF3xMCRqM29b 4=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: =?us-ascii?q?A0AOAQBv6uFa/5JdJa1bGQEBAQEBAQE?= =?us-ascii?q?BAQEBAQcBAQEBAYNDYXooCoNhiAKMd4FTIYEPkxMUgWQLGAsJhEACGoIqITQ?= =?us-ascii?q?YAQIBAQEBAQECbBwMhSMBAQEDAQEhEToLEAIBCA4KAgImAgICJQsVEAIEAQ0?= =?us-ascii?q?FG4R0D6gBghyEWINtgkAFgQmHCIFUP4EPIwyCXIMRAQGBSBgXgmkwggQgAow?= =?us-ascii?q?Ni3wIAo5GjFWQFAIREwGBJAEcOCaBLHAVOyoBghiCIBeIWYU+b4EVjVMrgQE?= =?us-ascii?q?BgRcBAQ?=
X-IronPort-AV: E=Sophos;i="5.49,330,1520899200"; d="scan'208";a="379401745"
Received: from rcdn-core-10.cisco.com ([173.37.93.146]) by rcdn-iport-9.cisco.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 26 Apr 2018 15:07:39 +0000
Received: from XCH-RCD-005.cisco.com (xch-rcd-005.cisco.com [173.37.102.15]) by rcdn-core-10.cisco.com (8.14.5/8.14.5) with ESMTP id w3QF7d9H032415 (version=TLSv1/SSLv3 cipher=AES256-SHA bits=256 verify=FAIL); Thu, 26 Apr 2018 15:07:39 GMT
Received: from xch-rcd-005.cisco.com (173.37.102.15) by XCH-RCD-005.cisco.com (173.37.102.15) with Microsoft SMTP Server (TLS) id 15.0.1320.4; Thu, 26 Apr 2018 10:07:38 -0500
Received: from xch-rcd-005.cisco.com ([173.37.102.15]) by XCH-RCD-005.cisco.com ([173.37.102.15]) with mapi id 15.00.1320.000; Thu, 26 Apr 2018 10:07:38 -0500
From: "Reshad Rahman (rrahman)" <rrahman@cisco.com>
To: Kent Watsen <kwatsen@juniper.net>, Martin Bjorklund <mbj@tail-f.com>
CC: "yang-doctors@ietf.org" <yang-doctors@ietf.org>
Thread-Topic: [yang-doctors] automating yang doctor reviews
Thread-Index: AQHT11P33vrEHgU4zkSHD96AJV2846QQwsGAgABC0oCAAj0ogA==
Date: Thu, 26 Apr 2018 15:07:38 +0000
Message-ID: <86441E66-EA63-4896-9BE5-DBC16F5A9BAD@cisco.com>
References: <FF0CC106-9C22-4E09-8940-E759753D0AC0@juniper.net> <20180424.225703.1856993049395921058.mbj@tail-f.com> <B4E19045-7239-408E-87B5-11747BFD25B7@juniper.net>
In-Reply-To: <B4E19045-7239-408E-87B5-11747BFD25B7@juniper.net>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
user-agent: Microsoft-MacOutlook/10.b.0.180311
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [161.44.212.92]
Content-Type: text/plain; charset="utf-8"
Content-ID: <02F3F4BEF8D8CB4B9F962F0D5573A362@emea.cisco.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
Archived-At: <https://mailarchive.ietf.org/arch/msg/yang-doctors/WcgLE0KonlCgCGs_YAWHBbvOSyw>
Subject: Re: [yang-doctors] automating yang doctor reviews
X-BeenThere: yang-doctors@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Email list of the yang-doctors directorate <yang-doctors.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/yang-doctors>, <mailto:yang-doctors-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/yang-doctors/>
List-Post: <mailto:yang-doctors@ietf.org>
List-Help: <mailto:yang-doctors-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/yang-doctors>, <mailto:yang-doctors-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 26 Apr 2018 15:07:42 -0000

Hi,

I'm all for "encouraging" Makefiles which generate trees, validate examples which are in separate etc. On 2 YANG models I work on, Mahesh pushed for this and it made a big difference.

For things such as whether import statements contain a reference and whether the reference is correct, it'd be good if we could have a tool to do the verification. It is very tedious work and error-prone.

Regards,
Reshad.

On 2018-04-24, 8:56 PM, "yang-doctors on behalf of Kent Watsen" <yang-doctors-bounces@ietf.org on behalf of kwatsen@juniper.net>; wrote:

    Martin,
    
    Regarding the problem trying to be solved, the end of the Section 1 reads:
    
       While the solution presented in this document facilitates "doctor"
       reviews (MIB, YANG, etc.), experience shows that doctor reviews are
       often out of synch with the document submitted for publication, some
       times by several draft revisions.  Thusly, it is currently common
       practice for the draft's shepherd to make some attempt at verifying
       the correctness of artwork containing structured code.  And, of
       course, as the document progresses in the publication process, it 
       may be subsequently updated by IESG and/or RFC Editor reviews.  By
       enabling the verification [and tree-diagram generation] to be 
       automated, correctness can be more assured throughout the 
       publication process.
    
    That said, I'll grant you that there could be a bug in the author's
    specification of the command line (or script) to validate the 
    <sourcecode> element (e.g., <sourcecode validate="true"> to hack
    it), but perhaps the YANG Doctor could at least verify that much
    in their review, with the reasonable assumption that it will not
    be removed (or become out-of-date) later. 
    
    Kent
    
    ===== original message =====
    
    Hi Kent,
    
    While I appreciate the effort, I'm not sure this will actually make
    the YANG doctor's job any easier.  For example, re including the
    command to generate the tree diagram - if people knew how to do this
    properly, they could as easily write a Makefile (or similar) that
    always generate the diagram so that the tree diagram is always up to
    date.  But we know that this is not the case.  So why do we think that
    the addition of the 'gen' attribute will make people produce good
    drafts?
    
    Also, I'm not sure it helps to include specific commands like this --
    what it tells me as a YD is that the probability that the examples
    validate is somewhat higher than w/o this attribute.  It does not tell
    me that the examples are valid.
    
    And sometimes examples and tree diagrams are manually edited for
    formatting reason (e.g., replace chunks of noise with "...").   This
    is difficult to capture formally, and people will still have to
    validate examples.
    
    As have been pointed out, there is often dependencies to other
    modules; capturing all this seems a bit tricky.  And the question is
    what problem is actually solves.
    
    Another approach might be to encourage people to keep their draft work
    on github, have separate files for examples, use 'make' to generate
    tree diagrams and to validate examples.  This is less formal, but
    achieves the same goal.
    
    
    /martin
    
    
    
    Kent Watsen <kwatsen@juniper.net>; wrote:
    > 
    > Doctors,
    > 
    > Here's a stab at how we might automate the basic parts of a YANG
    > Doctor review, something I've mentioned wanting at the YD-lunch
    > meeting at the last two IETF meetings.
    > 
    > I'll be the first to say that this proposal has issues, but hopefully
    > it's in the ballpark, and we can finish it off together, assuming
    > there is interest in bringing it forward at all...
    > 
    > Note, I assume that this document will be AD-sponsored, just like RFC
    > 7991 was, hence why the draft name is what it is.
    > 
    > PS: I'm falling into a black hole, and may not reply to any responses
    > until early next week.
    > 
    > Kent
    >  
    > 
    
    
    _______________________________________________
    yang-doctors mailing list
    yang-doctors@ietf.org
    https://www.ietf.org/mailman/listinfo/yang-doctors