Re: [yang-doctors] List key model definition order and results w/ pyang/yanglint

Martin Bjorklund <mbj@tail-f.com> Tue, 30 January 2018 15:09 UTC

Return-Path: <mbj@tail-f.com>
X-Original-To: yang-doctors@ietfa.amsl.com
Delivered-To: yang-doctors@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C8B4B12D96B for <yang-doctors@ietfa.amsl.com>; Tue, 30 Jan 2018 07:09:08 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.91
X-Spam-Level:
X-Spam-Status: No, score=-1.91 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NoBrfPqNmd3V for <yang-doctors@ietfa.amsl.com>; Tue, 30 Jan 2018 07:09:01 -0800 (PST)
Received: from mail.tail-f.com (mail.tail-f.com [46.21.102.45]) by ietfa.amsl.com (Postfix) with ESMTP id 56B5212D833 for <yang-doctors@ietf.org>; Tue, 30 Jan 2018 07:09:01 -0800 (PST)
Received: from localhost (unknown [173.38.220.56]) by mail.tail-f.com (Postfix) with ESMTPSA id 8713E1AE018A; Tue, 30 Jan 2018 16:09:00 +0100 (CET)
Date: Tue, 30 Jan 2018 16:09:00 +0100
Message-Id: <20180130.160900.1240244423754350192.mbj@tail-f.com>
To: exa@juniper.net
Cc: yang-doctors@ietf.org
From: Martin Bjorklund <mbj@tail-f.com>
In-Reply-To: <20180130011205.rr4aacdpo4tnovx2@smtp.juniper.net>
References: <20180125031650.jzdswdgpspontven@smtp.juniper.net> <20180125.083058.1067336366755903306.mbj@tail-f.com> <20180130011205.rr4aacdpo4tnovx2@smtp.juniper.net>
X-Mailer: Mew version 6.7 on Emacs 24.5 / Mule 6.0 (HANACHIRUSATO)
Mime-Version: 1.0
Content-Type: Text/Plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/yang-doctors/eIXYiPWJ_yPAOgEEKa0wp4UYlVM>
Subject: Re: [yang-doctors] List key model definition order and results w/ pyang/yanglint
X-BeenThere: yang-doctors@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Email list of the yang-doctors directorate <yang-doctors.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/yang-doctors>, <mailto:yang-doctors-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/yang-doctors/>
List-Post: <mailto:yang-doctors@ietf.org>
List-Help: <mailto:yang-doctors-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/yang-doctors>, <mailto:yang-doctors-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 30 Jan 2018 15:09:09 -0000

Hi,

Thank you for contributing with a pull request, I will have a look at
it!


/martin



Ebben Aries <exa@juniper.net> wrote:
> inline...
> 
> On Jan 25 08:30 AM, Martin Bjorklund wrote:
> > Hi,
> > 
> > Ebben Aries <exa@juniper.net> wrote:
> > > Possibly bringing up a subject that may have been discussed before but
> > > could not find anything on it so sending to the group.
> > > 
> > > RFC 6020 and 7950 both have the same wording on the subject around list
> > > keys per
> > > 
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc6020-23section-2D7.8.2&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=s0ueqUyec_0t88MK81g6lUGSHXLivei_aFPMFmdUATQ&e=
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc7950-23section-2D7.8.2&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=tnRmYIIjLA3KSnysiPYAaVjjLJ7GLK-No5AFu9bJ1hg&e=
> > > 
> > > From a model definition standpoint, there is nothing mandatory
> > > surrounding that list keys be the first leaf definitions underneath.
> > > 
> > > From a content encoding standpoint, the following XML Mapping Rules in
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc6020-23section-2D7.8.5&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=El_GT294heczTOGSYxV-s0MdzjwhB-vEBEVCQNYEdzs&e= do dictate an order
> > > that the keys be defined before other subelements.  JSON obviously does
> > > not utilize the same encoding technique.
> > > 
> > >    The list's key nodes are encoded as subelements to the list's
> > >    identifier element, in the same order as they are defined within the
> > >    "key" statement.
> > > 
> > >    The rest of the list's child nodes are encoded as subelements to the
> > >    list element, after the keys.  If the list defines RPC input or
> > >    output parameters, the subelements are encoded in the same order as
> > >    they are defined within the "list" statement.  Otherwise, the
> > >    subelements are encoded in any order.
> > > 
> > > Given the following simple example:
> > > 
> > > module test {
> > >     namespace "https://urldefense.proofpoint.com/v2/url?u=http-3A__test&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=DtGK_MVelzL9TNkronbZ5Jb77kpX3cG2aR0Sthda_Z4&e=";
> > >     prefix test;
> > > 
> > >     container foo {
> > >         leaf bar {
> > >             type string;
> > >         }
> > >         list car {
> > >             key "far mar";
> > > 
> > >             leaf par {
> > >                 type string;
> > >             }
> > >             leaf far {
> > >                 type string;
> > >             }
> > >             leaf mar {
> > >                 type string;
> > >             }
> > >             leaf nar {
> > >                 type string;
> > >             }
> > >         }
> > >     }
> > > }
> > > 
> > > leaf 'par' here is defined before the 'far' and 'mar' keys, however if
> > > we render out a sample-xml-skeleton with pyang, we do not take into
> > > account the XML encoding rules defined in Section 7.8.5 and merely keep
> > > the order as defined in the model.
> > > 
> > > $ pyang test.yang --strict -f sample-xml-skeleton
> > > <?xml version='1.0' encoding='UTF-8'?>
> > > <data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
> > >   <foo xmlns="https://urldefense.proofpoint.com/v2/url?u=http-3A__test&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=DtGK_MVelzL9TNkronbZ5Jb77kpX3cG2aR0Sthda_Z4&e=">
> > >     <bar />
> > >     <car>
> > >       <par />
> > >       <far />
> > >       <mar />
> > >       <nar />
> > >     </car>
> > >   </foo>
> > > </data>
> > > 
> > > With yanglint, given the following XML document
> > > 
> > > $ cat test.xml
> > > <foo xmlns="https://urldefense.proofpoint.com/v2/url?u=http-3A__test&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=DtGK_MVelzL9TNkronbZ5Jb77kpX3cG2aR0Sthda_Z4&e=">
> > >     <bar>BAR</bar>
> > >     <car>
> > >         <par>PAR</par>
> > >         <far>FAR</far>
> > >         <mar>MAR</mar>
> > >         <nar>NAR</nar>
> > >     </car>
> > > </foo>
> > > 
> > > The rendered XML output will actually reorder the list keys to be first
> > > 
> > > $ yanglint -f xml test.yang test.xml
> > > <foo xmlns="https://urldefense.proofpoint.com/v2/url?u=http-3A__test&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=DtGK_MVelzL9TNkronbZ5Jb77kpX3cG2aR0Sthda_Z4&e=">
> > >   <bar>BAR</bar>
> > >   <car>
> > >     <far>FAR</far>
> > >     <mar>MAR</mar>
> > >     <par>PAR</par>
> > >     <nar>NAR</nar>
> > >   </car>
> > > </foo>
> > > 
> > > Personally, from a BCP standpoint, defining list keys first eases
> > > readability and want to solicit others thoughts on this (rather than
> > > nested down in the list or via a grouping reference further down).
> > 
> > I agree.
> > 
> 
> Should we add text to rfc6087bis to suggest such?  Any reason not to?
> 
> > > Generation to yin does not rearrange in either pyang or yanglint which
> > > means any processing of yin would need to undergo some reordering
> > > possibly depending on how it is absorbed.
> > 
> > I don't understand what you mean.  Yin follows the same rules as basic
> > YANG syntax; it is the "key" statement that defines the XML element
> > order.
> > 
> 
> Just merely pointing out that the same handling that would be done for
> XML encoding rules would need to be done for potential processing of YIN
> if the consumer of said data needs to process ordered elements
> in whatever application.
> 
> Understood this is lossless and a direct mapping
> 
> > > For the JSON encoding case, there may be no strict ordering but
> > > following a BCP where folks *should* send list keys before other
> > > elements can ease implementation efforts in many cases.
> > 
> > I disagree.  Implementations MUST handle object members in any order
> > (std JSON), so they can never use a simpler implementation that is
> > built on the presumption that keys always are sent first.
> > 
> 
> Agreed
> 
> > (However, a clever implemention could check if the keys are sent
> > first, then it can parse more efficiently; it wouldn't have to buffer
> > data.  But this is the opposite of a simple implemention.)
> > 
> 
> This was precisely my point but agree, the assumption can never be made
> due to this
> 
> A BCP could very well be put into code for --ietf, -f yang (and other
> flags) that *could* enforce this ordering within the model itself.  If
> we do add text to rfc6087bis, this should come along w/ imo.
> 
> > > It is also
> > > likely that payloads will generally be constructed in the order in which
> > > they are defined in the model.
> > 
> > Yes.
> > 
> > > So either I see making the definition of such either mandatory or at a
> > > bare minimum pyang sample-xml-skeleton output to take into account
> > > ordering list key leafs prior to other subelements.
> > 
> > I think the bug in pyang should be fixed!
> > 
> 
> Have the fix in a local branch - pull request now at:
> 
> https://github.com/mbj4668/pyang/pull/359
> 
> /ebben
>