Re: [yang-doctors] List key model definition order and results w/ pyang/yanglint

[Ebben Aries <[e-mail]>]

inline...

On Jan 25 08:30 AM, Martin Bjorklund wrote:
> Hi,
> 
> Ebben Aries <[e-mail]> wrote:
> > Possibly bringing up a subject that may have been discussed before but
> > could not find anything on it so sending to the group.
> > 
> > RFC 6020 and 7950 both have the same wording on the subject around list
> > keys per
> > 
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc6020-23section-2D7.8.2&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=s0ueqUyec_0t88MK81g6lUGSHXLivei_aFPMFmdUATQ&e=
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc7950-23section-2D7.8.2&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=tnRmYIIjLA3KSnysiPYAaVjjLJ7GLK-No5AFu9bJ1hg&e=
> > 
> > From a model definition standpoint, there is nothing mandatory
> > surrounding that list keys be the first leaf definitions underneath.
> > 
> > From a content encoding standpoint, the following XML Mapping Rules in
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc6020-23section-2D7.8.5&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=El_GT294heczTOGSYxV-s0MdzjwhB-vEBEVCQNYEdzs&e= do dictate an order
> > that the keys be defined before other subelements.  JSON obviously does
> > not utilize the same encoding technique.
> > 
> >    The list's key nodes are encoded as subelements to the list's
> >    identifier element, in the same order as they are defined within the
> >    "key" statement.
> > 
> >    The rest of the list's child nodes are encoded as subelements to the
> >    list element, after the keys.  If the list defines RPC input or
> >    output parameters, the subelements are encoded in the same order as
> >    they are defined within the "list" statement.  Otherwise, the
> >    subelements are encoded in any order.
> > 
> > Given the following simple example:
> > 
> > module test {
> >     namespace "https://urldefense.proofpoint.com/v2/url?u=http-3A__test&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=DtGK_MVelzL9TNkronbZ5Jb77kpX3cG2aR0Sthda_Z4&e=";
> >     prefix test;
> > 
> >     container foo {
> >         leaf bar {
> >             type string;
> >         }
> >         list car {
> >             key "far mar";
> > 
> >             leaf par {
> >                 type string;
> >             }
> >             leaf far {
> >                 type string;
> >             }
> >             leaf mar {
> >                 type string;
> >             }
> >             leaf nar {
> >                 type string;
> >             }
> >         }
> >     }
> > }
> > 
> > leaf 'par' here is defined before the 'far' and 'mar' keys, however if
> > we render out a sample-xml-skeleton with pyang, we do not take into
> > account the XML encoding rules defined in Section 7.8.5 and merely keep
> > the order as defined in the model.
> > 
> > $ pyang test.yang --strict -f sample-xml-skeleton
> > <?xml version='1.0' encoding='UTF-8'?>
> > <data xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
> >   <foo xmlns="https://urldefense.proofpoint.com/v2/url?u=http-3A__test&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=DtGK_MVelzL9TNkronbZ5Jb77kpX3cG2aR0Sthda_Z4&e=">
> >     <bar />
> >     <car>
> >       <par />
> >       <far />
> >       <mar />
> >       <nar />
> >     </car>
> >   </foo>
> > </data>
> > 
> > With yanglint, given the following XML document
> > 
> > $ cat test.xml
> > <foo xmlns="https://urldefense.proofpoint.com/v2/url?u=http-3A__test&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=DtGK_MVelzL9TNkronbZ5Jb77kpX3cG2aR0Sthda_Z4&e=">
> >     <bar>BAR</bar>
> >     <car>
> >         <par>PAR</par>
> >         <far>FAR</far>
> >         <mar>MAR</mar>
> >         <nar>NAR</nar>
> >     </car>
> > </foo>
> > 
> > The rendered XML output will actually reorder the list keys to be first
> > 
> > $ yanglint -f xml test.yang test.xml
> > <foo xmlns="https://urldefense.proofpoint.com/v2/url?u=http-3A__test&d=DwICAg&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=GIehbDpQlo31lSi6WbnEkA&m=aIc-CQ8w2Asg5EZaRCunbRr33N3KJwrY_LQRZg_ED0A&s=DtGK_MVelzL9TNkronbZ5Jb77kpX3cG2aR0Sthda_Z4&e=">
> >   <bar>BAR</bar>
> >   <car>
> >     <far>FAR</far>
> >     <mar>MAR</mar>
> >     <par>PAR</par>
> >     <nar>NAR</nar>
> >   </car>
> > </foo>
> > 
> > Personally, from a BCP standpoint, defining list keys first eases
> > readability and want to solicit others thoughts on this (rather than
> > nested down in the list or via a grouping reference further down).
> 
> I agree.
> 

Should we add text to rfc6087bis to suggest such?  Any reason not to?

> > Generation to yin does not rearrange in either pyang or yanglint which
> > means any processing of yin would need to undergo some reordering
> > possibly depending on how it is absorbed.
> 
> I don't understand what you mean.  Yin follows the same rules as basic
> YANG syntax; it is the "key" statement that defines the XML element
> order.
> 

Just merely pointing out that the same handling that would be done for
XML encoding rules would need to be done for potential processing of YIN
if the consumer of said data needs to process ordered elements
in whatever application.

Understood this is lossless and a direct mapping

> > For the JSON encoding case, there may be no strict ordering but
> > following a BCP where folks *should* send list keys before other
> > elements can ease implementation efforts in many cases.
> 
> I disagree.  Implementations MUST handle object members in any order
> (std JSON), so they can never use a simpler implementation that is
> built on the presumption that keys always are sent first.
> 

Agreed

> (However, a clever implemention could check if the keys are sent
> first, then it can parse more efficiently; it wouldn't have to buffer
> data.  But this is the opposite of a simple implemention.)
> 

This was precisely my point but agree, the assumption can never be made
due to this

A BCP could very well be put into code for --ietf, -f yang (and other
flags) that *could* enforce this ordering within the model itself.  If
we do add text to rfc6087bis, this should come along w/ imo.

> > It is also
> > likely that payloads will generally be constructed in the order in which
> > they are defined in the model.
> 
> Yes.
> 
> > So either I see making the definition of such either mandatory or at a
> > bare minimum pyang sample-xml-skeleton output to take into account
> > ordering list key leafs prior to other subelements.
> 
> I think the bug in pyang should be fixed!
> 

Have the fix in a local branch - pull request now at:

https://github.com/mbj4668/pyang/pull/359

/ebben