Re: [netmod] Pattern statements [was Re: Query about augmenting module from submodule in YANG 1.0]

["Ivory, William" <[e-mail]>]

Personally I hate regexps as anything other than the simplest tend to be too cryptic to understand easily (or to test).  At least from YANG 1.1 we have the ability to have 'exclude' patterns (IIRC), but it's still not ideal.

What I'd like to see instead is bitwise operators (&, |, << and >>) along with 'for' loops and possibly local variables allowed in the version of XPATH  that we use.  Would likely need to define YANG-specific functions to extend XPATH, but we already have some (current() etc) so that wouldn't be a problem.  We have quite a few cases where we have to resort to calling out to scripts to provide local validation that would not be necessary if we had such operators in XPATH, and would allow NETCONF clients to perform validation off-box.

It would also be fun to implement - I rather enjoyed implementing our XPATH parser - but will concede that's probably not a priority when determining feature sets for future versions (-:

William

-----Original Message-----
From: t.petch [mailto:[e-mail]] 
Sent: 23 August 2017 10:46
To: Acee Lindem (acee) <[e-mail]>; Ivory, William <[e-mail]>; [e-mail]; Andy Bierman <[e-mail]>; Robert Wilton <[e-mail]>
Subject: Re: [netmod] Pattern statements [was Re: Query about augmenting module from submodule in YANG 1.0]

----- Original Message -----
From: "Robert Wilton" <[e-mail]>
Sent: Monday, August 21, 2017 4:14 PM

> That makes sense.
>
> The other thing that I think that we have got wrong is modelling regex 
> pattern statements.  I think that it would be much better if these
were
> written to be less exhaustive and much simpler.
>
> E.g. the "route distinguisher" pattern in
> draft-ietf-rtgwg-routing-types-09 is defined as this:
>
>           pattern
>             '(0:(6553[0-5]|655[0-2][0-9]|65[0-4][0-9]{2}|'
>           +     '6[0-4][0-9]{3}|'
>           +     '[0-5]?[0-9]{0,3}[0-9]):(429496729[0-5]|'
>           +     '42949672[0-8][0-9]|'
>           +     '4294967[01][0-9]{2}|429496[0-6][0-9]{3}|'
>           +     '42949[0-5][0-9]{4}|'
>           +     '4294[0-8][0-9]{5}|429[0-3][0-9]{6}|'
>           +     '42[0-8][0-9]{7}|4[01][0-9]{8}|'
>           +     '[0-3]?[0-9]{0,8}[0-9]))|'
>           + '(1:((([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|'
>           +     '25[0-5])\.){3}([0-9]|[1-9][0-9]|'
>           +     '1[0-9]{2}|2[0-4][0-9]|25[0-5])):(6553[0-5]|'
>           +     '655[0-2][0-9]|'
>           +     '65[0-4][0-9]{2}|6[0-4][0-9]{3}|'
>           +     '[0-5]?[0-9]{0,3}[0-9]))|'
>           + '(2:(429496729[0-5]|42949672[0-8][0-9]|'
>           +     '4294967[01][0-9]{2}|'
>           +     '429496[0-6][0-9]{3}|42949[0-5][0-9]{4}|'
>           +     '4294[0-8][0-9]{5}|'
>           +     '429[0-3][0-9]{6}|42[0-8][0-9]{7}|4[01][0-9]{8}|'
>           +     '[0-3]?[0-9]{0,8}[0-9]):'
>           +     '(6553[0-5]|655[0-2][0-9]|65[0-4][0-9]{2}|'
>           +     '6[0-4][0-9]{3}|'
>           +     '[0-5]?[0-9]{0,3}[0-9]))|'
>           + '(6(:[a-fA-F0-9]{2}){6})|'
>           + '(([3-57-9a-fA-F]|[1-9a-fA-F][0-9a-fA-F]{1,3}):'
>           +     '[0-9a-fA-F]{1,12})';
>         }
>
> But I think that it would be much easier to read, and quite possibly 
> more performant to execute, if the pattern regex was written something 
> like the following:
>
>   pattern:
>      '(0:[0-9]{1,5}:[0-9]{1,10})|
>       (1:([0-9]{1,3}\.){4}:[0-9]{1,5})|
>       (2:[0-9]{1,10}:0:[0-9]{1,5})|
>       (6(:[a-fA-F0-9]{2}){6})';
>
> Of course, this would allow more invalid values, but most servers
would
> be expected to reject those when it converts them into an internal 
> binary format any way.
>
> What do you, and others, think?

Simplify!

Bear in mind that the regex for an IPv6 address was wrong for a long time in base YANG before anyone noticed - it was just too complex.

And ABNF learnt long ago that just because something could be expressed in code does not mean that it is a good idea to do so.  If a simple English statement replaces many lines of ABNF, then that is a good tradeoff.

Pragmatically I am not sure what the cutoff for complexity should be but it should be less than we have now.

Paradoxically, given the original thread, the time when large expressions may work ok is when they have a 'sub-module' like structure, when I can look at a group of lines in isolation and form a view of what it does then move on to the next group and so on, building up an overall picture piece by piece.

Tom Petch

> Thanks,
> Rob
>
>
> On 21/08/2017 15:01, Acee Lindem (acee) wrote:
> > Hi William, Rob, Andy,
> >
> > Given their limited usefulness and the detriments, perhaps we should 
> > discourage the creation of new submodules in RFC6087Bis.
> >
> > Thanks,
> > Acee
> >
> > On 8/21/17, 9:44 AM, "netmod on behalf of Ivory, William"
> > <[e-mail] on behalf of [e-mail]>
wrote:
> >
> >> Hi Rob,
> >>
> >> That would make it very hard to update existing 1.x YANG models to
use
> >> new features in YANG 2.x if they used submodules.  Maybe that's
something
> >> that no one would ever consider doing anyway, or maybe YANG 1.1
already
> >> has similar differences to 1.0?  I had (perhaps naively) assumed
that you
> >> could migrate a namespace / model from YANG 1.0 to 2.0?
> >>
> >> Regards,
> >>
> >> William
> >>
> >> -----Original Message-----
> >> From: netmod [mailto:[e-mail]] On Behalf Of Robert
Wilton
> >> Sent: 21 August 2017 11:24
> >> To: [e-mail]
> >> Subject: Re: [netmod] Query about augmenting module from submodule
in
> >> YANG 1.0
> >>
> >>
> >>
> >> On 09/08/2017 16:13, Juergen Schoenwaelder wrote:
> >>> On Wed, Aug 09, 2017 at 05:01:09PM +0200, Ladislav Lhotka wrote:
> >>>> I remember that in early stages of YANG there was some irrational 
> >>>> fear of introducing too many namespaces, and submodules may be a 
> >>>> consequence of it. As you write, submodules provide no benefits 
> >>>> whatsoever in terms of modularity, but the overhead in terms of 
> >>>> metadata, IANA registration etc. is pretty much the same as for 
> >>>> modules.
> >>> In case YANG 2.0 is ever done, I suggest someone files a proposal
to
> >>> remove submodules if the cost/benefit ratio is at odds. There is 
> >>> nothing wrong with removing stuff that has been found problematic.
> >> I agree.
> >>
> >> I've added
> >>
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_netmod-2
Dw
> >>
g_yang-2Dnext_issues_26&d=DwICAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=p8kyeK3u4ZYi
aQ
> >>
2ZPGqwkyXmQgBH6r5jpYiYWzhqJ48&m=l7c4IPL049A2bVVO14fyBMly211xU61xSHgPlAT7
ow
> >> I&s=-kR4fUtXArQy0RwWb32DpT1bP4X_cNqt2zJVoC0JiX8&e=
> >>
> >> Rob
> >>
> >>> The motivation for submodules was that organizations maintaining
large
> >>> modules with multiple people can do so without having to mess
around
> >>> with tools like m4 scripts to produce a single module from
'snippets'
> >>> and to avoid integration surprises. But perhaps using m4 scripts
and
> >>> decent version control systems (that can integrate and compile on
> >>> checkin) is indeed cheaper than having submodules part of the YANG 
> >>> language itself.
> >>>
> >>> /js
> >>>
> >> _______________________________________________
> >> netmod mailing list
> >> [e-mail]
> >>
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailma
n_
> >>
listinfo_netmod&d=DwICAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=p8kyeK3u4ZYiaQ2ZPGqw
ky
> >>
XmQgBH6r5jpYiYWzhqJ48&m=l7c4IPL049A2bVVO14fyBMly211xU61xSHgPlAT7owI&s=t7
vG
> >> IH8ABuAm00e-bkSowD9eawModGq0N2OkjANtpYI&e=
> >>
> >> _______________________________________________
> >> netmod mailing list
> >> [e-mail]
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_m
> >> ailman_listinfo_netmod&d=DwICaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=p8kyeK3u
> >> 4ZYiaQ2ZPGqwkyXmQgBH6r5jpYiYWzhqJ48&m=V3Xd09IeBbuwHtIPd4YIt9kQBsxSD
> >> K7zldFfmT12OO8&s=Tkq4GWOUmeznETu_UVxLpOom3UXEiI5I2mLe7tpDoZw&e=
>
>


------------------------------------------------------------------------
--------


> _______________________________________________
> netmod mailing list
> [e-mail]
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mail
> man_listinfo_netmod&d=DwICaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=p8kyeK3u4ZYiaQ
> 2ZPGqwkyXmQgBH6r5jpYiYWzhqJ48&m=V3Xd09IeBbuwHtIPd4YIt9kQBsxSDK7zldFfmT
> 12OO8&s=Tkq4GWOUmeznETu_UVxLpOom3UXEiI5I2mLe7tpDoZw&e=
>