Re: [yang-doctors] YANG dialects?

[Ladislav Lhotka <[e-mail]>]

On Fri, 2019-12-20 at 10:27 -0800, Andy Bierman wrote:
> 
> 
> On Fri, Dec 20, 2019 at 6:44 AM Ladislav Lhotka <[e-mail]> wrote:
> > On Fri, 2019-12-20 at 11:07 +0100, Jan Lindblad wrote:
> > > Lada,
> > > 
> > > > > > > o  extension: An extension attaches non-YANG semantics to
> > statements.
> > > > > > > 
> > > > > > > This is not the case here, it is only syntactically compatible. If
> > > > > > > tool A
> > > > > > > says that some instance data is valid, but tool B says it isn't,
> > then
> > > > > > > the
> > > > > > > standard is gone.
> > > > > > > 
> > > > > > > Lada
> > > > > 
> > > > > Of course it would be legal. It's always legal for a NETCONF server to
> > > > > reject
> > > > > an edit-config for no apparent reason (from a YANG perspective), so
> > all
> > > > > device
> > > > > YANG models would still be legal if all pattern statements were
> > removed,
> > > > > but
> > > > > the same string pattern constraints still enforced by the devices. The
> > OC
> > > > > group could therefore remove their pattern statements and introduce a
> > YANG
> > > > > extension oc:posix-pattern (or oc:go-pattern, or whatever) that gives
> > a
> > > > > hint
> > > > > to some clients what is expected. The clients that don't understand
> > this
> > > > > extension would just se a rejection for no apparent reason (from a
> > YANG
> > > > > perspective).
> > > > 
> > > > I strongly disagree. Why then do we have so much text about constraints,
> > > > validity etc. in RFC 7950? The benefit of schema validation is that an
> > > > implementation can rely on an external validator and needn't clutter the
> > > > code
> > > > with all kinds of input data checks. But if the validation result
> > depends on
> > > > whether I choose library A or B, this concept simply falls apart.
> > > > 
> > > > Lada
> > > 
> > > I completely agree that the device interface behavior should be exposed
> > with
> > > much detail in the corresponding YANG.
> > > 
> > > Practically, however, I don't think it is reasonable to expect that every
> > > little rule of the implementation is exposed. YANG isn't expressive enough
> > to
> > > cover everything. And actually, there are sometimes cases where I advise
> > not
> > > to add as many constraints into the model as technically possible by the
> > YANG
> > > language. Half-a-page long XPath expressions may detract sharply from
> > > readability, performance and universality. Implementation details may
> > start to
> > > leak into the interface.
> > 
> > I understand this but I guess the problem is opposite: A server developer
> > uses a
> > YANG library for validating config before applying it. If an implemented
> > module
> > contained say
> > 
> >     oc:pattern '^x[123]$';
> > 
> > the developer might think that this constraint is checked by the library and
> > take it for granted. But if the library in fact silently ignores the oc:
> > extension, then arbitrary strings will be accepted and can do all kinds of
> > nasty
> > things.
> > 
> 
> 
> But this is not the openconfig solution.

I know, but if you look back in this thread, Martin stated (and Jan later
supported) that the above "oc:pattern" extension would be legal, and I have been
arguing with that. Of course, I do agree that it would still be much better than
the existing situation.

Lada

> The oc-ext:regext-posix extension is much worse that this oc:pattern (does it
> exist?)
> It can be ignored by the compiler, which of course is unaware that pattern-
> stmt
> is not the YANG-defined pattern syntax.  They have already added this
> module-level extension to all modules, so problem solved (they think).
> 
> I do not agree that the pattern-stmt is somehow "close to the implementation"
> and therefore one cannot expect it to work on all servers the same way.
> All YANG constraints are supposed to produce the same client results, even
> if they are not implemented the same way on each server.
> 
> There is a proper way to use YANG extensions to annotate the model
> with implementation details.   Writing YANG modules so they only
> work for 1 organization or only work if the server is written in a particular
> language
> is not the proper way.
> 
> There is no way to fix this correctly in YANG 1.1.
> A new language version with new statements is needed to force all tools
> to support any new pattern syntax.
> 
> Andy
> 
> > > 
> > > There are also operational aspects to consider, such as a device running
> > out
> > > of memory at some point. A client therefore has to be prepared that an
> > edit-
> > > config may be refused for no apparent reason. There is no text in the RFC
> > > stating that the YANG constraints in a module must be complete, even if I
> > > regard that as a goal. One reason UML models tend to be useless as
> > interface
> > > descriptions in practice is that they lack these constraints.
> > > 
> > > YANG extensions are a practical way to introduce validation concepts not
> > > feasible to express with YANG and XPath. Here are some examples:
> > > + Constraints on compatible colors in an optical YANG model (Not
> > expressive
> > > enough)
> > > + Constraints on configured paths being disjunct (Not computationally
> > > efficient)
> > > + Constraints on an interface being unused by all other services (Not
> > > universally reusable [layer violation] if service YANG depends on device
> > YANG)
> > 
> > All these constraints are related to a particular use case but oc:pattern
> > would
> > really affect YANG semantics. For example:
> > 
> > - if oc:pattern is used in a type definition, does it also apply to a
> > derived
> > type?
> > 
> > - according to sec. 8.1 in RFC 7950, this must be true in all data trees:
> > 'All
> > leaf data values MUST match the type constraints for the leaf, including
> > those
> > defined in the ... "pattern" properties.' But does this also apply to
> > oc:pattern?
> > 
> > In short: I believe that the notion of validity with respect to a YANG
> > schema
> > should not depend on extensions as they are defined in RFC 7950. That's why
> > I
> > proposed the critical/core extensions, because then YANG implementations can
> > learn that a modified YANG is assumed, and refuse to validate instances or
> > at
> > least warn the user that things may unexpectedly break.
> > 
> > Lada
> > 
> > > 
> > > In these cases it may not be possible or desirable to express the
> > constraints
> > > in YANG/XPath. It may be possible express the constraint using domain
> > specific
> > > extension keywords, which should be better than not expressing it at all.
> > > Clients that do not understand the added keywords would still work as long
> > as
> > > the configurations generated are valid. Clients with domain understanding,
> > > possibly including understanding of the extension keywords, have a better
> > > chance at generating valid configurations.
> > > 
> > > /jan
> > > 
-- 
Ladislav Lhotka
Head, CZ.NIC Labs
PGP Key ID: 0xB8F92B08A9F76C67