Re: [netmod] yang versioning solution complexity and alternative approaches

["Joe Clarke (jclarke)" <[e-mail]>]

Thanks, Andy.  We know it's been a while, and we're trying to take care of all of these comments.  See below.

On 3/9/22 13:13, Andy Bierman wrote:


On Wed, Mar 9, 2022 at 2:16 AM Jürgen Schönwälder <[e-mail]<mailto:[e-mail]>> wrote:
Hi,

the YANG versioning solution appears to be complex.


strongly agree.

There are many new procedures (for using extensions) that are not likely to
be adopted outside of the IETF/

[JC] Some of the approaches being proposed in the versioning work are already being implemented (both by vendors and other SDOs).


There was a valid original problem statement related to release trains.
This is the first disconnect. The problem is related to software release trains.
Vendors do not release YANG modules. The release software that contains and
implements YANG modules.

[JC] While vendors don't produce "standalone" modules, many of the modules they do produce are tied to software releases.  That is, they reflect the feature implementations in those releases, some of which may change in NBC ways release over release.


Multiple release trains can lead to 2 problems (that are currently handled by the implementation)

1) most recent release date for a module may not be the latest in all software release trains
    This affects "import-without-revision" for a client that tries to use the most recent revision

2) the same [name, date] tuple may be used in multiple release trains.
   This affects the YANG library and <get-schema>.

The solution for (1) is problematic because extensions are used and a YANG 1.1 compliant
tool MAY ignore ANY extension. I support a better import-stmt, but this should be
done in a new YANG version so it is not optional to implement.

[JC] If a client that doesn't support/understand the extension attempts to import a module and finds a revision that would otherwise NOT satisfy the extension import, then a compilation failure may or may not occur (depending on the type of changes) but that would happen today without this extension.  The extension provides a better chance of locating the correct module or failing fast and deterministically when it cannot be found.


The solution for (2) is not workable (as pointed out, it could take 5 days to release a change),
and the different revision dates add complexity and do not work if import-by-revision is used.

[JC] We feel that import-by-revision is inherently broken as it is too rigid.  But the solution for this is to use a more flexible import.


A great deal of effort is spent identifying that an NBC change occurred.
IMO it is not useful at the module level.  It would be useful at the object level,
but at a significant module maintenance cost.

[JC] It is useful at the module level to provide an overarching hint that something potentially breaking has occurred (or not), which then leads to deeper analysis.  The cost of doing so at the module level is not much more than keeping revisions and their descriptions up-to-date.




- We introduce version numbers that smell a bit like SemVers but then
  are not really SemVer.



This is very important.
IMO it will do harm to interoperability to have 2 SemVer schemes in use.
The early drafts said the OpenConfig SemVer 2.0 would be used.

[JC] The YANG Semver scheme is fully compatible with SemVer 2.0 (well, the revision of it we link to in the draft).  We extended it because of the requirements to support the indication of branching.  However, if one wants to use it like SemVer 2.0 or Openconfig, that is fine.



- We have no solution to do meaningful things with these version
  numbers, it does not even seem to be possible to decide whether
  X.Y.Z derives from x.y.z or not. We still seem to believe that
  having compatibility constraints embedded in module imports are a
  workable solution even though we acknowledge future breaking
  changes.


We have no plans whatsoever to write code that uses the NBC extensions.
The module revisions have to be compared and the real changes have to
be evaluated against real use-cases and real applications.  Since the tooling
has to do all the work anyway, having a "this module changed" flag is not really useful.



- We push for a reasonably complex algorithm to calculate deltas of
  YANG module revisions to let users of modules to determine the real
  impact of module updates on their work.



The semver is trying to characterize changes to the YANG module and also
seems to try to identify the software release train containing the module.

I wonder why we not consider the opposite approach, namely to have
author making NBC changes to document them right where the changes
are. If authors would write something like

      leaf foo {
        nbc-change "2022-03-01" {
          description "changed type from int32 to string";
        };
        // ...
      }



This is more work but also more useful.

[JC] In some cases this is useful, yes.  As you point out, tthis approach adds additional work which could lead to inaction, and thus the consumer ends up where they are today with no clear idea that NBC changes may have occurred.  Whereas, the top-level module indicator is still useful to the consumer to signal NBC changes and lead them into tooling to investigate.

[JC] Where this approach is very useful and necessary is when semantic changes occur that tooling cannot find (e.g., a temperature leaf suddenly changes from degrees F to degrees C and there is no units property).


I am confused about the revision updates for work-in-progress.
IMO there SHOULD NOT be any revision tracking from I-D to I-D.
This would be total noise and way too much work.
Consider the client-server groupings, which had constant NBC changes
across 27 revisions of the drafts.

[JC] In such long-running drafts, it has been seen that vendors choose to implement draft revisions.  Having YANG modules so tagged makes it clear what revision they are implementing and that it is not yet a ratified standard.  In terms of work, one only has to update the revision-label when the module changes, and they would otherwise be updating the revision statement (or changes in an appendix).  Simply bumping this doesn't require a great deal of additional work.


I-Ds currently reuse the same revision-stmt over and over in an update I-D,
just changing the date.  Maintaining a semver label throughout this process
is a waste of time.

There is also no way to tell that a 1.x.x release is a work-in-progress.
It is clear for 0.x.x  only.

[JC] YANG Semver (and SemVer 2.0) provide a way to signal such that.  A 1.1.0-draft... signals a WIP by having the '-' there.  That character indicates a pre-release version.


There is no easy way to identify a release as "published" or "unpublished",
especially since YANG modules are extracted from the source document.

[JC] Released versions would not have the pre-release marker.





then tool and humans can easily figure out in which revision NBC
changes occured and if they affect a certain usage of the module.

Instead of simply properly documenting the changes, we invent fuzzy
version numbers and complex algorithms to reverse engineer the facts
that could have easily been documented by whoever makes the change.

Some vendors actually avoid making NBC changes to their APIs.
IMO this IETF work is trying to normalize bad engineering practice.
The past WG (people creating YANG 1.0 and YANG 1.1) felt quite strongly
that changing the type of a leaf was something that should not be allowed.

[JC] This work is recognizing that these behaviors occur and providing a means whereby they can be signaled.  This is part of the requirements.  By having these conventions it does not mean that one has to make NBC changes.  It's just a way to indicate them and make it easier for consumers to know when such changes occur.



If the reason is that developers do not document their changes, then
go and develop tools to force them to document their changes. I do not
think it is fair to simply push the pain to the users of YANG modules.



1 more comment:

Errata not integrated:

Most organizations publish corrected documents when mistakes are found, but not the IETF.
Instead the WEB page will have a link "Errata Exists".  It is up to the reader to notice it,
read all the errata, apply all the changes manually, and maintain their own private copy
of the RFC.

This is especially bad for extracted YANG modules.
For example there is no way to create a revision label (or date string) that
indicates that this is  [ietf-netconf-notifications, 2012-02-06] + Errata 3957.
Vendors just hack the incorrect RFC version of the module.
It would be quite useful to have corrected YANG modules when errors are found.
Even the YANG repos simply maintain the uncorrected versions.

[JC] Errata on YANG modules should necessitate a new version of that module.  Admittedly, this takes time in the IETF, but incorporating errata on the fly without having a means to signal a change from the RFC version would make it even harder on the consumer.

Joe