Re: [netmod] yang versioning solution complexity and alternative approaches

[Andy Bierman <[e-mail]>]

On Tue, Jun 7, 2022 at 7:22 AM Joe Clarke (jclarke) <[e-mail]>
wrote:

> Thanks, Andy.  We know it's been a while, and we're trying to take care of
> all of these comments.  See below.
>
> On 3/9/22 13:13, Andy Bierman wrote:
>
>
>
> On Wed, Mar 9, 2022 at 2:16 AM Jürgen Schönwälder <
> [e-mail]> wrote:
>
>> Hi,
>>
>> the YANG versioning solution appears to be complex.
>>
>>
> strongly agree.
>
> There are many new procedures (for using extensions) that are not likely to
> be adopted outside of the IETF/
>
>
> [JC] Some of the approaches being proposed in the versioning work are
> already being implemented (both by vendors and other SDOs).
>
>

I support the original plan to use SemVer 2.0 and the adoption of the
'openconfig-version' extension.
Since each extension is optional, vendors can choose which ones help or not.
It is not a big issue.



> There was a valid original problem statement related to release trains.
> This is the first disconnect. The problem is related to software release
> trains.
> Vendors do not release YANG modules. The release software that contains and
> implements YANG modules.
>
>
> [JC] While vendors don't produce "standalone" modules, many of the modules
> they do produce are tied to software releases.  That is, they reflect the
> feature implementations in those releases, some of which may change in NBC
> ways release over release.
>
>

I understand NBC changes occur.
I just have not seen any customer demand for automation tooling to "fix"
NBC changes.
There are too many complex variants and NBC changes are rare enough that
they
can be handled manually by developers.

I strongly agree with Juergen that only the NBC changes need to be
documented.
IMO this can already be done with description-stmt so extensions do not
really help.



> Multiple release trains can lead to 2 problems (that are currently handled
> by the implementation)
>
> 1) most recent release date for a module may not be the latest in all
> software release trains
>     This affects "import-without-revision" for a client that tries to use
> the most recent revision
>
> 2) the same [name, date] tuple may be used in multiple release trains.
>    This affects the YANG library and <get-schema>.
>
>
> The solution for (1) is problematic because extensions are used and a YANG
> 1.1 compliant
> tool MAY ignore ANY extension. I support a better import-stmt, but this
> should be
> done in a new YANG version so it is not optional to implement.
>
>
> [JC] If a client that doesn't support/understand the extension attempts to
> import a module and finds a revision that would otherwise NOT satisfy the
> extension import, then a compilation failure may or may not occur
> (depending on the type of changes) but that would happen today without this
> extension.  The extension provides a better chance of locating the correct
> module or failing fast and deterministically when it cannot be found.
>


I think the problem of a client picking the wrong release train for a
[name, date] tuple is not causing
a lot of real problems. The <get-schema> operation is widely supported, and
there are
implementation-specific solutions available (e.g., configured in client).



>
>
> The solution for (2) is not workable (as pointed out, it could take 5 days
> to release a change),
> and the different revision dates add complexity and do not work if
> import-by-revision is used.
>
>
> [JC] We feel that import-by-revision is inherently broken as it is too
> rigid.  But the solution for this is to use a more flexible import.
>
>

Industry practice has not been to assign a unique revision-date to every
release train
that is getting updated.  This is not workable. It should not even be
required as long
as each revision-label is unique.




> A great deal of effort is spent identifying that an NBC change occurred.
> IMO it is not useful at the module level.  It would be useful at the
> object level,
> but at a significant module maintenance cost.
>
>
> [JC] It is useful at the module level to provide an overarching hint that
> something potentially breaking has occurred (or not), which then leads to
> deeper analysis.  The cost of doing so at the module level is not much more
> than keeping revisions and their descriptions up-to-date.
>


If the revision-label algorithm is too complex then it becomes confusing.
I support the controlled increment of the major version (e.g. 1.x.x to
2.x.x )
because it is simple and well understood.



>
>
>
>
>> - We introduce version numbers that smell a bit like SemVers but then
>>   are not really SemVer.
>>
>>
>
> This is very important.
> IMO it will do harm to interoperability to have 2 SemVer schemes in use.
> The early drafts said the OpenConfig SemVer 2.0 would be used.
>
>
> [JC] The YANG Semver scheme is fully compatible with SemVer 2.0 (well, the
> revision of it we link to in the draft).  We extended it because of the
> requirements to support the indication of branching.  However, if one wants
> to use it like SemVer 2.0 or Openconfig, that is fine.
>


Agreed. The openconfig-version extension can be used instead of, or in
addition to any IETF extensions.



>
>
>
>
>> - We have no solution to do meaningful things with these version
>>   numbers, it does not even seem to be possible to decide whether
>>   X.Y.Z derives from x.y.z or not. We still seem to believe that
>>   having compatibility constraints embedded in module imports are a
>>   workable solution even though we acknowledge future breaking
>>   changes.
>>
>
>
> We have no plans whatsoever to write code that uses the NBC extensions.
> The module revisions have to be compared and the real changes have to
> be evaluated against real use-cases and real applications.  Since the
> tooling
> has to do all the work anyway, having a "this module changed" flag is not
> really useful.
>
>
>
>>
>> - We push for a reasonably complex algorithm to calculate deltas of
>>   YANG module revisions to let users of modules to determine the real
>>   impact of module updates on their work.
>>
>>
>
> The semver is trying to characterize changes to the YANG module and also
> seems to try to identify the software release train containing the module.
>
> I wonder why we not consider the opposite approach, namely to have
>> author making NBC changes to document them right where the changes
>> are. If authors would write something like
>>
>>       leaf foo {
>>         nbc-change "2022-03-01" {
>>           description "changed type from int32 to string";
>>         };
>>         // ...
>>       }
>>
>>
>
> This is more work but also more useful.
>
>
> [JC] In some cases this is useful, yes.  As you point out, tthis approach
> adds additional work which could lead to inaction, and thus the consumer
> ends up where they are today with no clear idea that NBC changes may have
> occurred.  Whereas, the top-level module indicator is still useful to the
> consumer to signal NBC changes and lead them into tooling to investigate.
>
>

I think the description-stmt in the leaf is sufficient and no new
statements are needed.


[JC] Where this approach is very useful and necessary is when semantic
> changes occur that tooling cannot find (e.g., a temperature leaf suddenly
> changes from degrees F to degrees C and there is no units property).
>
>
> I am confused about the revision updates for work-in-progress.
> IMO there SHOULD NOT be any revision tracking from I-D to I-D.
> This would be total noise and way too much work.
> Consider the client-server groupings, which had constant NBC changes
> across 27 revisions of the drafts.
>
>
> [JC] In such long-running drafts, it has been seen that vendors choose to
> implement draft revisions.  Having YANG modules so tagged makes it clear
> what revision they are implementing and that it is not yet a ratified
> standard.  In terms of work, one only has to update the revision-label when
> the module changes, and they would otherwise be updating the revision
> statement (or changes in an appendix).  Simply bumping this doesn't require
> a great deal of additional work.
>
>
OK



>
> I-Ds currently reuse the same revision-stmt over and over in an update I-D,
> just changing the date.  Maintaining a semver label throughout this process
> is a waste of time.
>
> There is also no way to tell that a 1.x.x release is a work-in-progress.
> It is clear for 0.x.x  only.
>
>
> [JC] YANG Semver (and SemVer 2.0) provide a way to signal such that.  A
> 1.1.0-draft... signals a WIP by having the '-' there.  That character
> indicates a pre-release version.
>
>
>
OK


> There is no easy way to identify a release as "published" or "unpublished",
> especially since YANG modules are extracted from the source document.
>
>
> [JC] Released versions would not have the pre-release marker.
>
>
>

OK


>
>
>
>
>> then tool and humans can easily figure out in which revision NBC
>> changes occured and if they affect a certain usage of the module.
>>
>> Instead of simply properly documenting the changes, we invent fuzzy
>> version numbers and complex algorithms to reverse engineer the facts
>> that could have easily been documented by whoever makes the change.
>>
>
> Some vendors actually avoid making NBC changes to their APIs.
> IMO this IETF work is trying to normalize bad engineering practice.
> The past WG (people creating YANG 1.0 and YANG 1.1) felt quite strongly
> that changing the type of a leaf was something that should not be allowed.
>
>
> [JC] This work is recognizing that these behaviors occur and providing a
> means whereby they can be signaled.  This is part of the requirements.  By
> having these conventions it does not mean that one has to make NBC
> changes.  It's just a way to indicate them and make it easier for consumers
> to know when such changes occur.
>
>
I agree that vendors will not start making NBC changes now that the IETF
says it's OK.
It could be useful to flag the entire module as "NBC change happened"
depending on
the contents of the module.



>
>> If the reason is that developers do not document their changes, then
>> go and develop tools to force them to document their changes. I do not
>> think it is fair to simply push the pain to the users of YANG modules.
>>
>>
>
> 1 more comment:
>
> Errata not integrated:
>
> Most organizations publish corrected documents when mistakes are found,
> but not the IETF.
> Instead the WEB page will have a link "Errata Exists".  It is up to the
> reader to notice it,
> read all the errata, apply all the changes manually, and maintain their
> own private copy
> of the RFC.
>
> This is especially bad for extracted YANG modules.
> For example there is no way to create a revision label (or date string)
> that
> indicates that this is  [ietf-netconf-notifications, 2012-02-06] + Errata
> 3957.
> Vendors just hack the incorrect RFC version of the module.
> It would be quite useful to have corrected YANG modules when errors are
> found.
> Even the YANG repos simply maintain the uncorrected versions.
>
>
> [JC] Errata on YANG modules should necessitate a new version of that
> module.  Admittedly, this takes time in the IETF, but incorporating errata
> on the fly without having a means to signal a change from the RFC version
> would make it even harder on the consumer.
>
>
Currently each vendor may patch the standard version and not update
the revision-date.
Real clients may break if the revision-date is changed to the wrong value
and does not
match any known version.  IMO the extracted YANG module should be as
correct as possible.
Nobody is suggesting that published RFCs get altered in this way.







Joe
>
>
Andy