Re: [netmod] backward compatibility requirements in draft-verdt-netmod-yang-versioning-reqs-00

[Kent Watsen <[e-mail]>]

Before RESTCONF, I implemented a versioned REST API that maintained constant object URIs, while allowing the objects themselves to be versioned, by using the Content-Type and Accept headers (yes, we created a media type for every "object" in the system).  The server always natively understood the "latest", while being able to translate to/from the last few versions of the product.  It was effort to update the translators each release but, thankfully, there were only a few of them each time, and the unit-tests from the previous releases could be used to test correctness of the translators.

What's being discussed now sounds similar, with similar associated costs.  Whether or not we use a new module-name per "major" version (essentially what we have today, but further refined by Chris's idea) won't affect this cost much.  The primary advantage to using the same module name across major versions is that is gives to client more clue behind what is going on.  Perhaps similar clue can be achieved via an extension statement:

  module foo-3 {
    …
    replaces-module foo-2;
    …
  }


Kent // contributor


On 7/23/18, 2:46 PM, "netmod on behalf of Andy Bierman" <[e-mail]<mailto:[e-mail]> on behalf of [e-mail]<mailto:[e-mail]>> wrote:



On Mon, Jul 23, 2018 at 7:32 AM, Robert Wilton <[e-mail]<mailto:[e-mail]>> wrote:



On 23/07/2018 15:08, Andy Bierman wrote:


On Mon, Jul 23, 2018 at 6:24 AM, Robert Wilton <[e-mail]<mailto:[e-mail]>> wrote:



On 23/07/2018 12:54, Andy Bierman wrote:


On Mon, Jul 23, 2018 at 2:50 AM, Robert Wilton <[e-mail]<mailto:[e-mail]>> wrote:
Hi Chris, Andy,


On 21/07/2018 17:00, Christian Hopps wrote:
As I pointed out at the mic @102 this requirement derives directly from the 1.x requirement of not changing the name of the module/namespace. If you allow for changing the namespace/module name for "major" (i.e., incompatible) changes (i.e., like today) then this 3.1 requirement goes away.
Not sure that I agree.

I think that you have made an assumption here that the server will continue to support both old and new major revision (with different name) of the module at the same time.  However, these is nothing in the existing YANG upgrade rules that requires that.

Ultimately, there is a choice whether supporting older module versions is the servers problem or the clients problem, or perhaps a combination of the two.

The aim of requirement 3.1 is to ensure that there is a standard mechanism available so that server implementations that want the flexibility of supporting older client versions have a standard way of doing so.  My intention is that this part of the solution would be optional to implement and hence decided by the market, which is why the text in the requirement is "to allow servers" rather than "to require servers".


API versioning is usually done on the message exchanges.
Trying to do the same for datastore contents is not going to work.
A YANG schema can be considered an API.  Particularly looking at say the OpenConfig YANG schema.  I doubt all implementations will store all their configuration and operational state in a central place.


You can write the word MUST in all caps as many times as you want,
but that will not change anything.

I disagree.  Marking a requirement as a MUST means that requirement has to be met for a solution to be considered viable.

I previously had some text in the draft explaining how RFC 2119 text translates to evaluating the requirements, but was asked to take it out because it is obvious.  Perhaps it should go back in ...

So you can answer the question how YANG validation works when multiple revisions of a
module are implemented?

Ok, so the scheme that I was considering is:
 - The device implements only one version of the schema (i.e. probably all of the latest modules revisions/versions).

However, the protocols are extended to allow clients to select to use an older version set of the modules for that session.  YANG library for that session would report the chosen set of modules as "implemented" by the server for that session.

The server chooses how many different sets of modules are supported, and exactly what module revisions, features are included in each of those module sets.  Normally I would expect exact module sets to align with a previous software release.  The different module-sets can be exposed via YANG library bis.  New RPCs or protocol extensions are required to choose which version is being used for the session.

The server then has code to map from the older module set paths to the latest version that is "implemented" by the device.  The vast majority of the mappings would be trivial mappings of the same value on the same path.

This mapping is exactly the same as would have to be done on a client if it has to support servers running different revisions.

Not all changes can be mapped, some would have to just fail (perhaps could be covered by deviations).


  I do not doubt that you can find a leaf somewhere that can
be changed.  Not at all convinced the validation rules can be rewritten to account
for actual incompatible changes, like changing the type of data node or replacing nodes
with completely different configuration.
The mapping above will not be perfect in all cases, particularly if keys are changes, or support for software is removed, etc.  But they might still help.


  Even less convinced that this complexity is
worth the cost.

Well the complexity ends up going in the client or the server.  I think that this is generally a complex problem to solve.  Operators will likely argue that it is better that the complexity goes in the server to keep the client simple.  Server vendors will likely argue for the reverse.

Note, that I'm still somewhat open to what the right solution should be here.

Sounds very complicated to implement in the server, but at least you avoid multiple variants of a datastore.
Instead the server is providing various transformations between data models.

This will make the standards much more complicated and heavyweight to implement.
If it was so easy and so important, vendors would already support it in their proprietary APIs.
Instead vendors decide when to end-of-life products and features.



Thanks,
Rob

Andy






Thanks,
Rob

Andy


Thanks,
Rob

Andy



I think the plan is to reword some of these to get closer to the intention which I believe is to allow for smoother transition from one module to the next while making incompatible but mostly non-impacting changes.

Thanks,
Chris.

Andy Bierman <[e-mail]<mailto:[e-mail]>> writes:
Hi,

I strongly object to requirement 3.1:


    3.1  The solution MUST provide a mechanism to allow servers to
            support existing clients in a backward compatible way.



This is not what servers do today at all.
They provide only one version of an implemented module, as specified in RFC
7950.

It is a vendor and operator decision when to upgrade a server such that
non-backward compatible changes are made. They must decide if/when it is ok
based on the client applications in use.

This requirement says you cannot make backward-incompatible changes
which completely contradicts requirements 1.1 and 1.2.

IMO requirement 3.1 should be removed, or change MUST to MAY


Andy
_______________________________________________
netmod mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/netmod<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwMFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=NLKzB3RRvGkAdBdSqsbPf9vbI3OBcyg6nz2Pb0SjZEc&s=Uhv31M1fvzhyys-4tCgiHGkxvk-w-dKbAaehPUeQ_hY&e=>

_______________________________________________
netmod mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/netmod<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netmod&d=DwMFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=NLKzB3RRvGkAdBdSqsbPf9vbI3OBcyg6nz2Pb0SjZEc&s=Uhv31M1fvzhyys-4tCgiHGkxvk-w-dKbAaehPUeQ_hY&e=>
.