Re: [Netmod-ver-dt] Meeting minutes [was Today's Ver DT meeting]

["Joe Clarke (jclarke)" <[e-mail]>]

On Jan 13, 2020, at 11:04, Rob Wilton (rwilton) <[e-mail]<mailto:[e-mail]>> wrote:

Hi Joe,

Each of the examples has a different client experience.  There are rough notes above each individual examples hopefully explaining the key aspects.  I’ve copied the examples intro text inline below.  Hopefully, this help clarify, but please let me know if anything is still not clear.  I’ve also given brief answers to your questions inline below …

I’ve also noticed that I have a mistake in the ones with a selectable primary schema.  I should have illustrated a RPC at the end rather than configuration for selecting the primary schema.

Ah, thanks.  That addresses my question.  So the final thing should be that client issuing the RPC to select the desired schema?

Joe



Example 1:
Selectable primary version, no secondary
========================================

This example illustrates a device is that capable of supporting 3 different versions of its native manageability schema.
- The clients can decide (via RPC) which version of primary schema is used by the device, all interactions with the device use the primary schema version (with no version selection required in the RPCs).

This example supports 3 versions of its schema, that contain non-backwards-compatible changes between versions:
    3.0.0 (the default schema version of the software)
    2.1.0, a preceding nbc software version
    1.3.1, a preceding nbc software version.

The aim here is allow a client to entirely configure and interact with the device using the schema associated with an older version of the software.
- The device would be expected to store the schema and version alongside the persistent configuration.
- The schema version can only be modified via RPC, which allows the configuration to either be migrated to a new version, or to be replaced with the provided configuration written in the new schema version.
may optionally allow the configuration to be migrated to/from the new version, or replaced with a
 - How does the user update the software?
   - Could be done with a custom RPC, which could optionally include a configuration
     validated against a different schema version to replace it with.
   - Can it be done with a regular config update?
   - Can it be done with a secondary schema.

Example 2:

Fixed primary, configurable secondary
=====================================

This example illustrates a device is that capable of supporting 3 different versions of its native manageability schema, but using a different approach where the primary schema-set for the device is fixed to the latest version of the software, and cannot be changed (except by software upgrade)
   - Clients may configure secondary schema-sets to allow the configuration to be read/written using older versions of the schema, which are always immediately translated to/from configuration changes in the latest version of the schema, and which must result in a valid configuration for the configuration to be accepted to the system.

This example supports 3 versions of its schema, that contain non-backwards-compatible changes between versions:
    3.0.0 (the primary schema version of the software)
    2.1.0, a preceding nbc software version, may be configured as a secondary schema
    1.3.1, a preceding nbc software version, may be configured as a secondary schema

The example below assumes that the secondary-schema must be configured before they are available, but alternatively the server could automatically make the secondary schema available without requring client configuration.

Schema upgrade is outside the scope of this example, but normally a device would be expected to automatically upgrade the configuration perhaps with or without client intervention.


Example 3:

Configurable primary and secondary
==================================

This example illustrates a device is that capable of supporting 3 different versions of its native manageability schema.
- The clients can decide (via RPC) which version of primary schema is used by the device
- Clients can also optionally configure secondary schema to provide read-only views of the configuration using any configuration version higher than the primary schema version.

This example supports 3 versions of its schema, that contain non-backwards-compatible changes between versions:
    3.0.0 (the default schema version of the software)
    2.1.0, a preceding nbc software version
    1.3.1, a preceding nbc software version.

The aim here is allow a client to entirely configure and interact with the device using the schema associated with an older version of the software.
- The device would be expected to store the schema and version alongside the persistent configuration.
- The schema version can only be modified via RPC, which allows the configuration to either be migrated to a new version, or to be replaced with the provided configuration written in the new schema version.
may optionally allow the configuration to be migrated to/from the new version, or replaced with a
 - How does the user update the software?
   - Could be done with a custom RPC, which could optionally include a configuration
     validated against a different schema version to replace it with.
   - Can it be done with a regular config update?
   - Can it be done with a secondary schema.

From: Joe Clarke (jclarke) <[e-mail]<mailto:[e-mail]>>
Sent: 13 January 2020 15:47
To: Rob Wilton (rwilton) <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>
Subject: Re: [Netmod-ver-dt] Meeting minutes [was Today's Ver DT meeting]

Rob, should I interpret these examples as:

* Device supports N schema sets
[RW]
Yes.  In all three examples. N is 3.

* Only the default can be used out of the box
[RW]
Maybe.

A device can either automatically support secondary schema, or it can require that they be configured first.  This can be expressed by whether anything is contained in /schema-set-selection/secondary in the operational datastore.  E.g. perhaps whether secondary schema-sets are available might be determined by what packages are installed rather than requiring client configuration.  My aim with the solution is to give the server quite a lot of flexibility in how this can be implemented.


* An administrator MAY configure one or more additional, secondary schema sets
[RW]
Yes.

* If allowed, a client MAY choose a secondary schema set with an RPC/path
[RW]
Yes.


I’m trying to understand the client experience in these approaches just so I’m absolutely clear.  And if I’m right in my assumptions above, how does that last client know what secondary schemas are enabled on the device?
[RW]
By querying the list in “/schema-set-selection/secondary”.  in <running> this list allows a client (e.g. admin) to choose which secondary schema-sets are available.   In <operational> this represents the actual secondary schema-sets that are available on the device.  The entries in this list can be due to admin configuration, or because the device just automatically supports some secondary schema-sets, or even a combination of the two.  NMDA origin metadata would indicate the source of these entries (e.g. are they due to configuration or have they been added by the system).

Thanks,
Rob



Joe


On Jan 13, 2020, at 10:37, Rob Wilton (rwilton) <[e-mail]<mailto:[e-mail]>> wrote:

Hi,

I’ve been playing around with the schema selection YANG model and examples a bit more.

3 examples attached (all for selecting from 3 different versions of the YANG schema):

1.      3 choices of primary schema versions; no secondary schema.

2.      3 choices of primary schema versions; secondary schema are configurable read-only views of latest versions

3.      Primary schema fixed to latest version; older schema versions are allowed.


Comments and tree output inline below …

From: Netmod-ver-dt <[e-mail]<mailto:[e-mail]>> On Behalf Of Rob Wilton (rwilton)
Sent: 10 January 2020 10:09
To: [e-mail]<mailto:[e-mail]>
Subject: [Netmod-ver-dt] Meeting minutes [was Today's Ver DT meeting]

Points from the meeting what would be good for further discussion.  I think that a lot of these issues are probably still open and require further discussion:

1)     It is clear that we need to standardize supporting different versions of a schema, but do we also want to support different schema families (e.g. vendor vs IETF vs OC)?
a.       My personal view is that it would be good if the model is general enough that this can be described (and I think that it might be useful if there are some examples given in the appendix).
2)     For supporting non-native schema, there seem to be two approaches:
                                                                        i.     always translate into the native schema, reads from say OC convert back from the native schema
                                                                       ii.     store/persist configuration in the external schema (e.g. OC).  E.g. so a get-config on OC would only get back what had been written to the OC schema, not any other schema.
b.      Do we need to describe these two approaches?  For secondary schema does the model need to indicate which method is being performed, or is that purely an implementation detail?
[RW]
I think that the draft does want to describe the differences between a translated schema-sets vs filtered schema-sets that are just a subset of the primary schema-set.  At the moment, or in the examples that I have so far, I don’t think that these need to expressed in the YANG model.

3)     Can both of the approaches described in (2) above work for different versions of the schema?  Or must these always be handled by dynamic translation.
[RW]
My thinking is that schema with different module versions (relative to the primary schema) must always be translated.

4)     What is the schema that the configuration is persisted in?  Should this automatically be tied to the default schema, or should this a separate data node, that could potentially be configurable?
a.      Related to this, how does the system handle changing the default schema?  Can that be done as part of a config request?  Does the connection have to be closed and re-opened?  Or should it be an RPC outside of configuration?
[RW]
I still think that it makes sense to tie this to the primary schema.  I’ve introduced a RPC to change the primary schema rather than allowing this to be done via regular configuration.  I’m hoping that this solves the issue that we were discussing in the last meeting.

5)     We need a concise way of describing the relationship between schema and datastores.  The current approach in the model is somewhat verbose, if every datastore needs to be described separately (but is somewhat aligned to YANG library style).
a.      Should we allow schema to only support some datastores (e.g. only operational, or only conventional configuration)?
[RW]
Adding a common schema definition, so that it just becomes a list of supported datastores makes this more concise.  I think that we want the flexibility to allow different schema to express support for different datastores.

6)     Do we need secondary schema to have the ability to be defined or described as only being read-only?
a.      E.g. I can see a requirement to allow a client a configure a device using an older version of the schema.  I can also see a requirement of allowing a client to see what the instance data against a older version of a schema would look like in a more current version.  But I’m less convinced of the need to be able to convert a current configuration back into an older format.
[RW]
I think that this is useful, and have added this to the latest version.

Approximate changes to the schema:

•        Primary schema is now only changeable via RPC, it can’t be chosen via config (secondary schema are still configured, or a server can support them automatically)

•        Renamed stuff from “schema” to “schema-set” since that seems more correct

•        Added a list of “common packages” that apply to all datatstores, in addition to per datastore packages (the datastore schema is defined as the union of the common and per-datastore packages).

•        Added a flag to indicate that a datastore is read-only, for datastores that would normally be writable.  Actually, it might be cleaner to reverse the sense and change this to writable.

•        Renamed some of the containers/flags/etc.


Tree output:

module: ietf-schema-selection-2
  +--rw schema-set-selection
     +--ro primary?      -> /schema-set-selection/schema-set/name
     +--rw secondary*    -> /schema-set-selection/schema-set/name {secondary-schema-set}?
     +--rw custom* [name] {custom-schema-set}?
     |  +--rw name               string
     |  +--rw description?       string
     |  +--rw included-schema*   -> /schema-set-selection/schema-set/name
     +--ro schema-set* [name]
        +--ro name                    string
        +--ro common-package* [name version]
        |  +--ro name        -> /yanglib:yang-library/yl-pkg:package/name
        |  +--ro version     -> /yanglib:yang-library/yl-pkg:package[yl-pkg:name = current()/../name]/version
        |  +--ro checksum?   -> /yanglib:yang-library/yl-pkg:package[yl-pkg:name = current()/../name][yl-pkg:version = current()/../version] /yl-pkg:checksum
        +--ro datastore* [name]
        |  +--ro name                  ds:datastore-ref
        |  +--ro read-only?            empty
        |  +--ro additional-package* [name version]
        |     +--ro name        -> /yanglib:yang-library/yl-pkg:package/name
        |     +--ro version     -> /yanglib:yang-library/yl-pkg:package[yl-pkg:name = current()/../name]/version
        |     +--ro checksum?   -> /yanglib:yang-library/yl-pkg:package[yl-pkg:name = current()/../name][yl-pkg:version = current()/../version] /yl-pkg:checksum
        +--ro primary-selectable?     empty {primary-schema-set}?
        +--ro secondary-selectable! {secondary-schema-set}?
        |  +--ro compatible-with
        |     +--ro primary*     -> /schema-set-selection/schema-set/name
        |     +--ro secondary*   -> /schema-set-selection/schema-set/name
        +--ro custom-selectable! {custom-schema-set}?
           +--ro combinable*   -> /schema-set-selection/schema-set/name

  rpcs:
    +---x change-primary {primary-schema-set}?
       +---w input
          +---w primary?                    -> /schema-set-selection/schema-set/name
          +---w (config)
             +--:(translate)
             |  +---w translate?            empty
             +--:(replacement-config)
                +---w replacement-config?   <anydata>

YANG file, and examples are attached.

Please let me know your thoughts/comments.

Thanks,
Rob



Perhaps we can discuss some of these over email, and try and reach consensus, or otherwise we will pick up discussion in next week’s meeting.

Was there anything else that I missed?

Thanks,
Rob


From: Netmod-ver-dt <[e-mail]<mailto:[e-mail]>> On Behalf Of Rob Wilton (rwilton)
Sent: 09 January 2020 14:11
To: [e-mail]<mailto:[e-mail]>
Subject: [Netmod-ver-dt] Today's Ver DT meeting

Happy New Year!

From the last meeting I did say that I would work on producing some concrete examples of version selection 😊

I’ve started working on this, but more work is required.  I have attached one example that we can discuss in the meeting, more may be forthcoming before the meeting.

As part of that work, I’ve been thinking more about the default/secondary schema.

As a recap, I think that this is roughly where I am at:

(1)   The default schema:
o   is the schema that is used if a client doesn’t specify a secondary schema during their connection (i.e. NETCONF capabilities negotiation, or RESTCONF path).
o   is also the schema used for the persistent configuration (and the <startup> datastore). [I think that this is probably right, but might need further thought/discussion]

(2)   Secondary schema:
o   are additional schema that clients can choose to use, and are specified during their session (i.e. NETCONF capabilities negotiation, or RESTCONF path).
o   can probably take several forms:
•  A filtered view on the default schema (e.g. if the default schema supported vendor native + IETF + OC schema) then a filtered schema view might just be the OC subset.  This should be simple to support since it broadly equivalent to a subtree filter on any get operations.
•  A translated view on the default schema (e.g. perhaps a different version of a schema),  All reads/writes to a translated schema are converted to/from the default schema.  The configuration is persisted in the “default schema”.
o   It might be useful if some of the secondary schema could be labelled as read-only views on the data.  E.g. it might be reasonable to see the configuration using a secondary schema non native version, but not allow it to be modified.

Attached example:

Example 1, a simple example of a server supporting two schema versions, and the client can choose between which version is used for the primary schema.
-        One question related to this example is what happens to the current configuration if a client changes the default schema to a different version.  Does that configuration also automatically get translated as well?  What happens if that translation fails?
-        Do we need a simpler way to reporting the per datastore schema?  E.g. should we have a special value to identify all datastores, or all conventional datastores?

Thanks,
Rob
<example_vendor_versioning_fixed_primary_configurable_secondary.xml><example_vendor_versioning_selectable_primary_no_secondary.xml><example_vendor_versioning_configurable_primary_and_secondary.xml><ietf-schema-selection-2.yang>_______________________________________________
Netmod-ver-dt mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/netmod-ver-dt