Re: [core] progressing ietf-core-yang-cbor and ietf-core-sid

[Andy Bierman <[e-mail]>]

On Tue, Jan 21, 2020 at 2:55 AM Ivaylo Petrov <[e-mail]> wrote:

> Hello Andy,
>
> Find my answers below
>
> On Sun, Jan 19, 2020 at 6:37 AM Andy Bierman <[e-mail]> wrote:
> >
> >
> >
> > On Thu, Jan 16, 2020 at 9:47 AM Carsten Bormann <[e-mail]> wrote:
> >>
> >> On 2020-01-16, at 13:50, Andy Bierman <[e-mail]> wrote:
> >> >
> >> > There is a significant amount of metadata needed to generate a SID
> file,
> >> > starting with all the previous YANG revisions or SID files, and
> when/if
> >> > the SID files have been reset vs. updated with preserved numbers.
> >>
> >> Well, specifically, the current YANG module and the previous SID file;
> not the entire history of the universe.
> >>
> >> The need for the previous SID file means that there needs to be a
> single lineage; SID files cannot be independently progressed.  This calls
> for central entity (or a blockchain :-).
> >> What we need to define unambiguously is who that entity is.
> >> One we are at the IANA, the Designated Expert (DE) can play this role.
> >> But we need to define the process leading up to this.
> >> (Yes, that process will evolve, and we probably also need to allow
> exceptions.
> >> But there must be strong guide rails that work for the vast majority of
> cases.)
> >>
> >> Adding a version number to the SID file could indeed help with
> accidents/disasters.
> >> But it could also lead to people thinking they can get away with
> concurrent lineages; thinking that version numbers will fix any fallout.
> >> That would be a disaster on its own, most likely leading to permanent
> universe splits.
> >> So I’m not so sure we want to open that Pandora’s box.
> >>
> >
> >
> > We already opened Pandora's box by attempting to create a 1-flat-number
> globally unique, multi-module,
> > multi-owner object identifier. I think YANG is quite different than the
> MAC address example of success
> > in this endeavour because YANG modules need the numbers assigned in the
> development phase,
> > not the manufacturing phase.
> >
> > YANG has many features that make it easier on the writer and harder on
> the tool maker,
> > such as no order dependencies, but is has some guardrails. No matter how
> the writer may
> > possibly alter or refactor a YANG module, nothing unintentional can
> change the YANG identifier for
> > any defined data node.  SID has no such guardrails.
>
> [ivaylo]:
> It seems that you are implying that refactoring a YANG without changing the
> YANG identifiers and the paths will cause changes to the .sid file. Could
> you
> provide an example as I am not sure I see why this needs to be the case.
> Note that I added the restriction of not changing paths as otherwise I
> believe the
> NETCONF representation will also change, so CORECONF will be no different.
>
>

YANG modules under development do not follow any of the "module update"
rules
so there are no restrictions that can be relied on in real YANG development.
This requires the tools and developers to be more careful, especially
since YANG tools have no existing reason to be careful in these areas.



> >
> > I think there is a practical solution for deployment, which does not
> require the use of the
> > centrally maintained SID files, which is not realistic for 4 reasons:
> >   - no way to apply changes to an incorrect SID file (without a
> sid-file-identifier)
>
> [ivaylo]:
> I believe currently the way to achieve this is to have a new version
> of the YANG module,
> so that a new version of the SID file can be generated. Having a
> global version of .sid
> file that is an integer that progresses lineary seems as a good
> solution to me. It should
> also be able to accommodate your concern here I believe. As Michael
> said, the latest
> version of the .sid file should be able to handle both all clients of
> previous versions of
> the YANG module and the latest one. As there could be multiple versions
> for one
> version of a YANG module, errors could be corrected. Does this sound
> good to you?
>


This will not work at all.
There is no reason to update the YANG module.
What would change? Just the revision-date?

Even if we ignore all possibility that a SID file can have a flaw in it,
there is still the problem caused when imported modules change.
Reusable groupings in a common module are all the rage.
It is very possible that the target module may never even change after
release 1

   module foo {
        ...
        import A { prefix a; }
        revision 2019-01-01;

        container top {
           uses a:grouping1;
       }
  }

The SID file contents for /top are not determined by module foo.
Over time, module A will keep changing and module foo will never change.
All of the unspecified revisions imported by module foo determine the
expansion of 'grouping1.
This approach only works within a server implementation because all the
module revisions
are known to the client via the YANG library.




>
> >   - no way for vendors to apply deviations on any YANG modules, e,.g
> deviation { not-supported; }
>
> [ivaylo]:
> I am not sure I see why that is the case. In sec 3 of
> draft-ietf-core-yang-library we have
>
> * 'feature' and 'deviation' are implemented using a SID (i.e. an
> unsigned integer).
>
> While I agree that more details can be added here, I don't see any
> reason deviations
> not to be supported.
>
> >   - the SID file generation depends on the specific revisions of all
> modules imported
> >     and submodules included by the target YANG module
>
> [ivaylo]:
> I agree, but I believe this is a good thing.  As is said sec 5.1.1 of
> RFC6020
>
> ` modules need to be imported using specific revisions`.
>


This is not how YANG is used 95% of the time.
Even if I import-by-revision, it is very common for that module to
have its own imports, and I have no control over the revisions used there.



>
> Could you provide more details of the issue you see?
>
> >   - vendors are already failing to deploy YANG modules from a central
> source model.
> >     The NETCONF and RESTCONF protocols have robust mechanisms to
> discover and
> >     retrieve the actual YANG files used by the server, and vendors rely
> on this feature.
> >     Some vendors need model branching and other complexity that make SID
> assignment even harder
> >     (see
> https://tools.ietf.org/html/draft-ietf-netmod-yang-versioning-reqs-02)
> >
>
> [ivaylo]:
> I believe that the envisioned .sid file retrieval mechanism is:
> 1. You find the sid that you are interested in
> 2. You check in which mega range it is and who is the operator of that
> mega range in
> the IANA registry.
> 3. You contact that mega range operator through a given protocol to
> obtain further details.
>
>

No. I am interested in loading the SID data from each individual server, as
needed.

I understand some people want to hard-wire SIDs into their client or server
and they will not have the YANG identifiers.  I am not interested in this
deployment option for YANG-Push so the SID values will get added at runtime.
The client will consume or use cached the SID files at
session-startup-time, just like the YANG modules.



> Presently we believed as there are not going to be too many mega range
> registries and
> we have no experience operating them it might be unwise to specify the
> protocol that
> they need to implement in order for clients to be able to query them.
> Specifying such a
> protocol might be something we do if this is a really big concern for
> people.
> Would that retrieval mechanism cover the cases that you were thinking
> of? Are there other
> cases that you believe it should cover?
>


I think vendors will continue to fill in the gaps with proprietary
mechanisms.
The SID file format will get augmented, etc. to support real deployment
scenarios.
This could be considered a protocol issue and can be ignored for SID files
in general.

How the client gets the YANG and SID files could be considered an
implementation detail.
The IANA repo is there but there is no requirement to use it.



>
> > CORECONF (and SID in general) needs a SID file retrieval mechanism.
> > Perhaps based on YANG Data Instance Files
> https://tools.ietf.org/html/draft-ietf-netmod-yang-instance-file-format-06
> > A server will make the URI of this resource known somehow, which can be
> used if needed,
> > before a client starts using SIDs.
> >
> > In addition, or instead of listing YANG modules, the data is SID files,
> maybe 1 super-SID-file JSON object.
> > It is not realistic that a client will be able to use all global SID
> files based on the YANG module name
> > and revision date, especially in the development phase. The IANA SID
> file may be useful after the YANG
> > module is very stable.
> >
> > More solution details:
> >   - The WG or vendor can maintain an informal public archive of SID
> files.
> >   - The YangModels repo update process should include SID files.
> https://github.com/YangModels/yang
> >   - Vendors still need the public SID file for the starting point, or
> maybe use it unchanged
> >   - The SID files need <CODE BEGINS> support and probably some tool
> changes in the ID-submission process
> >   - IANA will only archive the RFC version, just like YANG modules.
> >
> >
> >
> >> Grüße, Carsten
> >
> >
> >
> > Andy
> >
>
> --
> Best regards,
> Ivaylo
>


Andy