Re: [core] progressing ietf-core-yang-cbor and ietf-core-sid

[Ivaylo Petrov <[e-mail]>]

Thank you for your clarification, I think we are getting closer to
understanding the difference of viewpoints. Find my comments below.

On Tue, Jan 21, 2020 at 8:29 PM Andy Bierman <[e-mail]> wrote:
>
>
>
> On Tue, Jan 21, 2020 at 2:55 AM Ivaylo Petrov <[e-mail]> wrote:
>>
>> Hello Andy,
>>
>> Find my answers below
>>
>> On Sun, Jan 19, 2020 at 6:37 AM Andy Bierman <[e-mail]> wrote:
>> >
>> >
>> >
>> > On Thu, Jan 16, 2020 at 9:47 AM Carsten Bormann <[e-mail]> wrote:
>> >>
>> >> On 2020-01-16, at 13:50, Andy Bierman <[e-mail]> wrote:
>> >> >
>> >> > There is a significant amount of metadata needed to generate a SID file,
>> >> > starting with all the previous YANG revisions or SID files, and when/if
>> >> > the SID files have been reset vs. updated with preserved numbers.
>> >>
>> >> Well, specifically, the current YANG module and the previous SID file; not the entire history of the universe.
>> >>
>> >> The need for the previous SID file means that there needs to be a single lineage; SID files cannot be independently progressed.  This calls for central entity (or a blockchain :-).
>> >> What we need to define unambiguously is who that entity is.
>> >> One we are at the IANA, the Designated Expert (DE) can play this role.
>> >> But we need to define the process leading up to this.
>> >> (Yes, that process will evolve, and we probably also need to allow exceptions.
>> >> But there must be strong guide rails that work for the vast majority of cases.)
>> >>
>> >> Adding a version number to the SID file could indeed help with accidents/disasters.
>> >> But it could also lead to people thinking they can get away with concurrent lineages; thinking that version numbers will fix any fallout.
>> >> That would be a disaster on its own, most likely leading to permanent universe splits.
>> >> So I’m not so sure we want to open that Pandora’s box.
>> >>
>> >
>> >
>> > We already opened Pandora's box by attempting to create a 1-flat-number globally unique, multi-module,
>> > multi-owner object identifier. I think YANG is quite different than the MAC address example of success
>> > in this endeavour because YANG modules need the numbers assigned in the development phase,
>> > not the manufacturing phase.
>> >
>> > YANG has many features that make it easier on the writer and harder on the tool maker,
>> > such as no order dependencies, but is has some guardrails. No matter how the writer may
>> > possibly alter or refactor a YANG module, nothing unintentional can change the YANG identifier for
>> > any defined data node.  SID has no such guardrails.
>>
>> [ivaylo]:
>> It seems that you are implying that refactoring a YANG without changing the
>> YANG identifiers and the paths will cause changes to the .sid file. Could you
>> provide an example as I am not sure I see why this needs to be the case.
>> Note that I added the restriction of not changing paths as otherwise I
>> believe the
>> NETCONF representation will also change, so CORECONF will be no different.
>>
>
>
> YANG modules under development do not follow any of the "module update" rules
> so there are no restrictions that can be relied on in real YANG development.
> This requires the tools and developers to be more careful, especially
> since YANG tools have no existing reason to be careful in these areas.
>

[ivaylo]:
If there are no restrictions to what a YANG module change can bring, I
am afraid we can not provide much help for .sid files. The suggestion
that any implementer of pre-WGLC version of the draft might need to
adjust her implementation due to SIDs change seems acceptable to me and
I don't see how we can provide anything better.

>
>>
>> >
>> > I think there is a practical solution for deployment, which does not require the use of the
>> > centrally maintained SID files, which is not realistic for 4 reasons:
>> >   - no way to apply changes to an incorrect SID file (without a sid-file-identifier)
>>
>> [ivaylo]:
>> I believe currently the way to achieve this is to have a new version
>> of the YANG module,
>> so that a new version of the SID file can be generated. Having a
>> global version of .sid
>> file that is an integer that progresses lineary seems as a good
>> solution to me. It should
>> also be able to accommodate your concern here I believe. As Michael
>> said, the latest
>> version of the .sid file should be able to handle both all clients of
>> previous versions of
>> the YANG module and the latest one. As there could be multiple versions for one
>> version of a YANG module, errors could be corrected. Does this sound
>> good to you?
>
>
>
> This will not work at all.
> There is no reason to update the YANG module.
> What would change? Just the revision-date?
>
> Even if we ignore all possibility that a SID file can have a flaw in it,
> there is still the problem caused when imported modules change.
> Reusable groupings in a common module are all the rage.
> It is very possible that the target module may never even change after release 1
>
>    module foo {
>         ...
>         import A { prefix a; }
>         revision 2019-01-01;
>
>         container top {
>            uses a:grouping1;
>        }
>   }
>
> The SID file contents for /top are not determined by module foo.
> Over time, module A will keep changing and module foo will never change.
> All of the unspecified revisions imported by module foo determine the expansion of 'grouping1.
> This approach only works within a server implementation because all the module revisions
> are known to the client via the YANG library.
>

[ivaylo]
I think the difference of viewpoints here came from my assumption that
all yang modules import specific versions of other modules. You point
out in your email that in practice this is not necessarily the case. If
we want to be able to handle this as I believe we are, I agree that we
might need to add information in the SID file about what version of each
of the dependent modules had been used to allocate any given SID as
otherwise we might have serious problems figuring this out at runtime,
if this is at all possible.

Storing this information can be solved by having many independent
versions of .sid files - one for a given set of YANG modules and
revisions or we can have each SID number allocation contain more
information than just the path to that node - namely the versions of all
the YANG modules and the revision information.  In both cases it appears
that this will complicate how SIDs allocation.

Andy, do you agree that I have captured the essence of your concern
correctly? Are there any thoughts whether we want to solve this problem
and which is the best approach (even if it is not any of the ones
already mentioned)?

I might be missing some historical background here, but my personal
preference is to go with the second approach as it would allow for
linear versioning of the .sid files, which I believe is lighter on the
constrained device. It seems that your preference, Andy, is to go with
the first one and have SIDs be more contextually dependent.

>
>
>>
>>
>> >   - no way for vendors to apply deviations on any YANG modules, e,.g deviation { not-supported; }
>>
>> [ivaylo]:
>> I am not sure I see why that is the case. In sec 3 of
>> draft-ietf-core-yang-library we have
>>
>> * 'feature' and 'deviation' are implemented using a SID (i.e. an
>> unsigned integer).
>>
>> While I agree that more details can be added here, I don't see any
>> reason deviations
>> not to be supported.
>>
>> >   - the SID file generation depends on the specific revisions of all modules imported
>> >     and submodules included by the target YANG module
>>
>> [ivaylo]:
>> I agree, but I believe this is a good thing.  As is said sec 5.1.1 of RFC6020
>>
>> ` modules need to be imported using specific revisions`.
>
>
>
> This is not how YANG is used 95% of the time.
> Even if I import-by-revision, it is very common for that module to
> have its own imports, and I have no control over the revisions used there.
>
>
>>
>>
>> Could you provide more details of the issue you see?
>>
>> >   - vendors are already failing to deploy YANG modules from a central source model.
>> >     The NETCONF and RESTCONF protocols have robust mechanisms to discover and
>> >     retrieve the actual YANG files used by the server, and vendors rely on this feature.
>> >     Some vendors need model branching and other complexity that make SID assignment even harder
>> >     (see https://tools.ietf.org/html/draft-ietf-netmod-yang-versioning-reqs-02)
>> >
>>
>> [ivaylo]:
>> I believe that the envisioned .sid file retrieval mechanism is:
>> 1. You find the sid that you are interested in
>> 2. You check in which mega range it is and who is the operator of that
>> mega range in
>> the IANA registry.
>> 3. You contact that mega range operator through a given protocol to
>> obtain further details.
>>
>
>
> No. I am interested in loading the SID data from each individual server, as needed.
>
> I understand some people want to hard-wire SIDs into their client or server
> and they will not have the YANG identifiers.  I am not interested in this
> deployment option for YANG-Push so the SID values will get added at runtime.
> The client will consume or use cached the SID files at session-startup-time, just like the YANG modules.

[ivaylo]:
You are correct that I was looking much more from the perspective of
hard-wiring the SIDs on constrained devices and assume that
introspection will be done as needed using the central IANA registry or
other server with such information. If you want to know what .sid file
is present for each server, wouldn't the mechanism from
draft-ietf-core-yang-library solve that? Or maybe you would like to have
a more compact reply that gives only the YANG module name, version and
version the SID file that can be found on a server - be it internal or
external one. If that is the need you have, we can see which is the best
approach to resolve it.

>
>
>>
>> Presently we believed as there are not going to be too many mega range
>> registries and
>> we have no experience operating them it might be unwise to specify the
>> protocol that
>> they need to implement in order for clients to be able to query them.
>> Specifying such a
>> protocol might be something we do if this is a really big concern for people.
>> Would that retrieval mechanism cover the cases that you were thinking
>> of? Are there other
>> cases that you believe it should cover?
>
>
>
> I think vendors will continue to fill in the gaps with proprietary mechanisms.
> The SID file format will get augmented, etc. to support real deployment scenarios.
> This could be considered a protocol issue and can be ignored for SID files in general.
>
> How the client gets the YANG and SID files could be considered an implementation detail.
> The IANA repo is there but there is no requirement to use it.
>

[ivaylo]:
I assume that one of the participants is a constrained device, which
might not be exactly what you are considering, but in that case if the
constrained device is a client, it is very unlikely for it to try to
discover at runtime what are the SIDs that a server will support. It
will generally send CORECONF requests that contain some SIDs and assume
that the server or a proxy in between will be able to understand their
meaning. That is where the IANA repository and a protocol for
inspection how a SID maps can be useful.

Then assuming that the constrained device is the server, a client can
discover the supported SIDs either through introspection protocol, which
will be heavy, but might be required in some cases or more likely
through provisioning/bootstrap.

It seems to me that you are considering also the question if none of the
endpoints is really constrained. In that case I believe the client will
still be able to use the same introspection protocol as before and find
the YANG modules looking at the SID file.

>
>>
>>
>> > CORECONF (and SID in general) needs a SID file retrieval mechanism.
>> > Perhaps based on YANG Data Instance Files https://tools.ietf.org/html/draft-ietf-netmod-yang-instance-file-format-06
>> > A server will make the URI of this resource known somehow, which can be used if needed,
>> > before a client starts using SIDs.
>> >
>> > In addition, or instead of listing YANG modules, the data is SID files, maybe 1 super-SID-file JSON object.
>> > It is not realistic that a client will be able to use all global SID files based on the YANG module name
>> > and revision date, especially in the development phase. The IANA SID file may be useful after the YANG
>> > module is very stable.
>> >
>> > More solution details:
>> >   - The WG or vendor can maintain an informal public archive of SID files.
>> >   - The YangModels repo update process should include SID files.  https://github.com/YangModels/yang
>> >   - Vendors still need the public SID file for the starting point, or maybe use it unchanged
>> >   - The SID files need <CODE BEGINS> support and probably some tool changes in the ID-submission process
>> >   - IANA will only archive the RFC version, just like YANG modules.
>> >
>> >
>> >
>> >> Grüße, Carsten
>> >
>> >
>> >
>> > Andy
>> >
>>
>> --
>> Best regards,
>> Ivaylo
>
>
>
> Andy
>

--
Best regards,
Ivaylo