[netmod] Re: Defining groupings after the fact? draft-jouqui-netmod-yang-full-include and the reuse of definitions

[Andy Bierman <[e-mail]>]

On Fri, Aug 2, 2024 at 8:15 AM Rob Wilton (rwilton) <rwilton=
[e-mail]> wrote:

> I think that there is a balance to how much groupings are used.  E.g., I
> would regard the OpenConfig models as an example of where groupings are
> overly used resulting in YANG models where it is impossible to glean the
> structure.
>
> I’m also not sure whether groupings really help here – e.g., if you put
> the top level of the module under a grouping, then another module may just
> want to reuse a subtree.
>
> I’m also slightly concerned about whether this can really be achieved in
> an extension (which tooling is allowed to ignore).
>
> I.e., it feels like this really need a new (probably major) version of the
> YANG language – and possible would need a more radical rethink about how to
> manage and reuse parts of YANG schema.
>
>
I agree with all your concerns.

I reject the basic premise of this 'design time' text in RFC 8528

 In principle, the mounted schema can be specified at three different
   phases of the data model life cycle:

   1.  Design time: The mounted schema is defined along with the mount
       point in the parent YANG module.  In this case, the mounted
       schema has to be the same for every implementation of the parent
       module.

   2.  Implementation time: The mounted schema is defined by a server
       implementor and is as stable as the YANG library information of
       the server.

   3.  Run time: The mounted schema is defined by instance data that is
       part of the mounted data model.  If there are multiple instances
       of the same mount point (e.g., in multiple entries of a list),
       the mounted data model may be different for each instance.



YANG is modular by design.
The design-time schema mount option was never thought out or explored.

   The schema mount mechanism defined in this document provides support
   only for the latter two cases.  Design-time mounts are outside the
   scope of this document and could be possibly dealt with in a future
   revision of the YANG data modeling language.


If the contents of a mount-point are hard-wired, then no augmentations are
possible
without rewriting the module with the mount-point.  This is not consistent
with the intent or design
of YANG augments.  It is too brittle for real deployment.




Regards,
> Rob
>


Andy


>
>
>
>
> *From: *Kent Watsen <[e-mail]>
> *Date: *Friday, 2 August 2024 at 14:21
> *To: *Shiya Ashraf (Nokia) <shiya.ashraf=[e-mail]>
> *Cc: *Alexander L Clemm <ludwig=[e-mail]>, Alexander L
> Clemm <[e-mail]>, Jean Quilbeuf <[e-mail]>,
> BOUCADAIR Mohamed IMT/OLN <[e-mail]>, [e-mail]
> <[e-mail]>, Rob Wilton (rwilton) <[e-mail]>
> *Subject: *Re: [netmod] Defining groupings after the fact?
> draft-jouqui-netmod-yang-full-include and the reuse of definitions
>
> [CC-ing Med]
>
>
>
> I wonder if rfc8047bis should have a recommendation to use groupings
> extensively?
>
>
>
> FWIW, my "client-server” suite of drafts in NETCONF use groupings
> extensively.  In fact, whenever a data-node is needed, it is always just a
> container that uses a grouping.
>
>
>
> Kent
>
>
>
> On Aug 2, 2024, at 4:54 AM, Shiya Ashraf (Nokia) <shiya.ashraf=
> [e-mail]> wrote:
>
>
>
> Hello Alex,
>
> " However, even in that case, an update to the original model is still
> required - you cannot simply say "let me use these data definitions from
> that other models", they need to be defined as a grouping"
> <Shiya> Oh sure, absolutely. As you pointed out in one of the earlier
> emails, if this could address 99+% of the cases, why not do it at least for
> the models which we think will have more chance of getting reused, say for
> instance, ietf-interfaces, ietf-hardware etc. And this we could do it today
> in YANG 1.1 and with no extra tools support - which is clearly an advantage
> over the new embed mechanisms for faster commercial deployments of the
> solutions.
> Isn't it?
>
> Thanks,
> Shiya
>
> -----Original Message-----
> From: Alexander L Clemm <ludwig=[e-mail]>
> Sent: Thursday, August 1, 2024 10:49 PM
> To: Shiya Ashraf (Nokia) <[e-mail]>; Alexander L Clemm <
> [e-mail]>; Jean Quilbeuf <[e-mail]>;
> [e-mail]
> Subject: Re: [netmod] Re: Defining groupings after the fact?
> draft-jouqui-netmod-yang-full-include and the reuse of definitions
>
> [You don't often get email from ludwig=[e-mail]. Learn
> why this is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> CAUTION: This is an external email. Please be very careful when clicking
> links or opening attachments. See the URL nok.it/ext for additional
> information.
>
>
>
> Hello Shiya,
>
> re your comment on the "Once models have been defined this way, they
> cannot be altered after the fact":  Well, I guess as William has pointed
> out, it is possible to update a model with another, equivalent model which
> pulls data node definitions into groupings and then uses those groupings.
> That would be a compatible change.  The same groupings will then also be
> free for other models to use.  However, even in that case, an update to the
> original model is still required - you cannot simply say "let me use these
> data definitions from that other models", they need to be defined as a
> grouping (or per Jean's proposal in the draft, you define a new construct
> that would let you "use" aka embed definitions without the need for a
> grouping to be defined).
>
> Cheers
>
> --- Alex
>
> On 8/1/2024 2:20 AM, Shiya Ashraf (Nokia) wrote:
>
> Hi Alex,
>
> "<AC> Correct, you cannot augment a grouping.  However, you can define a
> second grouping and then use both groupings.  I do think that with properly
> designed modules that make extensive use of groupings 99+% of reuse
> scenarios would be covered. "
> <Shiya> Thanks for bringing this point up and I tend to fully agree here.
> In fact when I was reading the schema mount RFC where it starts with the
> short comings of "grouping", I also felt that there could be use-cases
> where some of these aspects of the groupings can turn out to be its
> strengths. For eg: for cases where you need greater control on what you
> want to embed on the mounted tree, for instance, only a selection of the
> augments from the original module or add new augments only on the embedding
> context etc. So though schema-mount/full-embed are very good solutions for
> reusability of existing YANG modules for certain use-cases with its own
> advantages, for many cases the existing methods based on groupings might do
> the job and in a much more simpler way.
>
> But then you say: " Once models have been defined this way, they cannot be
> altered after the fact."
> <Shiya> Could you explain more on this? Technically, One can still define
> a new grouping with all the data nodes that are today in a standard module
> and then replaces the content of the standard module with a simple uses
> statement of the new grouping with out causing a backward compatibility
> issue or any functional change, can’t we ?
>
> Thanks,
> Shiya
>
> -----Original Message-----
> From: Alexander L Clemm <ludwig=[e-mail]>
> Sent: Thursday, August 1, 2024 12:37 AM
> To: Jean Quilbeuf <jean.quilbeuf=[e-mail]>;
> [e-mail]
> Subject: [netmod] Re: Defining groupings after the fact?
> draft-jouqui-netmod-yang-full-include and the reuse of definitions
>
> [You don't often get email from ludwig=[e-mail].
> Learn why this is important at
> https://aka.ms/LearnAboutSenderIdentification ]
>
> CAUTION: This is an external email. Please be very careful when clicking
> links or opening attachments. See the URL nok.it/ext for additional
> information.
>
>
>
> Hi Jean,
>
> thank you - quick replies in line
>
> --- Alex
>
> On 7/30/2024 2:35 AM, Jean Quilbeuf wrote:
>
> Hello Alexander,
> I put some answers inline.
>
>
> -----Original Message-----
> From: Alexander L Clemm <ludwig=[e-mail]>
> Sent: Monday, July 29, 2024 8:22 PM
> To: [e-mail]
> Subject: [netmod] Defining groupings after the fact?
> draft-jouqui-netmod-yang- full-include and the reuse of definitions
>
> Hello Jean, Benoit, Thomas,
>
> After your presentation at IETF 120, I looked at your draft
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fda
> tatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-jouqui-netmod-yang-full-i&da
> ta=05%7C02%7Cshiya.ashraf%40nokia.com%7Cf800a99bb9ed462e8f7108dcb26b
> a815%7C5d4717519675428d917b70f44f9630b0%7C0%7C0%7C638581422668726221
> %7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI
> 6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=NdC02cJcdQPE7hc76ekJos%2B
> UoPjRQ4NEjYfLe4%2Ft%2B0k%3D&reserved=0
> nclude-
> 02.
>
>
> I do have some questions regarding what happens if the embedded
> module is being augmented.  Is the augmentation automatically
> embedded as well; does such embedding need to be explicitly stated?
> Is there a way to augment an embedded module only within the context of
> the embedding module?
>
> Yes, as per the example in the slides: if you want ietf-interfaces
> augmented by ietf-ip, you have to embed them both.
>
> <AC> I.e., you would need to augment the embedding module as well to embed
> the augmentation.  The aumentation of the embedding module would then
> include a new "embed" statement to for the augmentation of the module that
> had been originally embedded. Correct?
>
> On a more general note, it strikes me that there is an increased
> need in reusing definitions.  In various forms, we see this in your
> use cases, in network inventory use cases, in schema-mount, in
> peer-mount.  YANG does not provide good support for that, which is
> somewhat ironic in that it does actually support several constructs
> with reuse and extensibility in mind, from identities to groupings.
> Hopefully the YANG-next effort will go a long ways towards improving
> definition reuse to that the need for after-the-fact bandaids can be
> avoided.
>
> Fully agree, the full embed as defined here should be a keyword in
> YANG-next. Similar constructs exist in protobuf and json-schema for
> instance.
>
> <AC> Cool. </ALEX>
>
> When it comes to reusing parts of definitions, it seems that a lot
> of grief could be avoided if portions that are to be reused would
> have been defined as groupings, which could then be used wherever needed.
> The problem is that the grouping construct is rarely used, so many
> YANG definitions are not available for reuse that otherwise might be.
>
> Grouping does not solve everything, you cannot augment a grouping so any
> augmentation would have to be repeated for each use of the grouping.
> I recommend reading the intro of RFC8528 YANG Schema Mount for a detailed
> description of these reuse issues.
>
> <AC> Correct, you cannot augment a grouping.  However, you can define
> a second grouping and then use both groupings.  I do think that with
> properly designed modules that make extensive use of groupings 99+% of
> reuse scenarios would be covered.  The problem of course that in
> general groupings are used only sparingly and in cases where the need
> for reuse becomes obvious already within the same model.  Once models
> have been defined this way, they cannot be altered after the fact.
> That is one of the shortcomings in YANG today, that it makes it easy
> to define models that are not as reusable as they should.  </AC>
>
> As a thought, it might be useful to introduce a construct that will
> allow to define a _grouping_ after-the-fact, for later reuse.  I.e.,
> allow groupings to be defined in a way that the new grouping embeds
> an existing definition, then simply make use of that grouping.  That
> would seem perhaps cleanest, able to address many of the use cases
> and have the additional advantage that the semantics here will be very
> clear since part of the exising YANG framework.
>
> There is still the augment issue from above, we have it in
> draft-ietf-opsawg-collected-data-manifest when reusing ietf-yang-push which
> augments ietf-subscribed-notifications. All these augments have to be
> rewritten with paths corresponding to the new location of the uses.
>
> <AC> I don't think that would be an issue, actually.  Just declare
> modular, fine grained groupings and use those.  Of course, this is somewhat
> a speculative discussion as YANG is what it is and does not support this
> today.  This discussion probably belongs in YANG next.
> Perhaps I'll put together some slides at some point to illustrate what
> I mean. </AC>
>
> I think the semantics for Schema Mount as defined in RFC8525 is the key to
> reuse the full semantics of YANG (i.e. not only groupings but also
> augmentations, rpcs ...) without having to modify existing modules.
> What we propose in full embed is just to enable a simplified version of
> schema mount, for design time.
>
> Best,
> Jean
>
> <AC> Cheers, Alex </AC>
>
> --- Alex
>
>
>
> _______________________________________________
> netmod mailing list -- [e-mail] To unsubscribe send an email
> to [e-mail]
>
> _______________________________________________
> netmod mailing list -- [e-mail] To unsubscribe send an email to
> [e-mail] _______________________________________________
> netmod mailing list -- [e-mail] To unsubscribe send an email to
> [e-mail]
>
> _______________________________________________
> netmod mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
>
>
> _______________________________________________
> netmod mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
>