[netmod] Re: Defining groupings after the fact? draft-jouqui-netmod-yang-full-include and the reuse of definitions

["Rob Wilton (rwilton)" <[e-mail]>]

I think that there is a balance to how much groupings are used.  E.g., I would regard the OpenConfig models as an example of where groupings are overly used resulting in YANG models where it is impossible to glean the structure.

I’m also not sure whether groupings really help here – e.g., if you put the top level of the module under a grouping, then another module may just want to reuse a subtree.

I’m also slightly concerned about whether this can really be achieved in an extension (which tooling is allowed to ignore).

I.e., it feels like this really need a new (probably major) version of the YANG language – and possible would need a more radical rethink about how to manage and reuse parts of YANG schema.

Regards,
Rob


From: Kent Watsen <[e-mail]>
Date: Friday, 2 August 2024 at 14:21
To: Shiya Ashraf (Nokia) <shiya.ashraf=[e-mail]>
Cc: Alexander L Clemm <ludwig=[e-mail]>, Alexander L Clemm <[e-mail]>, Jean Quilbeuf <[e-mail]>, BOUCADAIR Mohamed IMT/OLN <[e-mail]>, [e-mail] <[e-mail]>, Rob Wilton (rwilton) <[e-mail]>
Subject: Re: [netmod] Defining groupings after the fact? draft-jouqui-netmod-yang-full-include and the reuse of definitions
[CC-ing Med]

I wonder if rfc8047bis should have a recommendation to use groupings extensively?

FWIW, my "client-server” suite of drafts in NETCONF use groupings extensively.  In fact, whenever a data-node is needed, it is always just a container that uses a grouping.

Kent


On Aug 2, 2024, at 4:54 AM, Shiya Ashraf (Nokia) <shiya.ashraf=[e-mail]> wrote:

Hello Alex,

" However, even in that case, an update to the original model is still required - you cannot simply say "let me use these data definitions from that other models", they need to be defined as a grouping"
<Shiya> Oh sure, absolutely. As you pointed out in one of the earlier emails, if this could address 99+% of the cases, why not do it at least for the models which we think will have more chance of getting reused, say for instance, ietf-interfaces, ietf-hardware etc. And this we could do it today in YANG 1.1 and with no extra tools support - which is clearly an advantage over the new embed mechanisms for faster commercial deployments of the solutions.
Isn't it?

Thanks,
Shiya

-----Original Message-----
From: Alexander L Clemm <ludwig=[e-mail]>
Sent: Thursday, August 1, 2024 10:49 PM
To: Shiya Ashraf (Nokia) <[e-mail]>; Alexander L Clemm <[e-mail]>; Jean Quilbeuf <[e-mail]>; [e-mail]
Subject: Re: [netmod] Re: Defining groupings after the fact? draft-jouqui-netmod-yang-full-include and the reuse of definitions

[You don't often get email from ludwig=[e-mail]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

CAUTION: This is an external email. Please be very careful when clicking links or opening attachments. See the URL nok.it/ext for additional information.



Hello Shiya,

re your comment on the "Once models have been defined this way, they cannot be altered after the fact":  Well, I guess as William has pointed out, it is possible to update a model with another, equivalent model which pulls data node definitions into groupings and then uses those groupings.  That would be a compatible change.  The same groupings will then also be free for other models to use.  However, even in that case, an update to the original model is still required - you cannot simply say "let me use these data definitions from that other models", they need to be defined as a grouping (or per Jean's proposal in the draft, you define a new construct that would let you "use" aka embed definitions without the need for a grouping to be defined).

Cheers

--- Alex

On 8/1/2024 2:20 AM, Shiya Ashraf (Nokia) wrote:

Hi Alex,

"<AC> Correct, you cannot augment a grouping.  However, you can define a second grouping and then use both groupings.  I do think that with properly designed modules that make extensive use of groupings 99+% of reuse scenarios would be covered. "
<Shiya> Thanks for bringing this point up and I tend to fully agree here. In fact when I was reading the schema mount RFC where it starts with the short comings of "grouping", I also felt that there could be use-cases where some of these aspects of the groupings can turn out to be its strengths. For eg: for cases where you need greater control on what you want to embed on the mounted tree, for instance, only a selection of the augments from the original module or add new augments only on the embedding context etc. So though schema-mount/full-embed are very good solutions for reusability of existing YANG modules for certain use-cases with its own advantages, for many cases the existing methods based on groupings might do the job and in a much more simpler way.

But then you say: " Once models have been defined this way, they cannot be altered after the fact."
<Shiya> Could you explain more on this? Technically, One can still define a new grouping with all the data nodes that are today in a standard module and then replaces the content of the standard module with a simple uses statement of the new grouping with out causing a backward compatibility issue or any functional change, can’t we ?

Thanks,
Shiya

-----Original Message-----
From: Alexander L Clemm <ludwig=[e-mail]>
Sent: Thursday, August 1, 2024 12:37 AM
To: Jean Quilbeuf <jean.quilbeuf=[e-mail]>;
[e-mail]
Subject: [netmod] Re: Defining groupings after the fact?
draft-jouqui-netmod-yang-full-include and the reuse of definitions

[You don't often get email from ludwig=[e-mail].
Learn why this is important at
https://aka.ms/LearnAboutSenderIdentification ]

CAUTION: This is an external email. Please be very careful when clicking links or opening attachments. See the URL nok.it/ext for additional information.



Hi Jean,

thank you - quick replies in line

--- Alex

On 7/30/2024 2:35 AM, Jean Quilbeuf wrote:

Hello Alexander,
I put some answers inline.


-----Original Message-----
From: Alexander L Clemm <ludwig=[e-mail]>
Sent: Monday, July 29, 2024 8:22 PM
To: [e-mail]
Subject: [netmod] Defining groupings after the fact?
draft-jouqui-netmod-yang- full-include and the reuse of definitions

Hello Jean, Benoit, Thomas,

After your presentation at IETF 120, I looked at your draft
https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fda
tatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-jouqui-netmod-yang-full-i&da
ta=05%7C02%7Cshiya.ashraf%40nokia.com%7Cf800a99bb9ed462e8f7108dcb26b
a815%7C5d4717519675428d917b70f44f9630b0%7C0%7C0%7C638581422668726221
%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI
6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=NdC02cJcdQPE7hc76ekJos%2B
UoPjRQ4NEjYfLe4%2Ft%2B0k%3D&reserved=0
nclude-
02.


I do have some questions regarding what happens if the embedded
module is being augmented.  Is the augmentation automatically
embedded as well; does such embedding need to be explicitly stated?
Is there a way to augment an embedded module only within the context of the embedding module?
Yes, as per the example in the slides: if you want ietf-interfaces augmented by ietf-ip, you have to embed them both.
<AC> I.e., you would need to augment the embedding module as well to embed the augmentation.  The aumentation of the embedding module would then include a new "embed" statement to for the augmentation of the module that had been originally embedded. Correct?

On a more general note, it strikes me that there is an increased
need in reusing definitions.  In various forms, we see this in your
use cases, in network inventory use cases, in schema-mount, in
peer-mount.  YANG does not provide good support for that, which is
somewhat ironic in that it does actually support several constructs
with reuse and extensibility in mind, from identities to groupings.
Hopefully the YANG-next effort will go a long ways towards improving
definition reuse to that the need for after-the-fact bandaids can be avoided.
Fully agree, the full embed as defined here should be a keyword in YANG-next. Similar constructs exist in protobuf and json-schema for instance.
<AC> Cool. </ALEX>

When it comes to reusing parts of definitions, it seems that a lot
of grief could be avoided if portions that are to be reused would
have been defined as groupings, which could then be used wherever needed.
The problem is that the grouping construct is rarely used, so many
YANG definitions are not available for reuse that otherwise might be.
Grouping does not solve everything, you cannot augment a grouping so any augmentation would have to be repeated for each use of the grouping.
I recommend reading the intro of RFC8528 YANG Schema Mount for a detailed description of these reuse issues.
<AC> Correct, you cannot augment a grouping.  However, you can define
a second grouping and then use both groupings.  I do think that with
properly designed modules that make extensive use of groupings 99+% of
reuse scenarios would be covered.  The problem of course that in
general groupings are used only sparingly and in cases where the need
for reuse becomes obvious already within the same model.  Once models
have been defined this way, they cannot be altered after the fact.
That is one of the shortcomings in YANG today, that it makes it easy
to define models that are not as reusable as they should.  </AC>

As a thought, it might be useful to introduce a construct that will
allow to define a _grouping_ after-the-fact, for later reuse.  I.e.,
allow groupings to be defined in a way that the new grouping embeds
an existing definition, then simply make use of that grouping.  That
would seem perhaps cleanest, able to address many of the use cases
and have the additional advantage that the semantics here will be very clear since part of the exising YANG framework.
There is still the augment issue from above, we have it in draft-ietf-opsawg-collected-data-manifest when reusing ietf-yang-push which augments ietf-subscribed-notifications. All these augments have to be rewritten with paths corresponding to the new location of the uses.
<AC> I don't think that would be an issue, actually.  Just declare modular, fine grained groupings and use those.  Of course, this is somewhat a speculative discussion as YANG is what it is and does not support this today.  This discussion probably belongs in YANG next.
Perhaps I'll put together some slides at some point to illustrate what
I mean. </AC>

I think the semantics for Schema Mount as defined in RFC8525 is the key to reuse the full semantics of YANG (i.e. not only groupings but also augmentations, rpcs ...) without having to modify existing modules.
What we propose in full embed is just to enable a simplified version of schema mount, for design time.

Best,
Jean
<AC> Cheers, Alex </AC>

--- Alex



_______________________________________________
netmod mailing list -- [e-mail] To unsubscribe send an email
to [e-mail]
_______________________________________________
netmod mailing list -- [e-mail] To unsubscribe send an email to
[e-mail] _______________________________________________
netmod mailing list -- [e-mail] To unsubscribe send an email to
[e-mail]
_______________________________________________
netmod mailing list -- [e-mail]
To unsubscribe send an email to [e-mail]