[netmod] Re: Defining groupings after the fact? draft-jouqui-netmod-yang-full-include and the reuse of definitions

[Kent Watsen <[e-mail]>]

[CC-ing Med]

I wonder if rfc8047bis should have a recommendation to use groupings extensively?

FWIW, my "client-server” suite of drafts in NETCONF use groupings extensively.  In fact, whenever a data-node is needed, it is always just a container that uses a grouping.

Kent

> On Aug 2, 2024, at 4:54 AM, Shiya Ashraf (Nokia) <shiya.ashraf=[e-mail]> wrote:
> 
> Hello Alex,
> 
> " However, even in that case, an update to the original model is still required - you cannot simply say "let me use these data definitions from that other models", they need to be defined as a grouping"
> <Shiya> Oh sure, absolutely. As you pointed out in one of the earlier emails, if this could address 99+% of the cases, why not do it at least for the models which we think will have more chance of getting reused, say for instance, ietf-interfaces, ietf-hardware etc. And this we could do it today in YANG 1.1 and with no extra tools support - which is clearly an advantage over the new embed mechanisms for faster commercial deployments of the solutions. 
> Isn't it?
> 
> Thanks,
> Shiya
> 
> -----Original Message-----
> From: Alexander L Clemm <ludwig=[e-mail]> 
> Sent: Thursday, August 1, 2024 10:49 PM
> To: Shiya Ashraf (Nokia) <[e-mail]>; Alexander L Clemm <[e-mail]>; Jean Quilbeuf <[e-mail]>; [e-mail]
> Subject: Re: [netmod] Re: Defining groupings after the fact? draft-jouqui-netmod-yang-full-include and the reuse of definitions
> 
> [You don't often get email from ludwig=[e-mail]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
> 
> CAUTION: This is an external email. Please be very careful when clicking links or opening attachments. See the URL nok.it/ext for additional information.
> 
> 
> 
> Hello Shiya,
> 
> re your comment on the "Once models have been defined this way, they cannot be altered after the fact":  Well, I guess as William has pointed out, it is possible to update a model with another, equivalent model which pulls data node definitions into groupings and then uses those groupings.  That would be a compatible change.  The same groupings will then also be free for other models to use.  However, even in that case, an update to the original model is still required - you cannot simply say "let me use these data definitions from that other models", they need to be defined as a grouping (or per Jean's proposal in the draft, you define a new construct that would let you "use" aka embed definitions without the need for a grouping to be defined).
> 
> Cheers
> 
> --- Alex
> 
> On 8/1/2024 2:20 AM, Shiya Ashraf (Nokia) wrote:
>> Hi Alex,
>> 
>> "<AC> Correct, you cannot augment a grouping.  However, you can define a second grouping and then use both groupings.  I do think that with properly designed modules that make extensive use of groupings 99+% of reuse scenarios would be covered. "
>> <Shiya> Thanks for bringing this point up and I tend to fully agree here. In fact when I was reading the schema mount RFC where it starts with the short comings of "grouping", I also felt that there could be use-cases where some of these aspects of the groupings can turn out to be its strengths. For eg: for cases where you need greater control on what you want to embed on the mounted tree, for instance, only a selection of the augments from the original module or add new augments only on the embedding context etc. So though schema-mount/full-embed are very good solutions for reusability of existing YANG modules for certain use-cases with its own advantages, for many cases the existing methods based on groupings might do the job and in a much more simpler way.
>> 
>> But then you say: " Once models have been defined this way, they cannot be altered after the fact."
>> <Shiya> Could you explain more on this? Technically, One can still define a new grouping with all the data nodes that are today in a standard module and then replaces the content of the standard module with a simple uses statement of the new grouping with out causing a backward compatibility issue or any functional change, can’t we ?
>> 
>> Thanks,
>> Shiya
>> 
>> -----Original Message-----
>> From: Alexander L Clemm <ludwig=[e-mail]>
>> Sent: Thursday, August 1, 2024 12:37 AM
>> To: Jean Quilbeuf <jean.quilbeuf=[e-mail]>; 
>> [e-mail]
>> Subject: [netmod] Re: Defining groupings after the fact? 
>> draft-jouqui-netmod-yang-full-include and the reuse of definitions
>> 
>> [You don't often get email from ludwig=[e-mail]. 
>> Learn why this is important at 
>> https://aka.ms/LearnAboutSenderIdentification ]
>> 
>> CAUTION: This is an external email. Please be very careful when clicking links or opening attachments. See the URL nok.it/ext for additional information.
>> 
>> 
>> 
>> Hi Jean,
>> 
>> thank you - quick replies in line
>> 
>> --- Alex
>> 
>> On 7/30/2024 2:35 AM, Jean Quilbeuf wrote:
>>> Hello Alexander,
>>> I put some answers inline.
>>> 
>>>> -----Original Message-----
>>>> From: Alexander L Clemm <ludwig=[e-mail]>
>>>> Sent: Monday, July 29, 2024 8:22 PM
>>>> To: [e-mail]
>>>> Subject: [netmod] Defining groupings after the fact?
>>>> draft-jouqui-netmod-yang- full-include and the reuse of definitions
>>>> 
>>>> Hello Jean, Benoit, Thomas,
>>>> 
>>>> After your presentation at IETF 120, I looked at your draft
>>>> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fda
>>>> tatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-jouqui-netmod-yang-full-i&da
>>>> ta=05%7C02%7Cshiya.ashraf%40nokia.com%7Cf800a99bb9ed462e8f7108dcb26b
>>>> a815%7C5d4717519675428d917b70f44f9630b0%7C0%7C0%7C638581422668726221
>>>> %7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI
>>>> 6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=NdC02cJcdQPE7hc76ekJos%2B
>>>> UoPjRQ4NEjYfLe4%2Ft%2B0k%3D&reserved=0
>>>> nclude-
>>>> 02.
>>>> 
>>>> 
>>>> I do have some questions regarding what happens if the embedded 
>>>> module is being augmented.  Is the augmentation automatically 
>>>> embedded as well; does such embedding need to be explicitly stated?
>>>> Is there a way to augment an embedded module only within the context of the embedding module?
>>> Yes, as per the example in the slides: if you want ietf-interfaces augmented by ietf-ip, you have to embed them both.
>> <AC> I.e., you would need to augment the embedding module as well to embed the augmentation.  The aumentation of the embedding module would then include a new "embed" statement to for the augmentation of the module that had been originally embedded. Correct?
>>>> On a more general note, it strikes me that there is an increased 
>>>> need in reusing definitions.  In various forms, we see this in your 
>>>> use cases, in network inventory use cases, in schema-mount, in 
>>>> peer-mount.  YANG does not provide good support for that, which is 
>>>> somewhat ironic in that it does actually support several constructs 
>>>> with reuse and extensibility in mind, from identities to groupings.
>>>> Hopefully the YANG-next effort will go a long ways towards improving 
>>>> definition reuse to that the need for after-the-fact bandaids can be avoided.
>>> Fully agree, the full embed as defined here should be a keyword in YANG-next. Similar constructs exist in protobuf and json-schema for instance.
>> <AC> Cool. </ALEX>
>>>> When it comes to reusing parts of definitions, it seems that a lot 
>>>> of grief could be avoided if portions that are to be reused would 
>>>> have been defined as groupings, which could then be used wherever needed.
>>>> The problem is that the grouping construct is rarely used, so many 
>>>> YANG definitions are not available for reuse that otherwise might be.
>>> Grouping does not solve everything, you cannot augment a grouping so any augmentation would have to be repeated for each use of the grouping.
>>> I recommend reading the intro of RFC8528 YANG Schema Mount for a detailed description of these reuse issues.
>> <AC> Correct, you cannot augment a grouping.  However, you can define 
>> a second grouping and then use both groupings.  I do think that with 
>> properly designed modules that make extensive use of groupings 99+% of 
>> reuse scenarios would be covered.  The problem of course that in 
>> general groupings are used only sparingly and in cases where the need 
>> for reuse becomes obvious already within the same model.  Once models 
>> have been defined this way, they cannot be altered after the fact.  
>> That is one of the shortcomings in YANG today, that it makes it easy 
>> to define models that are not as reusable as they should.  </AC>
>>>> As a thought, it might be useful to introduce a construct that will 
>>>> allow to define a _grouping_ after-the-fact, for later reuse.  I.e., 
>>>> allow groupings to be defined in a way that the new grouping embeds 
>>>> an existing definition, then simply make use of that grouping.  That 
>>>> would seem perhaps cleanest, able to address many of the use cases 
>>>> and have the additional advantage that the semantics here will be very clear since part of the exising YANG framework.
>>> There is still the augment issue from above, we have it in draft-ietf-opsawg-collected-data-manifest when reusing ietf-yang-push which augments ietf-subscribed-notifications. All these augments have to be rewritten with paths corresponding to the new location of the uses.
>> <AC> I don't think that would be an issue, actually.  Just declare modular, fine grained groupings and use those.  Of course, this is somewhat a speculative discussion as YANG is what it is and does not support this today.  This discussion probably belongs in YANG next.
>> Perhaps I'll put together some slides at some point to illustrate what 
>> I mean. </AC>
>>> I think the semantics for Schema Mount as defined in RFC8525 is the key to reuse the full semantics of YANG (i.e. not only groupings but also augmentations, rpcs ...) without having to modify existing modules.
>>> What we propose in full embed is just to enable a simplified version of schema mount, for design time.
>>> 
>>> Best,
>>> Jean
>> <AC> Cheers, Alex </AC>
>>>> --- Alex
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> netmod mailing list -- [e-mail] To unsubscribe send an email 
>>>> to [e-mail]
>> _______________________________________________
>> netmod mailing list -- [e-mail] To unsubscribe send an email to 
>> [e-mail] _______________________________________________
>> netmod mailing list -- [e-mail] To unsubscribe send an email to 
>> [e-mail]
> _______________________________________________
> netmod mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]