[netmod] Re: Yang Scalability

["Robert Peschi (Nokia)" <[e-mail]>]

Hello Italo,

Thanks for commenting.

> The idea of using templates to avoid repeating the same set of information on multiple
> instances look a good idea and valid in general, as a YANG modelling specific issue.
> IMHO we should further explore what needs to be changed in existing models to support templates.

Yes, it was indeed one of the key message of our presentation slides-120-netmod-10-bbf-liaison-on-management-at-scale-projects<https://datatracker.ietf.org/meeting/120/materials/slides-120-netmod-10-bbf-liaison-on-management-at-scale-projects> that templates would greatly improve the observed scalability issue. We plan to write an RFC to clarify the template mechanism and its implications.

Using templates also means transmitting much less data from the client to the device server, e.g. during a copy-config. Naturally, this would accordingly reduce protocol transmission penalty. It also greatly reduces the footprint of the running data store on the persistent memory of the device.

Best regards,
Robert

From: Italo Busi <Italo.Busi=[e-mail]>
Sent: Thursday, July 25, 2024 1:49 AM
To: Deepak Rajaram (Nokia) <[e-mail]>; [e-mail]
Cc: [e-mail]
Subject: [netmod] Re: Yang Scalability

You don't often get email from italo.busi=[e-mail]<mailto:italo.busi=[e-mail]>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>


CAUTION: This is an external email. Please be very careful when clicking links or opening attachments. See the URL nok.it/ext for additional information.


Hi all,

I think that the YANG scalability issue should better addressed as a generic issue within the relevant IETF WGs

We have discovered similar issue also in IVY WG when working on the base network inventory model. You can see Appendix B of https://datatracker.ietf.org/doc/html/draft-ietf-ivy-network-inventory-yang and the conclusion was that this is mainly a protocol issue

I think that it would be also worthwhile understanding what issues are implementation specific (and outside the scope of standardization), a YANG modelling specific or a protocol specific.

The idea of using templates to avoid repeating the same set of information on multiple instances look a good idea and valid in general, as a YANG modelling specific issue. IMHO we should further explore what needs to be changed in existing models to support templates.

I have some concerns with the approach of re-defining the interface models under a different root. I think this would defeat one of the major advantages of re-using RFC8343 which is to be able to have a common model for managing all the interfaces of any system.

The scalability concerns here appears to me more a protocol specific issue (therefore I am cc-ing the Netconf WG for further feedbacks)

IMHO, there is a need to have a flexible definition of some filtering criteria (on the server side) when it is needed to retrieve only a limited set of instances on a list, pagination mechanisms (which is already a work in progress within Netconf WG) and/or a more efficient protocol encodings.

Italo

From: Deepak Rajaram (Nokia) <[e-mail]<mailto:[e-mail]>>
Sent: martedì 23 luglio 2024 01:52
To: [e-mail]<mailto:[e-mail]>
Subject: [netmod] Yang Scalability

Dear all

Thanks for the opportunity to present on yang scalability, this is a follow-up after having briefly introduced the real-life YANG scalability and performance challenges layed out in the Broadband Forum liaison.

I would encourage NETMOD participants to go over the slides in the meeting materials section of ietf-120/netmod. slides-120-netmod-10-bbf-liaison-on-management-at-scale-projects<https://datatracker.ietf.org/meeting/120/materials/slides-120-netmod-10-bbf-liaison-on-management-at-scale-projects>

Short summary:

Based on studies conducted by several Broadband Forum meeting participants, it is found that existing standard YANG implementations do not scale up to configurations that contain a very high number of interfaces; for instance in a Passive Optical Network, a single Optical Line Termination (OLT) can easily surpass 30.000 interfaces (i.e. a few per Optical Network Unit). This is a real challenge for network deployments. We are seeing scaling challenges in terms of datastore sizes and datastore manipulations (slow configuration, slow data retrieval).

While a PON network is taken as an example, it’s more than likely this scaling challenge will find its way to other parts of networks as products and industry evolves.

We believe this is something NETMOD needs to address with urgency.

As a result of the study, to address such scalability issues, few salient points were analyzed and translated into following requirements:


  1.  “Clustering” data nodes
  2.  Reducing datastore size by using shared profiles
  3.  Reducing datastore size by using “templates”

Existing ietf-schema-mount (RFC8528<https://www.rfc-editor.org/rfc/rfc8528>) and the new draft of full: embed<https://datatracker.ietf.org/doc/draft-jouqui-netmod-yang-full-include/> definitely prove to be useful for certain aspects, including reusability of modules as-is. Still, in their current form they fall short for overcoming the scalability issues, which we believe can be mitigated using “templates” and profiles.

I expect a more detailed ID will be brought forward explaining the proposal of templates/profiles. In anticipation of this ID, I would welcome the group to go over the slides for more details on the concepts. Any feedback/suggestions are more than welcome 😊

Regards
Deepak