[netmod] Re: Yang Scalability

["Deepak Rajaram (Nokia)" <[e-mail]>]

Hi Italo,

Thanks for your support and I totally agree with your observation to have a common mechanism to deal with scalability/templates.

The idea is to logically cluster the nodes that are applicable to the host device(OLT) under the /interfaces/interface and those of the peripheral devices(ONU's, which could run into thousands) under /onus/onu/interfaces/interface. ie: the internal nodes/structure of interface model(RFC8343) still remains the same but it's applicability changes depending on which device is being referred(either OLT or one of the ONU's). Please note here, we are talking about the OLT hosting the server, so organising/clustering nodes is much better than having everything as a flat interleaved list in order to search/edit applicable nodes.

Regards,
Deepak

From: Italo Busi <[e-mail]>
Sent: Friday, July 26, 2024 2:50 AM
To: Deepak Rajaram (Nokia) <[e-mail]>; [e-mail]
Cc: [e-mail]
Subject: RE: Yang Scalability

You don't often get email from [e-mail]<mailto:[e-mail]>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
Hi Deepak,

I think we are on the same page about the need to support templates. My only observation here is that it would be preferable to have one mechanism defined by IETF and re-used in multiple scenarios rather than different solutions in different environments

I am not sure I understand your point below:

> On the point on re-defining the interface models under a different root, it is not the intention to change the intrinsic nodes of the model, so functionally, it could be re-used. Ie: the interface model is rather augmented to /onus/onu.

From slide 8 in the LS, I have understood that you would like to list the interfaces under some container to “cluster” them:

<somewhere I do not know>/onus/onu/interfaces/interface

However, according to RFC8343, the interfaces are listed under a root:

/interfaces/interface

My concern here is that some interfaces will appear under the /interfaces/interface while other will appear under the <somewhere I do not know>/onus/onu/interfaces/interface thus under different roots

Let me take the opportunity to check why you are thinking that clustering the interfaces as shown in slide 8 is resolving the scalability problems? The number of interfaces look the same in both cases

Italo

From: Deepak Rajaram (Nokia) <[e-mail]<mailto:[e-mail]>>
Sent: giovedì 25 luglio 2024 06:26
To: Italo Busi <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>
Cc: [e-mail]<mailto:[e-mail]>
Subject: RE: Yang Scalability

Hi Italo,

Thanks for your comments, I agree that scalability solutions could be viewed from different perspectives, which could also include at a protocol level(as mentioned in Appendix B), at the same time modular model design could also solve many UC’s, such as avoiding locks like defaults and mandatories when used with templates especially when the deployment is in an embedded environment.

On the point on re-defining the interface models under a different root, it is not the intention to change the intrinsic nodes of the model, so functionally, it could be re-used. Ie: the interface model is rather augmented to /onus/onu.

As a client, I am sure mechanisms like pagination/effective filters will help, at the same time to complement, if the models too provide their two cents, it will further be helpful.

Regards,
Deepak


From: Italo Busi <[e-mail]<mailto:[e-mail]>>
Sent: Thursday, July 25, 2024 5:19 AM
To: Deepak Rajaram (Nokia) <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>
Cc: [e-mail]<mailto:[e-mail]>
Subject: RE: Yang Scalability

You don't often get email from [e-mail]<mailto:[e-mail]>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
Hi all,

I think that the YANG scalability issue should better addressed as a generic issue within the relevant IETF WGs

We have discovered similar issue also in IVY WG when working on the base network inventory model. You can see Appendix B of https://datatracker.ietf.org/doc/html/draft-ietf-ivy-network-inventory-yang and the conclusion was that this is mainly a protocol issue

I think that it would be also worthwhile understanding what issues are implementation specific (and outside the scope of standardization), a YANG modelling specific or a protocol specific.

The idea of using templates to avoid repeating the same set of information on multiple instances look a good idea and valid in general, as a YANG modelling specific issue. IMHO we should further explore what needs to be changed in existing models to support templates.

I have some concerns with the approach of re-defining the interface models under a different root. I think this would defeat one of the major advantages of re-using RFC8343 which is to be able to have a common model for managing all the interfaces of any system.

The scalability concerns here appears to me more a protocol specific issue (therefore I am cc-ing the Netconf WG for further feedbacks)

IMHO, there is a need to have a flexible definition of some filtering criteria (on the server side) when it is needed to retrieve only a limited set of instances on a list, pagination mechanisms (which is already a work in progress within Netconf WG) and/or a more efficient protocol encodings.

Italo

From: Deepak Rajaram (Nokia) <[e-mail]<mailto:[e-mail]>>
Sent: martedì 23 luglio 2024 01:52
To: [e-mail]<mailto:[e-mail]>
Subject: [netmod] Yang Scalability

Dear all

Thanks for the opportunity to present on yang scalability, this is a follow-up after having briefly introduced the real-life YANG scalability and performance challenges layed out in the Broadband Forum liaison.

I would encourage NETMOD participants to go over the slides in the meeting materials section of ietf-120/netmod. slides-120-netmod-10-bbf-liaison-on-management-at-scale-projects<https://datatracker.ietf.org/meeting/120/materials/slides-120-netmod-10-bbf-liaison-on-management-at-scale-projects>

Short summary:

Based on studies conducted by several Broadband Forum meeting participants, it is found that existing standard YANG implementations do not scale up to configurations that contain a very high number of interfaces; for instance in a Passive Optical Network, a single Optical Line Termination (OLT) can easily surpass 30.000 interfaces (i.e. a few per Optical Network Unit). This is a real challenge for network deployments. We are seeing scaling challenges in terms of datastore sizes and datastore manipulations (slow configuration, slow data retrieval).

While a PON network is taken as an example, it’s more than likely this scaling challenge will find its way to other parts of networks as products and industry evolves.

We believe this is something NETMOD needs to address with urgency.

As a result of the study, to address such scalability issues, few salient points were analyzed and translated into following requirements:


  1.  “Clustering” data nodes
  2.  Reducing datastore size by using shared profiles
  3.  Reducing datastore size by using “templates”

Existing ietf-schema-mount (RFC8528<https://www.rfc-editor.org/rfc/rfc8528>) and the new draft of full: embed<https://datatracker.ietf.org/doc/draft-jouqui-netmod-yang-full-include/> definitely prove to be useful for certain aspects, including reusability of modules as-is. Still, in their current form they fall short for overcoming the scalability issues, which we believe can be mitigated using “templates” and profiles.

I expect a more detailed ID will be brought forward explaining the proposal of templates/profiles. In anticipation of this ID, I would welcome the group to go over the slides for more details on the concepts. Any feedback/suggestions are more than welcome 😊

Regards
Deepak