[netmod] Re: Yang Scalability

["Deepak Rajaram (Nokia)" <[e-mail]>]

Hi Quifang,

Thanks a lot for your feedback and comments. Please find inline my replies prefixed with >>DR

Regards,
Deepak

From: maqiufang (A) <[e-mail]>
Sent: Wednesday, July 24, 2024 7:58 AM
To: Deepak Rajaram (Nokia) <[e-mail]>; [e-mail]
Subject: RE: Yang Scalability

You don't often get email from [e-mail]<mailto:[e-mail]>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification>
Hi, Deepak,

Thanks a lot for bring this to IETF, I’d like to comment on the #3 requirement you mentioned below, regarding the use of templates, I believe this is related to your slides from #11 to #14.

First, I think the default and mandatory statements defined in the YANG modules should follow the way it is supposed to be, it doesn’t make a lot of sense to me to remove them simply because we have issues when using them in template mechanism. Slide #13 (e.g., “Hence solving these issues requires new modules without mandatories and defaults”) seems to indicate that you’re finding ways to remove the default and mandatory statements in published modules, but I feel this should not be right way to go.
>>DR :The idea is definitely not to remove the defaults and mandatory statements in published modules, rather it is to make it more modular while still retaining the original functionality and structure. In other words, the defaults and mandatories where originally defined in std should be preserved when data nodes are used for configuring templates but not when configuring instances .

I guess I don’t really understand the issues you are describing here, I fully agree that the results should be a merge of the template and values explicitly provided, with the latter ones taking precedence over the template. But for the default configuration, the slides mentioned “A default statement will (silently !) overrule a different value coming from the template if not explicitly configured to repeat the template value.” To me, this is not the always the case, and it would probably depend on the with-defaults basic mode defined in RFC 6243 which defines how a server handles the default data. For example, if it is “report-all” basic mode, the server would consider every data node with a schedule default value to exist, and then it would probably override a template value silently; but if the server uses a “explicit” basic mode,  it won’t consider the default data to exist until it is explicitly provided by the client, so it feels to me that what is configured in the template should become the final merged result.
>>DR: Wrt to defaults handling, I agree, it would depend on the way servers interpret the defaults. The point of overriding comes only if it interprets the default data to be always present. There are also many use-cases where a default value in the server proves to be useful, however when it comes to templates, it is not just about how the server returns a value it is about how config merge happens between templates and instance configurations.


For mandatory node, could you please clarify a little bit on why “A mandatory statement forces an ONU instance to repeat a data node already configured in the template”? For validation purpose? But I think it is the merged results that should be subject to validation.
>>DR: Yes, validation is done only on the result, but in the ONU instance, there would be redundant configuration leading to unnecessary data which can be avoided.

I am sorry if I have any misunderstandings, I would wait for your I-D and see if that helps understand it better. Thanks.
>>DR: One of the objective is to explain the template mechanism in a upcoming draft  to have it discussed in detail in IETF. I believe that would provide more context .

Best Regards,
Qiufang
From: Deepak Rajaram (Nokia) <deepak.rajaram=[e-mail]<mailto:deepak.rajaram=[e-mail]>>
Sent: Tuesday, July 23, 2024 4:52 PM
To: [e-mail]<mailto:[e-mail]>
Subject: [netmod] Yang Scalability

Dear all

Thanks for the opportunity to present on yang scalability, this is a follow-up after having briefly introduced the real-life YANG scalability and performance challenges layed out in the Broadband Forum liaison.

I would encourage NETMOD participants to go over the slides in the meeting materials section of ietf-120/netmod. slides-120-netmod-10-bbf-liaison-on-management-at-scale-projects<https://datatracker.ietf.org/meeting/120/materials/slides-120-netmod-10-bbf-liaison-on-management-at-scale-projects>

Short summary:

Based on studies conducted by several Broadband Forum meeting participants, it is found that existing standard YANG implementations do not scale up to configurations that contain a very high number of interfaces; for instance in a Passive Optical Network, a single Optical Line Termination (OLT) can easily surpass 30.000 interfaces (i.e. a few per Optical Network Unit). This is a real challenge for network deployments. We are seeing scaling challenges in terms of datastore sizes and datastore manipulations (slow configuration, slow data retrieval).

While a PON network is taken as an example, it’s more than likely this scaling challenge will find its way to other parts of networks as products and industry evolves.

We believe this is something NETMOD needs to address with urgency.

As a result of the study, to address such scalability issues, few salient points were analyzed and translated into following requirements:


  1.  “Clustering” data nodes
  2.  Reducing datastore size by using shared profiles
  3.  Reducing datastore size by using “templates”

Existing ietf-schema-mount (RFC8528<https://www.rfc-editor.org/rfc/rfc8528>) and the new draft of full: embed<https://datatracker.ietf.org/doc/draft-jouqui-netmod-yang-full-include/> definitely prove to be useful for certain aspects, including reusability of modules as-is. Still, in their current form they fall short for overcoming the scalability issues, which we believe can be mitigated using “templates” and profiles.

I expect a more detailed ID will be brought forward explaining the proposal of templates/profiles. In anticipation of this ID, I would welcome the group to go over the slides for more details on the concepts. Any feedback/suggestions are more than welcome 😊

Regards
Deepak