Re: [netmod] [CCAMP][TEAS][NETMOD] Efficiency issues with YANG model data in integration

[tom petch <[e-mail]>]

Trimming the distribution to NETMOD (if I send a query to everyone then everyone may think that everyone  else will respond so no-one does)

From: netmod <[e-mail]> on behalf of yuchaode <yuchaode=[e-mail]>
Sent: 15 March 2022 01:30
To: '[e-mail]'; '[e-mail]'; '[e-mail]'
Cc: Fatai Zhang

I have a question about YANG data model efficiency issues discovered during integration with OSS.
This issue is also mentioned in section2.2 of draft-yg3bp-ccamp-network-inventory-yang that we are planning to present at the CCAMP WG during IETF 113.

YANG data models define schema trees such as:

module: example
   +--rw root
      +--rw leaf-a?   string
      +--rw list-b* [leaf-c]
      |  +--rw leaf-c    string
      |  +--rw list-d* [leaf-e]
      |     +--rw leaf-e    string
      |     +--rw list-f* [leaf-g]
      |        +--rw leaf-g    string
      |        +--rw leaf-h?   string
      +--rw list-i* [leaf-j]
         +--rw leaf-j    string
         +--rw leaf-k?   string

From the point of view of data model definition, it is convenient to define objects and their relationship using a tree structure.

However, from a practical perspective, relational databases, in which these objects are saved in different tables, are widely used by network controllers and OSS/BSS. In this type of implementations, the model data need to be cut into several pieces of small object data for management and this is causing some efficiency issues with the integration between systems.

<tp>
When the data definition language that we know as YANG was being specified, the question did arise of how object-oriented it should be and the consensus was that it should not be.  Seeking to retrofit such a concept might be a bit like finding late in the day that the class definitions that have been chosen do not go high up enough the tree:-(

An interesting idea but I am not sure how feasible it will prove to be.

Tom Petch

For example, in full data synchronization scenario, the YANG server needs to assemble the data of each small object into a tree structure based on the YANG model defined for the communication with its YANG client(s). The YANG client then disassembles the data into its own storage structure. This assembly-disassembly process is actually time consuming, especially when the storage structures between the upper and lower system are consistent, they don’t need this assembly-disassembly process at all. This efficiency issue is more prominent in large scale networks.

RESTCONF can also support querying small objects. For example, in the above YANG model, the YANG server can provide the list of list-f instances ({{restconf}}/data/example:root/list-b={leaf-c}/list-d={leaf-e}/list-f) to a YANG client, with no need to perform any assembly-disassembly process.

However, since list-f is a child list under list-d and list-b, if you want to get access the list-f list, the list-b’s identifier (leaf-c) and the list-d’s identifier (leaf-e) shall be specified in the URI. Therefore, in the full data synchronization scenario, a loop is required to get all the elements of list-f. If there are 100 list-b instances and there’re 100 list-d instances under each list-b instance, then 100*100 requests are needed to get all of the list-f instances.
Assume that there are only 10 instances by average in each list-d instance, there are totally 100,000 list-f instances. If, instead, it would possible to retrieve all the list-f instances without specifying the list-f’s parent and grandparent’s identifier, but using RESTFUL’s pagination, each RESTFUL request, can provide far more than 10 list-f instances, regardless of whether they belong to a same list-d or list-b instance or not. If RESTFUL interface can return 100 list-f instances per request, only 1000 requests are required, one tenth of RESTCONF interface. This approach will reduce the time taken by the integration.

So my question is whether, to address this efficiency issue, the only viable option is to define the YANG model with a more flat structure, as shown below:

module: example
   +--rw root
      +--rw root-summay-information
      |  +--rw leaf-a?             string
      |  +--rw contained-list-b*   string
      |  +--rw contianed-list-i*   string
      +--rw list-b* [leaf-c]
      |  +--rw leaf-c              string
      |  +--rw contained-list-d*   string
      +--rw list-d* [leaf-c leaf-e]
      |  +--rw leaf-c              leafref  (path "/exmp:root/exmp:list-b/exmp:leaf-c")
      |  +--rw leaf-e              string
      |  +--rw contained-list-f*   string
      +--rw list-f* [leaf-c leaf-e leaf-g]
      |  +--rw leaf-c    leafref  (path "/exmp:root/exmp:list-b/exmp:leaf-c")
      |  +--rw leaf-e    leafref  (path "/exmp:root/exmp:list-d/exmp:leaf-e")
      |  +--rw leaf-g    string
      |  +--rw leaf-h?   string
      +--rw list-i* [leaf-j]
         +--rw leaf-j    string
         +--rw leaf-k?   string

The leaf-list object contained-list-* is a list of identifier reference to list-* instance. With this kind of modeling, all the list-f instances in the whole network can be retrieved without the need to specify its parent’s and grandparent’s identifiers in URI.

We would appreciate receiving your comments and suggestions on whether there are other ways to resolve these integration efficiency issues.

Thank you!

B.R.
Chaode

玉朝德  Tel: [phone]
华为技术有限公司
[Huawei]