[netconf] hackathon project about adaptive-subscription

["maqiufang (A)" <[e-mail]>]

Hi, all
We've proposed a Hackathon project about adaptive-subscription in this IETF meeting, I'd like to thanks all of my teammates who've worked in this project(Kun from Hunan University, Zhixiong from Microsoft, Peng from China Mobile, Wei from China Telecom, etc).
Our next step, is to provide more performance data for adaptive subscription in different use cases.
Any other comments and suggestions are welcome.

As presented yesterday, Hackathon project details are following (hopefully the results can help clarifies why we need adaptive subscription):
We've used gRPC-based telemetry to collect data from access points, and ELK(acronym for three open source tools: Elasticsearch, Logstash, and Kibana) for collecting, indexing and visualization.
Two use cases have been evaluated, one is to stream and aggregate the rssi value from different APs to figure out WiFi roaming events during the movement of terminal, the other is to stream the bytes sent from the AP uplink.
For each use case, we've evaluated the high-rate data streaming, low-rate data streaming, and adaptive-rate data streaming.
And For adaptive-rate data streaming, take the rssi signal data collection use case for example, if the rssi<-65(default threshold for WIFI roaming handover), we stream data every 2 seconds; if the rssi>=-65, we stream data every 30 seconds.

There are 2 ways to evaluate adaptive-subscription, one is client-driven( the client evaluates the condition and modifies the period, YANG-PUSH supports this, e.g., via modify-subscription), the other is server-driven (to install the adaptive policy into the server and let the server monitor the object changes and switch to different periods automatically).

The results show that, for high-rate data collection at 2-second interval, it's easy to identify when a roaming happens and signal degeneration but also at the cost of a great volume of data because you always keep that high-frequency.
And the opposite is true for low-rate data collection, which greatly reduces the data volume but hard to detect important events and data.
For the client-driven adaptive telemetry, switching from low frequency to high cannot capture all important events and data, because the condition evaluation is at such a low-frequency data collection interval(30 seconds in rssi signal data collection use case) that it may miss a lot of important data during that interval.
While for the server-driven adaptive telemetry, the server will evaluate the condition at the end of each high-frequency interval(2 seconds in rssi signal use case), so it will check whether it needs to switch to another frequency every 2 seconds.

To be brief, server-driven adaptive telemetry can capture the operational statistics degradation as much as possible compared with client-driven adaptive telemetry, greatly reduce the data volume(by ~86% on average shown in our evaluation but depends on the selected threshold) and serve as a compromise between data management resource cost and data fidelity for network diagnosis.

Best Regards,
Qiufang