Re: [OPSAWG] Review of draft-ietf-opsawg-ntf

[Haoyu Song <[e-mail]>]

Hi Adrian,

Thanks for the detailed review. I'll update the draft accordingly. Meanwhile I'd like to also request all the other authors and WG members to help review and improve this document, so we can have a new version before next IETF meeting.

Haoyu

-----Original Message-----
From: Adrian Farrel <[e-mail]> 
Sent: Friday, September 27, 2019 11:30 AM
To: [e-mail]; [e-mail]
Subject: Review of draft-ietf-opsawg-ntf

Hi authors, WG,

I needed to read this document to check out a couple of things, and so here is a review that I collected along the way. I hope it is useful.

In general, I found the document a help collection of thoughts on the space and I encourage more work to refine it.

Best regards,

Adrian

===

I think that the Introduction launches in with the use of two terms that could use just a little more explanation. They are "Network Visibility"
and "Network Telemetry".

You don't need pages of text to explain them, but just very quick context-setting text. Such as...

   Network visibility is the ability of management tools to see the
   state and behavior of a network.  It is essential for successful
   network operation.

   Network telemetry is the process of measuring, recording, and
   distributing information about the behavior of a network.  Network
   telemetry has been considered as an ideal means to gain sufficient
   network visibility with better flexibility, scalability, accuracy,
   coverage, and performance than conventional OAM technologies.

---

Although you expand "OAM" in the Abstract, you need to also expand it on first use in the body of the text.

---

Section 1

s/technique/techniques/

---

I don't think you use any normative language, so you can drop section
1.1 and the two references.

---

The start of Section 2 is a bit abrupt.

   Thanks to the advance of the computing and storage technologies,
   today's big data analytics gives network operators an unprecedented
   opportunity to gain network insights and move towards network
   autonomy.

While this is sort of true, it steps over the fact that "big data" has to be collected (where collection means both recorded and moved into a single dataset).

So, I would break this down into:
- what is big data?
- what development in analytic tools is now meaning can be done with
  big data
- how this might be applicable to networks if the right data can be
  recorded, collected, and analysed.

You get to this last point nicely in paragraph 3, so you only need to set that up.

---
   
You need to check for all abbreviations being expanded on first use.   
I found:
ROI
CAPEX
RIB
ACL
MIB
QoS
CPU
GPB

---
   
2.2

s/are lack of formal/lack a formal/

---

2.3

This is a good collection of terms and explanations. Thanks. It would be even better if you were able to add (informative) references to point the reader at places to go for more background information.

Maybe the explanation of "YANG" should include an expansion of the abbreviation.

---

I don't find the inclusion of "YANG FSM" in this document convincing.
You only include it in the terminology section and in Figures 5 and 6.
But you don't have any explanation of what the term means, nor any indication of how it is relevant to this document.

---

2.4

draft-kumar-rtgwg-grpc-protocol may not be the best reference for gRPC.
I know you're only looking for an informative reference, but this draft expired 30 months ago.

I don't have any suggestions for a better reference, but there is probably something on https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgrpc.io&amp;data=02%7C01%7Chaoyu.song%40futurewei.com%7C5cabc98d38124de1042908d74378b3b4%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637052058022774410&amp;sdata=mdqzSprio5yJNZbDP6NHid437Olw54YJMT9qy5mYwVY%3D&amp;reserved=0

---

2.4

I think I would add to the second list of bullets in this section to describe "in-network data aggregation/correlation". One of the applications AI and other "smart algorithmics" is to improve the collection and reporting of data from the network. That is, network devices and aggregation points can work out which events and what data needs to be stored, reported, or discarded thus reducing the load on the central collection and processing points while still ensuring that the right information is ready to be processed in a timely way.

I might also add "in-network processing". That is, there is not a necessary step to gather all information to a central point so that it can be processed and actions taken - it is possible for some processing to be done in the network, and actions taken more locally and more responsively.

This might link in to section 4.3 for the discussion of "Data Query, Analysis, and Storage" because it is not inconceivable that the analysis and storage will be distributed, and that the queries may be hierarchical.  That wouldn't require a change to the structure of Figure 4.

---

Section 3

   So far, some telemetry related work has been done within IETF.
   However, the work is fragmented and scattered in different working
   groups.  The lack of coherence makes it difficult to assemble a
   comprehensive network telemetry system and causes repetitive and
   redundant work.

This will not age well. Maybe say instead...

   A telemetry framework collects together all of the telemetry-related
   work from different sources and working groups within the IETF.  This
   makes it possible to assemble a comprehensive network telemetry 
   system and to avoid repetitious or redundant work.

---

4.1

s/mutual exclusive/mutually exclusive/

---

4.2

   as shown in Figure 2.  Therefore, we categorize the
   network telemetry into four distinct modules.

Figure 2 then shows five boxes. So either...

   as shown in Figure 2.  Therefore, we categorize the
   network telemetry into five distinct modules.

...or...

   as shown in Figure 2.  Therefore, we categorize the
   network telemetry into four distinct modules together with Network
   Operation Applications.

---

It would be helpful if there was some text to accompany Figure 3 to announce and explain the six row categories.

---

Figure 5 and 6 you have "IOAM" and "In-situ OAM". Can you be consistent.

---

Figure 5 has "Custom Data" is this "Complex Data"?

---

Something doesn't see right in section 5. You talk about the "telemetry data" being, for example "static". I don't think it would be helpful if the data was static. You possibly mean that the counters can or cannot be configured. Or something like that.

---

You don't seem to use the reference to RFC 1157.

---

It would be helpful to note at the top of the Appendix that it is non- normative.