[OPSAWG] Review comments (Re: I-D Action: draft-ietf-opsawg-ntf-03.txt)

[Alexander L Clemm <[e-mail]>]

Hello Haoyu and draft authors,

I have reviewed draft-ietf-opsawg-ntf-03, Network Telemetry Framework,
and a number of comments and suggestions for your consideration for
further improvements of the document.  I am aware this is a bit lengthy,
but hopefully you will find at least some of it useful.  FWIW, here goes
(in sequential order in the document, not in order of relevance of the
comment):

- Intoduction (p3):

As a general comment, I noticed that the document tends to conflate (1)
telemetry data, (2) modules of a framework architecture to generate,
collect, and process telemetry data, as well as (3) functionality of
applications or use cases that leverage such framework architecture.  It
would be useful to clearly distinguish between these concepts and call
out their relationship here. 

An example is the sentence "We show how network telemetry can meet ...
network operation requirements, and the challenges each telemetry module
is facing".  It is not clear at this point what "network telemetry"
really refers to:  is it network telemetry data (but that does not meet
network operation requirements by itself, although it contributes to a
solution), or is the framework, but then what does "telemetry module"
refer to which would presumably be part of the framework, not an
independent entity whose challenges are addressed through the framework. 

- Motivation (p4):

For intent, may want to add a reference to
draft-irtf-nmrg-ibn-concepts-definitions

I am not sure if "actionable information" (and the activities used to
translate "network data" into such information) is part of telemetry. 
To me, "network data" referred to here is really the telemetry data. 
Sure, this data will get processed, aggregated, abstracted, etc, but
telemetry is really the "raw data" that fuels all of those activities. 
I think what is needed is a clear definition or introduction of what
"telemetry data" entails, and what will be covered by the framework -
just the framework to generate and collect that data, or any other
applications "on top" that process that data further and use it for
different purposes (which would make this more of a service assurance
framework, not merely a telemetry framework IMHO).  Either way, I think
it would be good to frame the scope more clearly. 

- Section 2.1 (p5):

There is an inline definition of intent that equates it with a policy. 
This is not consistent with the definition in other places.  Please
refer to the definition e.g. in draft-irtf-nmrg-ibn-concepts-definitions

For use cases, I think the two most important ones are missing; I would
suggest adding these and leading with these, actually.  This concerns
Security (e.g., intrusion detection systems analyzing telemetry data to
detect suspicious activities and traffic), as well as Monitoring.  These
are also arguably the most important use cases for e.g. Netflow/IPFIX
today; it is not clear if flow records qualify as telemetry data in your
definition but the document hints at them in several places. 

- Section 2.x

In general, what is missing is a section 2.x that gives a brief overview
of what is considered as "telemetry data".  This is left to the
imagination of the reader.  Just statistics and snapshots of state?  Or
also config data?  Flow records?  What about event records and logs? 
Measurements?  Packets (including sampled ones) stored/copied for
analysis?  Control packets?  All of the above, or if not, what are
things that would not be considered "telemetry data"? 

- Section 2.2

The state of the art includes a lot more than SNMP, CLI, syslog.  At a
minimum, you need to mention flow records here.  Possibly also
measurements, YANG-Push, etc etc. 

- Section 2.3 (p8)
Update reference to YANG-Push (RFC 8641+8639). 

- Section 2.4
Missing here is mention of flow information export (Netflow/IPFIX).
 
It is not clear what the document is doing here.  In some places it
sounds as if it argues that other types of telemetry (which ones?) or
other protocols / techniques (which ones/ for what needs?), but why - I
thought the purpose was to define a framework that describes what pieces
are needed, where different pieces fit, and how they are expected to
interwork, not so much criticize the current state of the art.  To
analyze the state of the art, important pieces are missing and not
mentioned - for example, I am missing a mention of measurement,
including e.g. OWAMP/TWAMP and RFC 6812 (currently missing entirely from
the references, but widely deployed in the industry so should be added). 

It is not clear if In-network processing and action should be included
here.  See also my earlier comment.  This looks like scope creep to
include not just telemetry but service assurance functionality that
would operate on top of it.

The mention of SDN and centralized processing is a bit confusing, its
purpose in this context not clear. 

- Section 3

It would be good to distinguish here between the need for a telemetry
framework vs the need for telemetry data.  The need for the latter is
clear; the need for the former should be explained as well.  

- Section 4.1

Not clear on sub-pub vs pub-sub.  May need an editorial scrub. 

It is not clear why you mention on-request queries.  Do you consider the
ability to request any type of management data as part of the telemetry
framework?  If so, where do you draw the line between a telemetry
framework and a broader management framework (the telemetry framework
becomes fairly all-encompassing at that point)? 

I am not sure I would categorize complex data that has been synthesized
and processed using complex algorithms as "telemetry data".  This goes
again to the question of what is considered telemetry data.  Is it the
raw data generated by the network (that's where I would draw the line),
or does it include any data that can be derived from that data? If it's
the latter, there is a slipper slope - you may need to include also data
derived from correlation with service and other data, learned models,
etc etc - before you know it it will encompass any data, information,
knowledge involving a network.  

- Figure 1 (p 13) 

I don't think the data type relationships as depicted are correct. 
Simple data could be streamed in its own right, and its generation also
triggered through events.  I think what triggers the generation/sending
of data is independent of the type of data itself.  Really, you might
have simple data and complex data (arguably, complex data might not be
telemetry data, but ok), and then different ways to trigger generation
of that data: As part of an event, as part of a subscription
(streaming), when some condition is met, etc etc. 

- Section 4.2.1

There is no section 4.2.2.  Having a single subsection looks a bit
strange.  Suggest to either remove 4.2.1 as a separate subsection, or to
elevate it to a second-level section, or to add a 4.2.2. 

- Section 4.2.1.3.1

Your discussion of methods active, passive, and hybrid covers very
different types of information - anything from sampled packet to flow
records to measurements to OAM.  I think it would be good to
differentiate by type of information.  Also, please consider mentioning
RFC 6812 (IPSLA) in addition to OWAMP/TWAMP. 

- Section 4.3 and Figure 4

While per se there is nothing wrong with what you describe here, and
perhaps it is best to keep things simple and to the basics as there will
be myriads of implementation variations, I think expectations for what
the framework actually entails should be framed a bit better earlier in
the document.  After all this leadup, finally seeing the proposed
framework appears a bit anti-climactic. This pretty much describes best
current practice today, pretty much describing a generic management
agent on a node will provide.  It does not cover aspects such as
end-to-end measurements (covering multiple nodes), any security
components (signing of data to prevent tampering, perhaps), integration
with the real resources and data sources (this starts at the "data
object", what about collecting data across multiple line card etc etc). 
At a minimum, it would be good to state what it is in scope and out of
scope.   

In the tables of the section, as Netconf and YANG are mentioned, SMIv2
should probably be mentioned and referenced along with SNMP as well. 

- Section 5

The difference betwen level 1 (dynamic) and level 2 (interactive)
telemetry is not very clear.  Any telemetry data can be used in a closed
loop; it is not clear why this is called level 3 (you can use level 0
telemetry data for plenty of closed loops). 

- Section 6

For the security considerations, there are a number of additional
possible telemetry attack vectors that could be mentioned here.  E.g.,
attacks aiming at generating telemetry data to exhaust network resources
as well as resources on the node, attacks aimed at falsifying results
and tampering with telemetry, swamping of receivers (for streaming data).  


--- Alex

On 4/13/2020 11:59 AM, [e-mail] wrote:
> A New Internet-Draft is available from the on-line Internet-Drafts directories.
> This draft is a work item of the Operations and Management Area Working Group WG of the IETF.
>
>         Title           : Network Telemetry Framework
>         Authors         : Haoyu Song
>                           Fengwei Qin
>                           Pedro Martinez-Julia
>                           Laurent Ciavaglia
>                           Aijun Wang
> 	Filename        : draft-ietf-opsawg-ntf-03.txt
> 	Pages           : 34
> 	Date            : 2020-04-13
>
> Abstract:
>    Network telemetry is the technology for gaining network insight and
>    facilitating efficient and automated network management.  It engages
>    various techniques for remote data collection, correlation, and
>    consumption.  This document provides an architectural framework for
>    network telemetry, motivated by the network operation challenges and
>    requirements.  As evidenced by some key characteristics and industry
>    practices, network telemetry covers technologies and protocols beyond
>    the conventional network Operations, Administration, and Management
>    (OAM).  It promises better flexibility, scalability, accuracy,
>    coverage, and performance and allows automated control loops to suit
>    both today's and tomorrow's network operation.  This document
>    clarifies the terminologies and classifies the modules and components
>    of a network telemetry system from several different perspectives.
>    The framework and taxonomy help to set a common ground for the
>    collection of related work and provide guidance for related technique
>    and standard developments.
>
>
> The IETF datatracker status page for this draft is:
> https://datatracker.ietf.org/doc/draft-ietf-opsawg-ntf/
>
> There are also htmlized versions available at:
> https://tools.ietf.org/html/draft-ietf-opsawg-ntf-03
> https://datatracker.ietf.org/doc/html/draft-ietf-opsawg-ntf-03
>
> A diff from the previous version is available at:
> https://www.ietf.org/rfcdiff?url2=draft-ietf-opsawg-ntf-03
>
>
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
>
> Internet-Drafts are also available by anonymous FTP at:
> ftp://ftp.ietf.org/internet-drafts/
>
>
> _______________________________________________
> OPSAWG mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/opsawg