Re: [lmap] draft-ietf-lmap-framework-08: AD review - part 1

[<[e-mail]>]

Benoit,
Thanks very much for the review - some comments in line
Best wishes
phil

From: lmap [mailto:[e-mail]] On Behalf Of Benoit Claise
Sent: 09 November 2014 07:43
To: [e-mail]; [e-mail]
Cc: Paul Aitken; Aamer Akhter
Subject: [lmap] draft-ietf-lmap-framework-08: AD review - part 1

Dear all,

Here is my AD review.
Note: I haven't reviewed yet the section 7 "security considerations" and section 8 "privacy considerations", but thought I should anyway send this before the meeting this week.
Note2: if some points were discussed already, don't hesitate to let me know.

A first impression is that there are some repetitions in the draft. It seems like some sections, written by different persons, were glued together. Reading this draft again with a fresh mind would highlight this.

[phil] ok will try to reduce. I think some redundancy is OK. for example, I think there should be some repetition between S1 & 2 (trying to provide an introduction, for people who want to read no further or haven't come across measurement stuff before); S3 (summarises terminology) and S5 (details the protocol model).


One important issue, which I mentioned already during the interim meeting: the lack of passive monitoring aspects in the draft.

   Section 2



   The MA may

   generate Measurement Traffic and measure some metric associated with

   its transfer, or the MA may observe existing traffic, or there may be

   some kind of hybrid of these two possibilities.


So the document speaks a little bit about active and passive monitoring. Good, because this is mentioned in the charter:
Both active and passive measurements are in scope, although there may be differences in their applicability to specific use cases, or in the security measures needed according to the threats specific to each measurement category. LMAP will not standardize performance metrics.
I recall the charter discussion. I specifically asked: "do you want to have passive monitoring in the charter? Do you understand all the consequences (for example, traffic filters, for observed packet/flow, which may imply RFC6728)?". Both answers were "yes".
So, the draft clearly misses the passive monitoring aspects, starting with the primary figure in this framework, figure 1, which only covers the active monitoring

[phil] I'll mainly leave Paul & Aamer to handle this comment.
In terms of Figure 1, I can add a pointer to Appendix A4 which shows a passive monitoring case.
We can think about how to show passive monitoring in Fig 1 (not trivial - it doesn't show all active monitoring cases either, as doesn't show the scenario with a second MA instead of a M Peer).



Some other issues, simply listed in order
-

   These devices could be software based

   agents on PCs, embedded agents in consumer devices (e.g.  Blu-ray

   players),



Does a blu-ray player need a broadband test?



[phil[ >tablets?



-

 o  Diversity - a measurement system should handle different types of

      Measurement Agents - for example Measurement Agents may come from

      different vendors, be in wired and wireless networks, be able to

      execute different sorts of Measurement Task and be on devices with

      IPv4 or IPv6 addresses.



Is "different types" the right term?

You have introduced the term "capabilities"

[phil] don't mind. (The term capabilities hasn't yet been introduced)

-

   It is also useful to define a registry

   for commonly-used Metrics [I-D.manyfolks-ippm-metric-registry<http://tools.ietf.org/html/draft-ietf-lmap-framework-08#ref-I-D.manyfolks-ippm-metric-registry>] so

   that a Metric with its associated Measurement Method can be referred

   to simply by its identifier in the registry.


The new draft is http://tools.ietf.org/html/draft-ietf-ippm-metric-registry

-
OLD:

   Finally we introduce several components that are outside the scope of

   initial LMAP work and will be provided through existing protocols or

   applications.
NEW:

   Finally we introduce several components that are outside the scope of

   initial LMAP work (see in figure 1 "out of scope" and will be provided

   through existing protocols or applications.
Then I would list four bullet points for the next paragraph.
Otherwise it's not obvious which components you speak about
[phil] ok

- figure 1
    * Since the above mentions the Results repository  (The data analysis tools receive the results from the Collector or via the Results repository. ), the figure should replace repository by Results repository
    * "IPPM scope" is wrong (see my first remark)
[phil] sorry, I don't understand. Would you prefer something like "Scope of eg IPPM"?

    * Mention the Control Protocol and Report Protocol, or Control Channel/Report Channel. Note: if you replace Instruction by Control Protocol, then the arrow must be bidirectional (according to the Control Protocol definition)
[phil] Prefer to keep a mention of instruction and report.
Suggested change: to "Instruction (over Control Channel)" and "Report (over Report Channel)" and keep the arrows uni-directional.
-

   Control Channel: A Channel between a Controller and a MA over which

   Instruction Messages and Capabilities, Failure and Logging

   Information are sent.



   Control Protocol: The protocol delivering Instruction(s) from a

   Controller to a Measurement Agent.  It also delivers Capabilities,

   Failure and Logging Information from the Measurement Agent to the

   Controller.  It can also be used to update the MA's Configuration.



Do you want to mention that the Control Protocol runs on the Control Channel?



-

   Measurement Method: The process for assessing the value of a Metric;

   the process of measuring some performance or reliability parameter

   associated with the transfer of traffic; where this process involves

   multiple MAs or MPs, each may perform different roles.



I don't understand what "where this process involves multiple MAs or MPs, each may perform different roles." adds to the def.

[phil] ok.



-

  Report Channel: A Channel between a Collector and a MA over which

   Report messages are sent.



Report messages -> Report Messages





-

4.2

 An operator may have several Controllers, perhaps with a Controller

   for different types of MA (home gateways, tablets) or location

   (Ipswich, Edinburgh).



BT specific, maybe? :-)

[phil] I'll add Diegem!



- Section 5.

Here is yet another example of repetition

[phil] ok, will shorten these (they're basically introducing the sub-sections of S5)

(see the last 3 bullets below. Why do you have to repeat what the Control Protocol does)



   An LMAP system goes through the following phases:



   o  a Bootstrapping process before the MA can take part in the other

      three phases



   o  a Control Protocol, which delivers Instruction Messages from a

      Controller to a MA, detailing what Measurement Tasks the MA should

      perform and when, and how it should report the Measurement

      Results.  It also delivers Capabilities, Failure and logging

      Information from a MA to its Controller.  Finally, it allows the

      Controller to update the MA's Configuration.



   o  the actual Measurement Tasks, which measure some performance or

      reliability parameter(s) associated with the transfer of packets.



      The LMAP work does not define Metrics and Measurement Methods,

      these are defined elsewhere (e.g.  IPPM).



   o  a Report Protocol, which delivers Reports from a MA to a

      Collector.  The Report contains the Measurement Results.





-

Would be nice to have "measurement system" in the terminology section.

It's mentioned so many times in the draft



[phil] with trepidation... since terminology always provokes excitement...

Measurement System: the set of LMAP-defined and related components that are operated by a single organisation, for the purpose of measuring performance aspects of the network.





-

5.2.  Control Protocol



   The primary purpose of the Control Protocol is to allow the

   Controller to configure a Measurement Agent with an Instruction about

   what Measurement Tasks to do, when to do them, and how to report the

   Measurement Results (Section 5.2.2).  The Measurement Agent then acts

   on the Instruction autonomously.  The Control Protocol also enables

   the MA to inform the Controller about its Capabilities and any

   Failure and Logging Information (Section 5.2.2).  Finally, the

   Control Protocol allows the Controller to update the MA's

   Configuration.



Another example of repetition.

>From the terminology:

   Control Protocol: The protocol delivering Instruction(s) from a

   Controller to a Measurement Agent.  It also delivers Capabilities,

   Failure and Logging Information from the Measurement Agent to the

   Controller.  It can also be used to update the MA's Configuration.



[phil] personally I think this repetition is ok. Section 5 is providing the protocol model, so needs to describe what the control protocol does.

-

section 5.2.2

Instruction:                            ->

[(Measurement Task configuration(

   [Input Parameter],

   (interface),

   (Cycle-ID))),

 (Report Channel),

 (Schedule),

 (Suppression information)]

                                         <-          Response(details)





It seems from the above that "Measurement Task configuration" takes 3 arguments.

I'm surprised not to see the Metric (which is described in the section btw) above.

According to the Input Parameter def., I understand that the Metric is NOT part of the Input Parameter.



[phil] good spot, thanks.



-

5.2.2.1.  Suppression



   The Instruction may include Suppression information.  The purpose of

   Suppression is to enable the Controller to instruct the MA not to

   perform Measurement Tasks.  It is used if the measurement system

   wants to eliminate inessential traffic, because there is some

   unexpected network issue for example.





Again, "eliminate inessential traffic". This is an active probing only view.

[phil] does adding something like this solve the issue?

"or inessential processing by a networking component"

Same remark with

   o  a set of Measurement Schedules to suppress; the others are not

      suppressed.  For example, suppose the measurement system has

      defined two Schedules, one with the most critical Measurement

      Tasks and the other with less critical ones that create a lot of

      Measurement Traffic, then it may only want to suppress the second.

[phil] Paul, Aamer - please suggest some text - don't know what's appropriate for passive tasks



-

5.3.1.  Starting and Stopping Measurement Tasks



That's typical section completely dedicated to active probing:

   Before sending Measurement Traffic the MA may run a pre-check.  (The

   pre-check could be defined as a separate, preceding Task or as the

   first part of a larger Task.)  Action could include:



   o  the MA checking that there is no cross-traffic.  In other words, a

      check that the end-user isn't already sending traffic;



For passive monitoring, you typically want to check that there is traffic of interest...



[phil] ok. will expand the list.

-

5.4.1.  Reporting of Subscriber's service parameters



The charter says:

Service parameters, such as product category, can be useful to decide

which measurements to run and how to interpret the results. These

parameters are already gathered and stored by existing operations

systems.

Discovering the service parameters on the MAs or sharing

the service parameters between MAs are out of the scope. However, if the

 service parameters are available to the MAs, they could be reported

with the measurement results in the Report Protocol.





The key sentence is: "if the service parameters are available to the MAs"

I don't find back this concept in section 5.4.1



[phil] I think S5.4.1 does mention this:
The information could be transferred directly from the Subscriber
   parameter database to the data analysis tools.  It may also be
   possible to transfer the information via the MA.  How (and if) the MA
   knows such information is likely to depend on the device type.  The
   MA could either include the information in a Measurement Report or
   separately.





-

section 5.5



   It is also possible that a different choice is made for the

   Control and Report Protocols, for example NETCONF-YANG and IPFIX

   (Internet Protocol Flow Information Export) respectively.



Missing references





-

   For example, there

   could be different Controllers for different types of MA (home

   gateways, tablets) or locations (Ipswich, Edinburgh), for load

   balancing or to cope with failure of one Controller.



Another example of repetition.

[phil] ok, will delete this paragraph



-

section 6.1



   The measurement system also needs to consider carefully how to

   interpret missing Results; for example, if the missing Results are

   ignored and the lack of a Report is caused by its broadband being

   broken, then the estimate of overall performance, averaged across all

   MAs, would be too optimistic.  The correct interpretation may depend

   on the specifics of the Measurement Task and Measurement Schedule.



What do you mean by "missing Results"?

Missing from a measurement or reporting point of view?

And if it relates to measurement, it's completely different from a passive monitoring or active probing point of view.

[phil] How about:

 The measurement system also needs to consider carefully how to

   interpret missing Results. The correct interpretation depends

   on why the Results are missing, and potentially on the specifics of the Measurement Task and Measurement Schedule.





- Appendix A:



One more example that there is some repetition, why do you need this:



   The LMAP framework defines two types of components involved in the

   actual measurement task, namely the Measurement Agent (MA) and the

   Measurement Peer (MP).  The fundamental difference conveyed in the

   definition of these terms is that the MA has a interface with the

   Controller/Collector while the MP does not.  The MP is broadly

   defined as a function that assists the MA in the Measurement Task but

   has no interface with the Controller/Collector.  There are many

   elements in the network that can fall into this broad definition of

   MP.  We believe that the MP terminology is useful to allow us to

   refer an element of the network that plays a role that is

   conceptually important to understand and describe the measurement

   task being performed.  We next illustrate these concepts by

   describing several deployment scenarios.



[phil] ok. will delete.



- Appendix Figure 2.



OLD:

      +----------------+    OWAMP     +----------------+    ^

      | OWAMP          |<--control--->| MP:            |    |

      | control-client |>test-traffic>| OWAMP server & |   IPPM





NEW:

      +----------------+    OWAMP     +----------------+    ^

      | OWAMP          |<--control--->| MP:            |    |

      | control-client |-test-traffic>| OWAMP server & |   IPPM






- Appendix: figure A4

   +-----+   +----------------+              +------+   ^

   | MP  |   |  MA: Monitor   |              | MP   | IPPM

   |     |<--|----------------|---traffic--->|      | Scope

   +-----+   |                |              +------+   |

      .......|................|.........................v...........





You mentioned in relation to this figure "Next, we consider Measurement Methods that measure user traffic."
So "IPPM scope" is wrong. This might be IPFIX/PSAMP
[phil] ok, will replace with "IPFX/PSAMP/IPPM Scope"


- Appendix: figure A4
I don't understand this sentence:

   if packets are not forwarded, the measurement task

   will not work.

- Also, related to the previous comment. What about a remote span to the probe? Is this in scope or not?
[phil] I'd be ok to delete the whole of the last paragraph.


Regards, Benoit