[lmap] draft-ietf-lmap-framework-08: AD review - part 1

[Benoit Claise <[e-mail]>]

Dear all,

Here is my AD review.
Note: I haven't reviewed yet the section 7 "security considerations" and 
section 8 "privacy considerations", but thought I should anyway send 
this before the meeting this week.
Note2: if some points were discussed already, don't hesitate to let me know.

A first impression is that there are some repetitions in the draft. It 
seems like some sections, written by different persons, were glued 
together. Reading this draft again with a fresh mind would highlight this.

One important issue, which I mentioned already during the interim 
meeting: the lack of passive monitoring aspects in the draft.

    Section 2

    The MA may
    generate Measurement Traffic and measure some metric associated with
    its transfer, or the MA may observe existing traffic, or there may be
    some kind of hybrid of these two possibilities.

So the document speaks a little bit about active and passive monitoring. 
Good, because this is mentioned in the charter:

    Both active and passive measurements are in scope, although there
    may be differences in their applicability to specific use cases, or
    in the security measures needed according to the threats specific to
    each measurement category. LMAP will not standardize performance
    metrics.

I recall the charter discussion. I specifically asked: "do you want to 
have passive monitoring in the charter? Do you understand all the 
consequences (for example, traffic filters, for observed packet/flow, 
which may imply RFC6728)?". Both answers were "yes".
So, the draft clearly misses the passive monitoring aspects, starting 
with the primary figure in this framework, figure 1, which only covers 
the active monitoring



Some other issues, simply listed in order
-

    These devices could be software based
    agents on PCs, embedded agents in consumer devices (e.g.  Blu-ray
    players),

Does a blu-ray player need a broadband test?

-
  o  Diversity - a measurement system should handle different types of
       Measurement Agents - for example Measurement Agents may come from
       different vendors, be in wired and wireless networks, be able to
       execute different sorts of Measurement Task and be on devices with
       IPv4 or IPv6 addresses.

Is "different types" the right term?
You have introduced the term "capabilities"


-

    It is also useful to define a registry
    for commonly-used Metrics [I-D.manyfolks-ippm-metric-registry  <http://tools.ietf.org/html/draft-ietf-lmap-framework-08#ref-I-D.manyfolks-ippm-metric-registry>] so
    that a Metric with its associated Measurement Method can be referred
    to simply by its identifier in the registry.

The new draft is http://tools.ietf.org/html/draft-ietf-ippm-metric-registry

-
OLD:

    Finally we introduce several components that are outside the scope of
    initial LMAP work and will be provided through existing protocols or
    applications.

NEW:

    Finally we introduce several components that are outside the scope of
    initial LMAP work (see in figure 1 "out of scope" and will be provided
    through existing protocols or applications.

Then I would list four bullet points for the next paragraph.
Otherwise it's not obvious which components you speak about

- figure 1
     * Since the above mentions the Results repository  (The data 
analysis tools receive the results from the Collector or via the Results 
repository. ), the figure should replace repository by Results repository
     * "IPPM scope" is wrong (see my first remark)
     * Mention the Control Protocol and Report Protocol, or Control 
Channel/Report Channel. Note: if you replace Instruction by Control 
Protocol, then the arrow must be bidirectional (according to the Control 
Protocol definition)

-

    Control Channel: A Channel between a Controller and a MA over which
    Instruction Messages and Capabilities, Failure and Logging
    Information are sent.

    Control Protocol: The protocol delivering Instruction(s) from a
    Controller to a Measurement Agent.  It also delivers Capabilities,
    Failure and Logging Information from the Measurement Agent to the
    Controller.  It can also be used to update the MA's Configuration.

Do you want to mention that the Control Protocol runs on the Control Channel?

-
    Measurement Method: The process for assessing the value of a Metric;
    the process of measuring some performance or reliability parameter
    associated with the transfer of traffic; where this process involves
    multiple MAs or MPs, each may perform different roles.

I don't understand what "where this process involves multiple MAs or MPs, each may perform different roles." adds to the def.

-
   Report Channel: A Channel between a Collector and a MA over which
    Report messages are sent.

Report messages -> Report Messages


-
4.2
  An operator may have several Controllers, perhaps with a Controller
    for different types of MA (home gateways, tablets) or location
    (Ipswich, Edinburgh).

BT specific, maybe? :-)


- Section 5.
Here is yet another example of repetition (see the last 3 bullets below. Why do you have to repeat what the Control Protocol does)

    An LMAP system goes through the following phases:

    o  a Bootstrapping process before the MA can take part in the other
       three phases

    o  a Control Protocol, which delivers Instruction Messages from a
       Controller to a MA, detailing what Measurement Tasks the MA should
       perform and when, and how it should report the Measurement
       Results.  It also delivers Capabilities, Failure and logging
       Information from a MA to its Controller.  Finally, it allows the
       Controller to update the MA's Configuration.

    o  the actual Measurement Tasks, which measure some performance or
       reliability parameter(s) associated with the transfer of packets.

       The LMAP work does not define Metrics and Measurement Methods,
       these are defined elsewhere (e.g.  IPPM).

    o  a Report Protocol, which delivers Reports from a MA to a
       Collector.  The Report contains the Measurement Results.


-
Would be nice to have "measurement system" in the terminology section.
It's mentioned so many times in the draft


-
5.2.  Control Protocol

    The primary purpose of the Control Protocol is to allow the
    Controller to configure a Measurement Agent with an Instruction about
    what Measurement Tasks to do, when to do them, and how to report the
    Measurement Results (Section 5.2.2).  The Measurement Agent then acts
    on the Instruction autonomously.  The Control Protocol also enables
    the MA to inform the Controller about its Capabilities and any
    Failure and Logging Information (Section 5.2.2).  Finally, the
    Control Protocol allows the Controller to update the MA's
    Configuration.

Another example of repetition.
 From the terminology:
    Control Protocol: The protocol delivering Instruction(s) from a
    Controller to a Measurement Agent.  It also delivers Capabilities,
    Failure and Logging Information from the Measurement Agent to the
    Controller.  It can also be used to update the MA's Configuration.


-
section 5.2.2
Instruction:                            ->
[(Measurement Task configuration(
    [Input Parameter],
    (interface),
    (Cycle-ID))),
  (Report Channel),
  (Schedule),
  (Suppression information)]
                                          <-          Response(details)


It seems from the above that "Measurement Task configuration" takes 3 arguments.
I'm surprised not to see the Metric (which is described in the section btw) above.
According to the Input Parameter def., I understand that the Metric is NOT part of the Input Parameter.


-
5.2.2.1.  Suppression

    The Instruction may include Suppression information.  The purpose of
    Suppression is to enable the Controller to instruct the MA not to
    perform Measurement Tasks.  It is used if the measurement system
    wants to eliminate inessential traffic, because there is some
    unexpected network issue for example.


Again, "eliminate inessential traffic". This is an active probing only view.
Same remark with
    o  a set of Measurement Schedules to suppress; the others are not
       suppressed.  For example, suppose the measurement system has
       defined two Schedules, one with the most critical Measurement
       Tasks and the other with less critical ones that create a lot of
       Measurement Traffic, then it may only want to suppress the second.


-
5.3.1.  Starting and Stopping Measurement Tasks

That's typical section completely dedicated to active probing:
    _Before sending Measurement Traffic_  the MA may run a pre-check.  (The
    pre-check could be defined as a separate, preceding Task or as the
    first part of a larger Task.)  Action could include:

    o_the MA checking that there is no cross-traffic_.  In other words, a
       check that the end-user isn't already sending traffic;

For passive monitoring, you typically want to check that there is traffic of interest...


-
5.4.1.  Reporting of Subscriber's service parameters

The charter says:
Service parameters, such as product category, can be useful to decide
which measurements to run and how to interpret the results. These
parameters are already gathered and stored by existing operations
systems.
Discovering the service parameters on the MAs or sharing
the service parameters between MAs are out of the scope. However, if the
  service parameters are available to the MAs, they could be reported
with the measurement results in the Report Protocol.


The key sentence is: "if the service parameters are available to the MAs"
I don't find back this concept in section 5.4.1

-
section 5.5

    It is also possible that a different choice is made for the
    Control and Report Protocols, for example NETCONF-YANG and IPFIX
    (Internet Protocol Flow Information Export) respectively.
  
Missing references


-
    For example, there
    could be different Controllers for different types of MA (home
    gateways, tablets) or locations (Ipswich, Edinburgh), for load
    balancing or to cope with failure of one Controller.

Another example of repetition.

-
section 6.1

    The measurement system also needs to consider carefully how to
    interpret missing Results; for example, if the missing Results are
    ignored and the lack of a Report is caused by its broadband being
    broken, then the estimate of overall performance, averaged across all
    MAs, would be too optimistic.  The correct interpretation may depend
    on the specifics of the Measurement Task and Measurement Schedule.

What do you mean by "missing Results"?
Missing from a measurement or reporting point of view?
And if it relates to measurement, it's completely different from a passive monitoring or active probing point of view.





- Appendix A:

One more example that there is some repetition, why do you need this:

    The LMAP framework defines two types of components involved in the
    actual measurement task, namely the Measurement Agent (MA) and the
    Measurement Peer (MP).  The fundamental difference conveyed in the
    definition of these terms is that the MA has a interface with the
    Controller/Collector while the MP does not.  The MP is broadly
    defined as a function that assists the MA in the Measurement Task but
    has no interface with the Controller/Collector.  There are many
    elements in the network that can fall into this broad definition of
    MP.  We believe that the MP terminology is useful to allow us to
    refer an element of the network that plays a role that is
    conceptually important to understand and describe the measurement
    task being performed.  We next illustrate these concepts by
    describing several deployment scenarios.


- Appendix Figure 2.

OLD:
       +----------------+    OWAMP     +----------------+    ^
       | OWAMP          |<--control--->| MP:            |    |
       | control-client |>test-traffic>| OWAMP server & |   IPPM


NEW:
       +----------------+    OWAMP     +----------------+    ^
       | OWAMP          |<--control--->| MP:            |    |
       | control-client |-test-traffic>| OWAMP server & |   IPPM
  


- Appendix: figure A4

    +-----+   +----------------+              +------+   ^
    | MP  |   |  MA: Monitor   |              | MP   | IPPM
    |     |<--|----------------|---traffic--->|      | Scope
    +-----+   |                |              +------+   |
       .......|................|.........................v...........


You mentioned in relation to this figure "Next, we consider Measurement Methods that measure user traffic."

So "IPPM scope" is wrong. This might be IPFIX/PSAMP

- Appendix: figure A4
I don't understand this sentence:

    if packets are not forwarded, the measurement task
    will not work.


- Also, related to the previous comment. What about a remote span to the 
probe? Is this in scope or not?

Regards, Benoit