[lmap] Feedback on draft-ietf-lmap-information-model

[Benoit Claise <[e-mail]>]

Dear all,

Since we will be spending time on draft-ietf-lmap-information-model 
tomorrow, here is some more feedback. I haven't had the time to review 
it all, so here is part 1.
If some points were already discussed, don't hesitate to let me know.

> Network Working Group                                       T. Burbridge
> Internet-Draft                                                P. Eardley
> Intended status: Standards Track                                      BT
> Expires: February 21, 2015                                    M. Bagnulo
>                                         Universidad Carlos III de Madrid
>                                                         J. Schoenwaelder
>                                                 Jacobs University Bremen
>                                                          August 20, 2014
>
>
>      Information Model for Large-Scale Measurement Platforms (LMAP)
>                   draft-ietf-lmap-information-model-02
What does LMAP stand for?
In the use cases draft, it says "Large-scale Measurement of Broadband 
Performance (LMAP)"
Both the framework and the information model say: Large-Scale 
Measurement Platforms (LMAP)

>
> Abstract
>
>    This Information Model applies to the Measurement Agent within a
>    Large-Scale Measurement Platform.  As such it outlines the
>    information that is (pre-)configured on the MA or exists in
>    communications with a Controller or Collector within an LMAP
>    framework.  The purpose of such an Information Model is to provide a
>    protocol and device independent view of the MA that can be
>    implemented via one or more Control and Report protocols.
>
> Requirements Language
>
>    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
>    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
>    document are to be interpreted as described in RFC 2119 [RFC2119].
>
> Status of This Memo
>
>    This Internet-Draft is submitted in full conformance with the
>    provisions of BCP 78 and BCP 79.
>
>    Internet-Drafts are working documents of the Internet Engineering
>    Task Force (IETF).  Note that other groups may also distribute
>    working documents as Internet-Drafts.  The list of current Internet-
>    Drafts is at http://datatracker.ietf.org/drafts/current/.
>
>    Internet-Drafts are draft documents valid for a maximum of six months
>    and may be updated, replaced, or obsoleted by other documents at any
>    time.  It is inappropriate to use Internet-Drafts as reference
>    material or to cite them other than as "work in progress."
>
>    This Internet-Draft will expire on February 21, 2015.
>
>
>
>
>
>
> Burbridge, et al.       Expires February 21, 2015 [Page 1]
> 
> Internet-Draft           LMAP Information Model August 2014
>
>
> Copyright Notice
>
>    Copyright (c) 2014 IETF Trust and the persons identified as the
>    document authors.  All rights reserved.
>
>    This document is subject to BCP 78 and the IETF Trust's Legal
>    Provisions Relating to IETF Documents
>    (http://trustee.ietf.org/license-info) in effect on the date of
>    publication of this document.  Please review these documents
>    carefully, as they describe your rights and restrictions with respect
>    to this document.  Code Components extracted from this document must
>    include Simplified BSD License text as described in Section 4.e of
>    the Trust Legal Provisions and are provided without warranty as
>    described in the Simplified BSD License.
>
> Table of Contents
>
>    1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
>    2.  Notation  . . . . . . . . . . . . . . . . . . . . . . . . . .   4
>    3.  LMAP Information Model  . . . . . . . . . . . . . . . . . . .   4
>      3.1.  Information Structure . . . . . . . . . . . . . . . . . .   4
>      3.2.  Pre-Configuration Information . . . . . . . . . . . . . .   8
>      3.3.  Configuration Information . . . . . . . . . . . . . . . .   9
>      3.4.  Instruction Information . . . . . . . . . . . . . . . . .  11
>      3.5.  Logging Information . . . . . . . . . . . . . . . . . . .  13
>      3.6.  Capability and Status Information . . . . . . . . . . . .  15
>      3.7.  Reporting Information . . . . . . . . . . . . . . . . . .  17
>      3.8.  Schedules . . . . . . . . . . . . . . . . . . . . . . . .  18
>      3.9.  Channels  . . . . . . . . . . . . . . . . . . . . . . . .  21
>      3.10. Task Configurations . . . . . . . . . . . . . . . . . . .  22
>      3.11. Timing Information  . . . . . . . . . . . . . . . . . . .  23
>        3.11.1.  Periodic Timing  . . . . . . . . . . . . . . . . . .  24
>        3.11.2.  Calendar Timing  . . . . . . . . . . . . . . . . . .  25
>        3.11.3.  One-Off Timing . . . . . . . . . . . . . . . . . . .  26
>        3.11.4.  Immediate Timing . . . . . . . . . . . . . . . . . .  26
>        3.11.5.  Startup Timing . . . . . . . . . . . . . . . . . . .  26
>    4.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  26
>    5.  Security Considerations . . . . . . . . . . . . . . . . . . .  26
>    6.  Appendix: JSON Example  . . . . . . . . . . . . . . . . . . .  27
>    7.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  35
>    8.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  35
>      8.1.  Normative References  . . . . . . . . . . . . . . . . . .  35
>      8.2.  Informative References  . . . . . . . . . . . . . . . . .  35
>    Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  35
>
>
>
>
>
>
>
> Burbridge, et al.       Expires February 21, 2015 [Page 2]
> 
> Internet-Draft           LMAP Information Model August 2014
>
>
> 1.  Introduction
>
>    A large-scale measurement platform is a collection of components that
>    work in a coordinated fashion to perform measurements from a large
>    number of vantage points.  The main components of a large-scale
>    measurement platform are the Measurement Agents (hereafter MAs), the
>    Controller(s) and the Collector(s).
>
>    The MAs are the elements actually performing the measurements. The
>    MAs are controlled by exactly one Controller at a time and the
>    Collectors gather the results generated by the MAs.  In a nutshell,
>    the normal operation of a large-scale measurement platform starts
>    with the Controller instructing a set of one or more MAs to perform a
>    set of one or more Measurement Tasks at a certain point in time.  The
>    MAs execute the instructions from a Controller, and once they have
>    done so, they report the results of the measurements to one or more
>    Collectors.  The overall framework for a Large Measurement platform
>    as used in this document is described in detail in
>    [I-D.ietf-lmap-framework].
>
>    A large-scale measurement platform involves basically three
>    protocols, namely, a Control protocol between a Controller and the
Control Protocol
> MAs, a Report protocol between the MAs and the Collector(s) and
>    several measurement protocols between the MAs and Measurement Peers
>    (MPs), used to actually perform the measurements.  In addition some
>    information is required to be configured on the MA prior to any
>    communication with the initial Controller.
"initial" confused me.
It's only later (section 3.3) that I understood that the Controller 
could be changed.
Candidate for removal, improvement, or forward reference?
>
>    This document defines the information model for both the Control and
>    the Report protocol along with pre-configuration information that is
>    required 
add "on the MA"
> before communicating with the Controller, broadly named as
>    the LMAP Information Model.  The measurement protocols are out of the
>    scope of this document.
>
>    As defined in [RFC3444], the LMAP IM defines the concepts involved in
IM = Information Model
this is the first occurrence.

>    a large-scale measurement platform at a high level of abstraction,
>    independent of any specific implementation or actual protocol used to
>    exchange the information.  It is expected that the proposed
>    information model can be used with different protocols in different
>    measurement platform architectures and across different types of MA
>    devices (e.g., home gateway, smartphone, PC, router).
>
>    The definition of an Information Model serves a number of purposes:
>
>    1.  To guide the standardisation of one or more Control and Report
>        protocol and data model implementations
>
>
>
>
>
> Burbridge, et al.       Expires February 21, 2015 [Page 3]
> 
> Internet-Draft           LMAP Information Model August 2014
>
>
>    2.  To enable high-level inter-operability between different Control
>        and Report protocols by facilitating translation between their
>        respective data models such that a Controller could instruct sub-
>        populations of MAs using different protocols
>
>    3.  To form agreement of what information needs to be held by an MA
>        and passed over the Control and Report interfaces and support the
>        functionality described in the LMAP framework
>
>    4.  Enable existing protocols and data models to be assessed for
>        their suitability as part of a large-scale measurement system
>
> 2.  Notation
>
>    This document use an object-oriented programming-like notation to
>    define the parameters (names/values) of the objects of the
>    information model.  An optional field is enclosed by [ ], and an
>    array is indicated by two numbers in angle brackets, <m..n>, where m
>    indicates the minimal number of values, and n is the maximum. The
>    symbol * for n means no upper bound.
>
> 3.  LMAP Information Model
>
> 3.1.  Information Structure
>
>    The information described herein relates to the information stored,
>    received or transmitted by a Measurement Agent as described within
>    the LMAP framework [I-D.ietf-lmap-framework]. 
Should the framework be normative? I believe so, specifically when I see 
all those Capitalized terms that are only defined in the framework.
This leads to another point. You miss a terminology section because some 
terms are specific to this document. Example: Task Suppression.
> As such, some subsets
>    of this information model are applicable to the measurement
>    Controller, Collector 
add a ","
Otherwise we can believe that the Collector could pre-configure the MA.
> and systems that pre-configure the Measurement
>    Agent.  The information described in these models will be transmitted
>    by protocols using interfaces between the Measurement Agent and such
>    systems according to a Data Model.
>
>    For clarity the information model is divided into six sections:
>
>    1.  Pre-Configuration Information.  Information pre-configured on the
>        Measurement Agent prior to any communication with other
>        components of the LMAP architecture (i.e., the Controller,
>        Collector and Measurement Peers), specifically detailing how to
>        communicate with a Controller and whether the device is enabled
>        to participate as an MA.
>
>    2.  Configuration Information.  Update of the pre-configuration
>        information during the registration of the MA or subsequent
>        communication with the Controller, along with the configuration
>        of further parameters about the MA (rather than the Tasks it
>
>
>
>
> Burbridge, et al.       Expires February 21, 2015 [Page 4]
> 
> Internet-Draft           LMAP Information Model August 2014
>
>
>        should perform) that were not mandatory for the initial
>        communication between the MA and a Controller.
>
>    3.  Instruction Information.  Information that is received by the MA
>        from the Controller pertaining to the Tasks that should be
>        executed.  This includes the task execution Schedules (other than
>        the Controller communication Schedule supplied as
>        (pre)configuration information) and related information such as
>        the Task Configuration, communication Channels to Collectors and
>        schedule Timing information.  It also inlcudes Task Suppression
>        information that is used to over-ride normal Task execution
>        during emergency situations.
>
>    4.  Logging Information.  Information transmitted from the MA to the
>        Controller detailing the results of any configuration operations
>        along with error and status information from the operation of the
>        MA.
>
>    5.  Capability and Status Information.  Information on the general
>        status and capabilities of the MA.  For example, the set of
>        measurements that are supported on the device.
>
>    6.  Reporting Information.  Information transmitted from the MA to
>        one or more Collectors including measurement results and the
>        context in which they were conducted.
>
>    In addition the MA may hold further information not described herein,
>    and which may be optionally transferred to or from other systems
>    including the Controller and Collector.  One example of information
>    in this category is subscriber or line information that may be
>    extracted by a task and reported by the MA in the reporting
>    communication to a Collector.
>
>    It should also be noted that the MA may be in communication with
The "MA" or the "MA device"?
I'm asking because the rest of the sentence speaks about "configuring", 
and we said that MA can only be configured by one and only one Controller.
> other management systems which may be responsible for configuring and
>    retrieving information from the MA device.  Such systems, where
>    available, can perform an important role in transferring the pre-
>    configuration information to the MA or enabling/disabling the
>    measurement functionality of the MA.
"such systemS" ... "enabling/disabling the measurement functionality of 
the MA."
This is not possible. See my previous point.
>
>    The Information Model is divided into sub-sections for a number of
>    reasons.  Firstly the grouping of information facilitates reader
>    understanding.  Secondly, the particular groupings chosen are
>    expected to map to different protocols or different transmissions
>    within those protocols.
>
>    The granularity of data transmitted in each operation of the Control
>    and Report Protocols is not dictated by the Information Model. For
>
>
>
> Burbridge, et al.       Expires February 21, 2015 [Page 5]
> 
> Internet-Draft           LMAP Information Model August 2014
>
>
>    example, the Instruction object may be delivered in a single
>    operation.  Alternatively, Schedules and Task Configurations may be
>    separated or even each Schdule/Task Configuration may be delivered
>    individually.  Similarly the Information Model does not dictate
>    whether data is read, write, or read/write.  For example, some
>    Control Protocols may have the ability to read back Configuration and
>    Instruction information which have been previosuly set on the MA.
>    Lastly, while some protocols may simply overwrite information (for
>    example refreshing the entire Instruction Information), other
>    protocols may have the ability to update or delete selected items of
>    information.
>
>    The information in these six sections is captured by a number of
>    common information objects.  These objects are also described later
>    in this document and comprise of:
>
>    1.  Schedules.  A set of Schedules tell the MA to do something.
>        Without a Schedule no Task (from a measurement to reporting or
>        communicating with the Controller) is ever executed. Schedules
>        are used within the Instruction to specify what tasks should be
>        performed, when, and how to direct their results.  A Schedule is
>        also used within the pre-Configuration and Configuration
>        information in order to execute the Task or Tasks required to
>        communicate with the Controller.
>
>    2.  Channels.  A set of Channel objects are used to communicate with
>        a number of endpoints (i.e. the Controller and Collectors). 
OLD: (i.e. the Controller and Collectors).
NEW: (the Controller, Collectors, and MAs).

These are the only 3 possibilities, right?
Logging always goes to the Collector, right?

> Each
>        Channel object contains the information required for the
>        communication with a single endpoint such as the target location
>        and security details.  Channels are referenced from within
>        Schedules in order to say how Tasks should communicate.
>
>    3.  Task Configurations.  A set of Task Configurations is used to
>        configure the Tasks that are run by the MA.  This includes the
>        registry entry for the Task and any configuration parameters.
>        Task Configurations are referenced from a Schedule in order to
>        specify what Tasks the MA should execute.
>
>    4.  Timings.  A set of Timing objects that can be referenced from the
>        Schedules.  Each Schedule always references exactly one Timing
>        object.  A Timing object specfies either a singleton or series of
>        time events.  They are used to indicate when Tasks should be
>        executed.
>
>    The following diagram illustrates the structure in which these common
>    information objects are referenced.  The references are achieved by
>    each object (Channel, Task Configuration, Timing) being given a short
>
>
>
>
> Burbridge, et al.       Expires February 21, 2015 [Page 6]
> 
> Internet-Draft           LMAP Information Model August 2014
>
>
>    text name that is used by other objects.  The objects shown in
>    parenthesis are part of the internal object structure of a Schedule.
>
>           Schedule
>               |----------> Timing
>               |----------> (Scheduled Tasks)
>                                    |----------> Task Configuration
>                                    |----------> (Task Channels and 
> downstream Tasks)
> |----------> Channels
> |----------> Downstream Tasks
Please number the figures.

Why is only "configuration" mentioned in the figure?
I understood that everything is now a task:
     controller communication
     reporting
     measurement
     data aggregation
     ...
This was confusing to me.


>
>    It should be clear that the top-level bahaviour of an MA is simply to
>    execute Schedules.  Every action referenced by a Schedule is defined
>    as a Task.  As such, these actions are configured through Task
>    Configurations and executed according to the Timing referenced by the
>    Schedule in which they appear.  Tasks can implement a variety of
>    different types of actions.  While in terms of the Information Model,
>    all Tasks have the same structure, it can help conceptually to think
>    of different Task categories:
>
>    1.  Measurement Tasks
>
>        A.  Measurement Tasks measure some aspect of network performance
>            or traffic
>
>        B.  Data Capture Tasks capture and analyse passive information
Why capture? We can analyse without capture.
> stored on the MA device such as counters and device/network
>            status information

Why not traffic?
 From the charter:

    Both active and passive measurements are in scope, although there
    may be differences in their applicability to specific use cases, or
    in the security measures needed according to the threats specific to
    each measurement category


>
>    2.  Data Transfer Tasks
>
>        A.  Reporting Tasks report the results or Measurement Tasks to
>            Collectors
"Reporting Tasks report Measurement Tasks to Collectors"
Really? So the Controller configures the Reporting Tasks on the MA, and 
the MA reports them to the Collector?

Maybe you meant?
        A.  Reporting Tasks report the results _of _Measurement Tasks to
            Collectors
>
>        B.  Control Task(s) implement the Control Protocol and
>            communicate with the Controller.  Depending on the Control
>            Protocol this may be a number of specialist tasks such as:
What is "this"?
> Configuration Task; Instruction Task; Suppression Task;
>            Capabilities Task; Logging Task etc.
>
>    3.  Data Analysis Tasks can exist to analyse data from other
>        Measurement Tasks locally on the MA
>
>    4.  Data Management Tasks may exist to clean-up, filter or compress
>        data on the MA such as Measurement Task results
>
>
>
>
>
>
> Burbridge, et al.       Expires February 21, 2015 [Page 7]
> 
> Internet-Draft           LMAP Information Model August 2014
>
>
> 3.2.  Pre-Configuration Information
>
>    This information is the minimal information that needs to be pre-
>    configured to the MA in order for it to successfully communicate with
>    a Controller during the registration process. 
In section 3.1, we learned:

    1.  Pre-Configuration Information.  Information pre-configured on the
        Measurement Agent prior to any communication with other
        components of the LMAP architecture (i.e., the Controller,
        Collector and Measurement Peers), specifically detailing how to
        communicate with a Controller and whether the device is enabled
        to participate as an MA.

So the pre-configuration information is only for the Controller 
communication (I guess so) or also for the collector and measurement peers?
> The pre-configuration
>    information is a subset of the Configuration Information along with
>    some parameters that are not under the control of the LMAP framework
>    (such as the the device identifier and device security credentials).
I can't parse "not under the control of the LMAP framework"
>
>    This pre-configuration information needs to include a URL of the
>    initial Controller where configuration information can be retrieved
OLD: retrieved
NEW: communicated
NEW (alternative): pulled or pushed

Justification: the next paragraphs make the distinction.
> along with the security information required for the communication
>    including the certificate of the Controller (or the certificate of
>    the Certification Authority which was used to issue the certificate
>    for the Controller).  All this is expressed as a Channel. While
>    multiple Channels may be provided in the pre-configuration
>    information they must all be associated with a single Controller
>    (e.g. over different interfaces or network protocols).
>
>    Where the MA pulls information from the Controller, the Pre-
>    Configuration Information also needs to contain the timing of the
>    communication with the Controller as well as the nature of the
>    communication itself (such as the protocol and data to be
>    transfered).  The timing is given as a Schedule that executes the
>    Task(s) responsible for communication with the Controller.  It is
>    this Task (or Tasks) that implement the Control protocol between the
>    MA and the Controller.  The Task(s) may take additional parameters in
>    which case a Task Configuration can also be included.
>
>    Even where information is pushed to the MA from the Controller
>    (rather than pulled by the MA), a Schedule still needs to be
>    supplied.  In this case the Schedule will simply execute a Controller
>    listener task when the MA is started.  A Channel is still required
>    for the MA to establish secure communication with the Controller.
>
>    It can be seen that these Channels, Schedules and Task Configurations
>    for the initial MA-Controller communication are no different in terms
>    of the Information Model to any other Channel, Schedule or Task
>    Configuration that might execute a Measurement Task or report the
>    measurement results (as described later).
>
>    The MA may be pre-configured with an MA ID, or may use a Device ID in
>    the initial Controller contact before it is assigned an MA ID. 
Again, I'm confused by this initial Controller.

> The
>    Device ID may be a MAC address or some other device identifier
>    expressed as a URN.  If the MA ID is not provided at this stage then
>    it must be provided by the Controller during Configuration.
>
>    Detail of the information model elements:
>
>
>
> Burbridge, et al.       Expires February 21, 2015 [Page 8]
> 
> Internet-Draft           LMAP Information Model August 2014
>
>
> // MA pre-configuration minimal information to communicate initially 
> with Controller
>
> object {
>     [uuid                ma-agent-id;]
>      ma-task-obj         ma-control-tasks<1..*>;
>      ma-channel-obj      ma-control-channels<1..*>;
>      ma-schedule-obj     ma-control-schedules<1..*>;
>     [urn                 ma-device-id;]
>      credentials         ma-credentials;
> } ma-config-obj;
>
>    The detail of the Channel and Schedule objects are described later
>    since they are common to several parts of the information model.
>
> 3.3.  Configuration Information
>
>    During registration or at any later point at which the MA contacts
>    the Controller (or vice-versa), the choice of Controller, 
"The choice of Controller", do you want to say "an alternate 
Controller", because at this point the MA is already in contact with the 
Controller.
> details for
>    the timing of communication with the Controller or parameters for the
>    communication Task(s) can be changed (as captured by the Channels,
>    Schedules and Task Configurations objects).  For example the pre-
>    configured Controller (specified as a Channel or Channels) may be
>    replaced with a specific Controller that is more appropriate to the
>    MA device type, location or characteristics of the network (e.g.
>    access technology type or broadband product).  The initial
>    communication Schedule may be replaced with one more relevant to
>    routine communications between the MA and the Controller.
>
>    While some Control protocols and uses may only use a single Schedule,
>    other protocols and uses may uses several Schedules (and related data
>    transfer Tasks) to update the Configuration Information, transfer the
>    Instruction Information, transfer Capability and Status Information
>    and send other information to the Controller such as log or error
>    notifications.  Multiple Channels may be used to communicate with the
>    same Controller over multiple interfaces (e.g. to send logging
>    information over a different network).
>
>    In addition the MA will be given further items of information that
>    relate specifically to the MA rather than the measurements it is to
>    conduct or how to report results.  The assignment of an ID to the MA
>    is mandatory.  If the MA Agent ID was not optionally provided during
>    the pre-configuration then one must be provided by the Controller
>    during Configuration.  Optionally a Group ID may also be given which
>    identifies a group of interest to which that MA belongs.  For example
>    the group could represent an ISP, broadband product, technology,
>    market classification, geographic region, or a combination of
>    multiple such characteristics.  Where the Measurement Group ID is set
>    an additional flag (the Report MA ID flag) is required to control
>
>
>
> Burbridge, et al.       Expires February 21, 2015 [Page 9]
> 
> Internet-Draft           LMAP Information Model August 2014
>
>
>    whether the Measurement Agent ID is also to be reported.  The
>    reporting of a Group ID without the MA ID allows the MA to remain
>    anonymous, which may be particularly useful to prevent tracking of
>    mobile MA devices.
>
>    Optionally an MA can also be configured to stop executing any
>    Instruction Schedule if the Controller is unreachable.  This can be
>    used as a fail-safe to stop Measurement and other Tasks being
>    conducted when there is doubt that the Instruction Information is
>    still valid.  This is simply represented as a time window in
>    milliseconds since the last communication with the Controller after
>    which Instruction Schedules are to be suspended.  The appropriate
>    vaue of the time window will depend on the specified communication
value
> Schedule with the Controller and the duration for which the system is
>    willing to tolerate continued operation with potentially stale
>    Instruction Information.
>
>    While pre-configuration is persistent upon device reset or power
>    cycle due to its very nature, the persistency of the addtional
>    configuration information may be control protocol dependent. 
Why "Control Protocol" dependent?
Why isn't the persistence IM (or DM) specific?
> Some
>    protocols may assume that reset devices will revert back to their
>    pre-configuration state, while other protocols may assume that all
>    configuration and instruction information is held in persistent
>    storage.
>
>    It should be noted that control shedules and tasks cannot be
>    suppressed as evidenced by the lack of suppression information in the
>    Configuration.  The control schedule must only reference tasks listed
>    as control tasks.  Any suppress-by-default flag against control tasks
>    will be ignored.
>
>    Detail of the additional and updated information model elements:
>
>    // MA Configuration
>
>    object {
>        uuid                ma-agent-id;
>       [ma-task-obj         ma-control-tasks<0..*>;]
>        ma-channel-obj      ma-control-channels<1..*>;
>       [ma-schedule-obj    ma-control-schedules<0..*>];
>       [urn                 ma-device-id;]
>        credentials         ma-credentials;
>       [string              ma-group-id;]
>       [boolean             ma-report-ma-id-flag;]
>       [int                 ma-control-channel-failure-threshold;]
>    } ma-config-obj;
>
>
That's where I arrived.

And now, time for a Guinness or two. I'm in Dublin after all :-)

Regards, Benoit