[PMOL] PMOL directorate: meeting at the IETF?

[Benoit Claise <[e-mail]>]

Dear PMOL directorate members,

During my review of the latest AVTCORE and  XRBLOCK drafts ( 
draft-ietf-xrblock-rtcp-xr-pdv and draft-ietf-avtcore-monarch), I came 
to the conclusion that we have an issue in terms of performance metrics 
at the IETF, and actually in the industry.

As background information, here is my DISCUSS on the two drafts

    ----------------------------------------------------------------------
    DISCUSS:
    ----------------------------------------------------------------------

    My entire point is more a DISCUSS-DISCUSS, for both
    draft-ietf-avtcore-monarch-19 and draft-ietf-xrblock-rtcp-xr-pdv-05.txt.
    Sorry to pick on these two drafts, but we need to have an IESG
    performance metrics discussion.
    Where does the list of performance metric definitions come from at the
    IETF?
    We have multiple sources:
    - IPPM for IP performance metrics
    - RTCP for RTP performance metrics:
       Definitions in the document themselves or potentially referencing some
    other SDOs
       Example:http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-05
        bits 014-011
                 0: MAPDV2, Clause 6.2.3.2 of [G.1020],
                 1: 2-point PDV, Clause 6.2.4 of [Y.1540].
    - PMOL: Performance Metrics at Other Layers, with
       RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics
    - IPFIX will one day or the other exports performance metrics.
       I see for example
       http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03
       It's again a redefinition, and it should not be!

    My concerns are that we start to define performance metrics in different
    parts of the IETF, without consistency.

    We have defined RFC 6390 on "Guidelines for Considering New Performance
    Metric Development", which ask for specific definition
    Seehttp://tools.ietf.org/html/rfc6390#section-5.4.4

    I believe that the IETF should at least:
    - define the performance metrics in a consistent way according to
    RFC6390.
    - document those performance metrics in a single location

    So my questions are:
    - are we defining the performance metrics the right way?
    - where is this shared repository of performance metrics (at least for
    the ones created in the IETF)?
    - is the PMOL directorate (RFC 6390) used effectively?

After discussing with Dan Romascanu, we came to this conclusion

    I had a discussion with Dan Romascanu, and we settled on:
    - RFC 6390 template is required for new perf metric definition
    - RFC 6390 template is a nice-to-have when we refer to an existing perf metric
    Nice-to-have because the performance metric reference doesn't always include
    all the required information about: measurement points, measurement timing, use
    and applications, reporting model, etc... but focus only on the "Method of
    Measurement or Calculation"

I would like to have a meeting during the IETF, with the following agenda

- are we defining the performance metrics the right way?
- where is this shared repository of performance metrics (at least for
the ones created in the IETF)?
- is the PMOL directorate (RFC 6390) used effectively?
- conclusion discussed with Dan

Here is a doodle invite. Please let me know if/when you are available.
Your feedback on this mailer is also welcome.

Regards, Benoit