Re: [PMOL] PMOL directorate: meeting at the IETF?

["Aamer Akhter (aakhter)" <[e-mail]>]

Just as an FYI:


"  http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03<http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03>"

Explicitly does NOT define any performance metrics-just the IPFIX transport. There are references to other documents (the XRBLOCK, RFC3550, the perf-mon methodology draft, etc.) that do define the metrics. Both this draft and the perf-mon methodology draft (it's expired currently, and I need to update it), are using the templates defined RFC6390. There might be some mistakes here and there, but that is the intent anyway and I'm hopeful that further reviews will make these drafts sharper.

==

That said, I do not disagree with the premise below that the entire performance monitoring (IMHO) creation and dissemination is (frankly) a mess. I would like to propose an enforced clean separation between metrics definition (the methodology and usage etc-which should follow the templates in RFC6390) from the transport-expression of these metrics (RTCP, MIB, IPFIX, syslog whatever), which refer back to the methodology.

Unfortunately, I've got other commitments and will not be in ATL, but very interested in moving this discussion along.


From: [e-mail] [mailto:[e-mail]] On Behalf Of Benoit Claise (bclaise)
Sent: Monday, October 22, 2012 6:29 PM
To: [e-mail]
Cc: [e-mail]; Ron Bonica; [e-mail]; Wesley Eddy
Subject: [PMOL] PMOL directorate: meeting at the IETF?

Dear PMOL directorate members,

During my review of the latest AVTCORE and  XRBLOCK drafts ( draft-ietf-xrblock-rtcp-xr-pdv and draft-ietf-avtcore-monarch), I came to the conclusion that we have an issue in terms of performance metrics at the IETF, and actually in the industry.

As background information, here is my DISCUSS on the two drafts

----------------------------------------------------------------------

DISCUSS:

----------------------------------------------------------------------



My entire point is more a DISCUSS-DISCUSS, for both

draft-ietf-avtcore-monarch-19 and draft-ietf-xrblock-rtcp-xr-pdv-05.txt.

Sorry to pick on these two drafts, but we need to have an IESG

performance metrics discussion.

Where does the list of performance metric definitions come from at the

IETF?

We have multiple sources:

- IPPM for IP performance metrics

- RTCP for RTP performance metrics:

  Definitions in the document themselves or potentially referencing some

other SDOs

  Example: http://tools.ietf.org/html/draft-ietf-xrblock-rtcp-xr-pdv-05

   bits 014-011

            0: MAPDV2, Clause 6.2.3.2 of [G.1020],

            1: 2-point PDV, Clause 6.2.4 of [Y.1540].

- PMOL: Performance Metrics at Other Layers, with

  RFC 6076 on Basic Telephony SIP End-to-End Performance Metrics

- IPFIX will one day or the other exports performance metrics.

  I see for example

  http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03<http://tools.ietf.org/html/draft-akhter-opsawg-perfmon-ipfix-03>

  It's again a redefinition, and it should not be!



My concerns are that we start to define performance metrics in different

parts of the IETF, without consistency.



We have defined RFC 6390 on "Guidelines for Considering New Performance

Metric Development", which ask for specific definition

See http://tools.ietf.org/html/rfc6390#section-5.4.4



I believe that the IETF should at least:

- define the performance metrics in a consistent way according to

RFC6390.

- document those performance metrics in a single location



So my questions are:

- are we defining the performance metrics the right way?

- where is this shared repository of performance metrics (at least for

the ones created in the IETF)?

- is the PMOL directorate (RFC 6390) used effectively?
After discussing with Dan Romascanu, we came to this conclusion

I had a discussion with Dan Romascanu, and we settled on:

- RFC 6390 template is required for new perf metric definition

- RFC 6390 template is a nice-to-have when we refer to an existing perf metric

Nice-to-have because the performance metric reference doesn't always include

all the required information about: measurement points, measurement timing, use

and applications, reporting model, etc... but focus only on the "Method of

Measurement or Calculation"
I would like to have a meeting during the IETF, with the following agenda


- are we defining the performance metrics the right way?

- where is this shared repository of performance metrics (at least for

the ones created in the IETF)?

- is the PMOL directorate (RFC 6390) used effectively?

- conclusion discussed with Dan



Here is a doodle invite. Please let me know if/when you are available.

Your feedback on this mailer is also welcome.
Regards, Benoit