Re: [ippm] Proposed charter updates for IPPM

["MORTON, ALFRED C (AL)" <[e-mail]>]

Hi Brian and Bill,

Thanks for preparing a new charter proposal and organizing
this message to highlight the changes. I agree with your
proposals.

I have one further proposal for the last sentence of
paragraph 1.

OLD (and your NEW):
Metrics developed by the IPPM WG are intended to provide unbiased 
quantitative performance measurements and not a value judgement.

I propose to delete/replace the last phrase, which I think has been present
since the very first charter. IIRC, the concern was that contributions 
might attempt to establish service acceptance thresholds, or other
numerical performance requirements/objectives using the metrics.
The work of setting numerical objectives proceeded elsewhere
(ITU-T Rec Y.1541), and the Earth continued to rotate.

If we want to ensure the status quo, we could say:
s/and not a value judgement/but numerical thresholds for the metrics are out-of-scope/

I also suggest this change with MBM in mind, where we have
testing outcomes of Pass/Fail/Inconclusive (PFI). Of course, the user
supplies the target rate, RTT, and/or loss ratio numerical values,
not the working group.

So we could express an explicit restriction that does not hamper MBM PFI:

NEW^2:
Metrics developed by the IPPM WG are intended to provide unbiased 
quantitative performance measurements, but numerical thresholds for 
the metrics are out-of-scope.

Or, we could remove the phrase, and refer any contributions on 
numerical thresholds to ITU-T SG 12, Question 17, to avoid 
overlapping work with another SDO (coordination with this SDO
should also be mentioned somewhere).

NEW^3:
Metrics developed by the IPPM WG are intended to provide unbiased 
quantitative performance measurements. 

thanks for considering this proposal, and regards,
Al

> -----Original Message-----
> From: ippm [mailto:[e-mail]] On Behalf Of Brian Trammell
> Sent: Sunday, July 09, 2017 11:24 AM
> To: IETF IPPM WG
> Subject: [ippm] Proposed charter updates for IPPM
> 
> Greetings, all,
> 
> Bill and I met briefly last week to discuss a proposed change to the
> charter, as a way to resolve the rough consensus to adopt the IOAM work
> within IPPM with concerns that have been raised that that work doesn't
> really fit on the charter. We arrived at a minimal suggested set of
> edits. Though inspired by IOAM, these aren't so much focused on bringing
> IOAM on charter as they are making the charter reflect the direction
> work in IPPM has been going over the five years since we last updated
> the charter.
> 
> We propose only to touch three paragraphs, as below:
> 
> ---
> 
> OLD Paragraph 1:
> 
> The IP Performance Metrics (IPPM) Working Group develops and maintains
> standard metrics that can be applied to the quality, performance, and
> reliability of Internet data delivery services and applications running
> over transport layer protocols (e.g. TCP, UDP) over IP. Specifying
> network or lower layer OAM mechanisms is out of scope of the IPPM
> charter. It also develops and maintains protocols for the measurement of
> these metrics. These metrics are designed such that they can be used by
> network operators, end users, or independent testing groups. Metrics
> developed by the IPPM WG are intended to provide unbiased quantitative
> performance measurements and not a value judgement.
> 
> NEW Paragraph 1:
> 
> The IP Performance Measurement (IPPM) Working Group develops and
> maintains standard metrics that can be applied to the quality,
> performance, and reliability of Internet data delivery services and
> applications running over transport layer protocols (e.g. TCP, UDP) over
> IP. It also develops and maintains methodologies and protocols for the
> measurement of these metrics. These metrics are designed such that they
> can be used by network operators, end users, or independent testing
> groups. Metrics developed by the IPPM WG are intended to provide
> unbiased quantitative performance measurements and not a value
> judgement.
> 
> 
> Two basic edits here:
> 
> (1) Rename working group from IP Performance Metrics to IP Performance
> Measurement, and add "methodologies" to the set of things we do. In our
> opinion, this reflects work that has been happening on charter for a
> while: MBM, PDM, and alt-mark are all methodology work as opposed to
> metric work.
> 
> (2) Remove the sentence "Specifying network or lower layer OAM
> mechanisms is out of scope of the IPPM charter". In my personal opinion,
> this isn't really necessary to bring the IOAM work on, since though IOAM
> can be carried on lower-layer headers, it doesn't really specify an OAM
> mechanism as I understand that term to be defined in the routing area.
> However, this restriction on the IPPM doesn't appear to do anything
> useful beyond reinforcing an artificial silo dividing data-plane and
> control-plane measurement, make it difficult for us to work together
> with other (e.g. INT or RTG area) working groups on performance
> measurement, and cause confusion as to whether we can do a thing called
> "IOAM" in IPPM.
> 
> ---
> 
> OLD Paragraph 6:
> 
> The WG has produced protocols for communication among test equipment to
> enable the measurement of the one- and two-way metrics (OWAMP and TWAMP
> respectively). These protocols will be advanced along the standards
> track. The work of the WG will take into account the suitability of
> measurements for automation, in order to support large-scale measurement
> efforts. This may result in further developments in protocols such as
> OWAMP and TWAMP. Agreement about the definitions of metrics and methods
> of measurement enables accurate, reproducible, and equivalent results
> across different implementations. To this end, the WG will define and
> maintain a registry of metric definitions. The WG encourages work which
> assesses the comparability of measurements of IPPM metrics with metrics
> developed elsewhere.
> 
> NEW Paragraph 6:
> 
> The WG has produced protocols for communication among test equipment to
> enable the measurement of the one- and two-way metrics (OWAMP and TWAMP
> respectively). These protocols will be advanced along the standards
> track. The work of the WG will take into account the suitability of
> measurements for automation, in order to support large-scale measurement
> efforts. This may result in further developments in protocols such as
> OWAMP and TWAMP. Agreement about the definitions of metrics and methods
> of measurement enables accurate, reproducible, and equivalent results
> across different implementations. To this end, the WG defines and
> maintains a registry of metric definitions. The WG encourages work which
> assesses the comparability of measurements of IPPM metrics with metrics
> developed elsewhere.
> 
> 
> This is a purely editorial change: move definition and maintenance of
> registry to present tense, since we're doing it.
> 
> ---
> 
> OLD Paragraph 7:
> 
> The WG also encourages work which improves the availability of
> information about the context in which measurements were taken.
> 
> NEW Paragraph 7:
> 
> The WG also encourages work which improves the availability of
> information about the context in which measurements were taken, for
> example, measurement implementation information, conditions on the
> networks on which measurements are taken, or information about the data-
> plane topology of the measured network.
> 
> 
> This is an arguably editorial change. It makes it clear that work
> targeted at IPPM that focuses on measurement on the spatial
> characteristics of paths as context for metrics (e.g. parts of IOAM, but
> also work like draft-amf-ippm-route, which attempts to answer the
> question "what about traceroute" applied to the registry).
> 
> ---
> 
> We additionally propose to strike the (out of date) near term milestones
> from the charter text; milestones are separately managed.
> 
> 
> Beyond these updates, we propose to leave the charter unchanged. We've
> got some time to have this discussion on the agenda in Prague, but in
> the meantime please send your thoughts on this proposal to the mailing
> list.
> 
> 
> Many thanks, best regards,
> 
> Brian (for the chairs)