Re: [ippm] Proposed charter updates for IPPM

["Brian Trammell (IETF)" <[e-mail]>]

hi Al,

Briefly, inline below.

> On 09 Jul 2017, at 19:22, MORTON, ALFRED C (AL) <[e-mail]> wrote:
> 
> Hi Brian and Bill,
> 
> Thanks for preparing a new charter proposal and organizing
> this message to highlight the changes. I agree with your
> proposals.
> 
> I have one further proposal for the last sentence of
> paragraph 1.
> 
> OLD (and your NEW):
> Metrics developed by the IPPM WG are intended to provide unbiased
> quantitative performance measurements and not a value judgement.
> 
> I propose to delete/replace the last phrase, which I think has been present
> since the very first charter. IIRC, the concern was that contributions
> might attempt to establish service acceptance thresholds, or other
> numerical performance requirements/objectives using the metrics.
> The work of setting numerical objectives proceeded elsewhere
> (ITU-T Rec Y.1541), and the Earth continued to rotate.
> 
> If we want to ensure the status quo, we could say:
> s/and not a value judgement/but numerical thresholds for the metrics are out-of-scope/
> 
> I also suggest this change with MBM in mind, where we have
> testing outcomes of Pass/Fail/Inconclusive (PFI). Of course, the user
> supplies the target rate, RTT, and/or loss ratio numerical values,
> not the working group.
> 
> So we could express an explicit restriction that does not hamper MBM PFI:
> 
> NEW^2:
> Metrics developed by the IPPM WG are intended to provide unbiased
> quantitative performance measurements, but numerical thresholds for
> the metrics are out-of-scope.
> 
> Or, we could remove the phrase, and refer any contributions on
> numerical thresholds to ITU-T SG 12, Question 17, to avoid
> overlapping work with another SDO (coordination with this SDO
> should also be mentioned somewhere).
> 
> NEW^3:
> Metrics developed by the IPPM WG are intended to provide unbiased
> quantitative performance measurements.

Thanks for this history... this change (NEW^3) seems to make sense to me, and I'd propose we do it as well.

Cheers,

Brian


> 
> thanks for considering this proposal, and regards,
> Al
> 
>> -----Original Message-----
>> From: ippm [mailto:[e-mail]] On Behalf Of Brian Trammell
>> Sent: Sunday, July 09, 2017 11:24 AM
>> To: IETF IPPM WG
>> Subject: [ippm] Proposed charter updates for IPPM
>> 
>> Greetings, all,
>> 
>> Bill and I met briefly last week to discuss a proposed change to the
>> charter, as a way to resolve the rough consensus to adopt the IOAM work
>> within IPPM with concerns that have been raised that that work doesn't
>> really fit on the charter. We arrived at a minimal suggested set of
>> edits. Though inspired by IOAM, these aren't so much focused on bringing
>> IOAM on charter as they are making the charter reflect the direction
>> work in IPPM has been going over the five years since we last updated
>> the charter.
>> 
>> We propose only to touch three paragraphs, as below:
>> 
>> ---
>> 
>> OLD Paragraph 1:
>> 
>> The IP Performance Metrics (IPPM) Working Group develops and maintains
>> standard metrics that can be applied to the quality, performance, and
>> reliability of Internet data delivery services and applications running
>> over transport layer protocols (e.g. TCP, UDP) over IP. Specifying
>> network or lower layer OAM mechanisms is out of scope of the IPPM
>> charter. It also develops and maintains protocols for the measurement of
>> these metrics. These metrics are designed such that they can be used by
>> network operators, end users, or independent testing groups. Metrics
>> developed by the IPPM WG are intended to provide unbiased quantitative
>> performance measurements and not a value judgement.
>> 
>> NEW Paragraph 1:
>> 
>> The IP Performance Measurement (IPPM) Working Group develops and
>> maintains standard metrics that can be applied to the quality,
>> performance, and reliability of Internet data delivery services and
>> applications running over transport layer protocols (e.g. TCP, UDP) over
>> IP. It also develops and maintains methodologies and protocols for the
>> measurement of these metrics. These metrics are designed such that they
>> can be used by network operators, end users, or independent testing
>> groups. Metrics developed by the IPPM WG are intended to provide
>> unbiased quantitative performance measurements and not a value
>> judgement.
>> 
>> 
>> Two basic edits here:
>> 
>> (1) Rename working group from IP Performance Metrics to IP Performance
>> Measurement, and add "methodologies" to the set of things we do. In our
>> opinion, this reflects work that has been happening on charter for a
>> while: MBM, PDM, and alt-mark are all methodology work as opposed to
>> metric work.
>> 
>> (2) Remove the sentence "Specifying network or lower layer OAM
>> mechanisms is out of scope of the IPPM charter". In my personal opinion,
>> this isn't really necessary to bring the IOAM work on, since though IOAM
>> can be carried on lower-layer headers, it doesn't really specify an OAM
>> mechanism as I understand that term to be defined in the routing area.
>> However, this restriction on the IPPM doesn't appear to do anything
>> useful beyond reinforcing an artificial silo dividing data-plane and
>> control-plane measurement, make it difficult for us to work together
>> with other (e.g. INT or RTG area) working groups on performance
>> measurement, and cause confusion as to whether we can do a thing called
>> "IOAM" in IPPM.
>> 
>> ---
>> 
>> OLD Paragraph 6:
>> 
>> The WG has produced protocols for communication among test equipment to
>> enable the measurement of the one- and two-way metrics (OWAMP and TWAMP
>> respectively). These protocols will be advanced along the standards
>> track. The work of the WG will take into account the suitability of
>> measurements for automation, in order to support large-scale measurement
>> efforts. This may result in further developments in protocols such as
>> OWAMP and TWAMP. Agreement about the definitions of metrics and methods
>> of measurement enables accurate, reproducible, and equivalent results
>> across different implementations. To this end, the WG will define and
>> maintain a registry of metric definitions. The WG encourages work which
>> assesses the comparability of measurements of IPPM metrics with metrics
>> developed elsewhere.
>> 
>> NEW Paragraph 6:
>> 
>> The WG has produced protocols for communication among test equipment to
>> enable the measurement of the one- and two-way metrics (OWAMP and TWAMP
>> respectively). These protocols will be advanced along the standards
>> track. The work of the WG will take into account the suitability of
>> measurements for automation, in order to support large-scale measurement
>> efforts. This may result in further developments in protocols such as
>> OWAMP and TWAMP. Agreement about the definitions of metrics and methods
>> of measurement enables accurate, reproducible, and equivalent results
>> across different implementations. To this end, the WG defines and
>> maintains a registry of metric definitions. The WG encourages work which
>> assesses the comparability of measurements of IPPM metrics with metrics
>> developed elsewhere.
>> 
>> 
>> This is a purely editorial change: move definition and maintenance of
>> registry to present tense, since we're doing it.
>> 
>> ---
>> 
>> OLD Paragraph 7:
>> 
>> The WG also encourages work which improves the availability of
>> information about the context in which measurements were taken.
>> 
>> NEW Paragraph 7:
>> 
>> The WG also encourages work which improves the availability of
>> information about the context in which measurements were taken, for
>> example, measurement implementation information, conditions on the
>> networks on which measurements are taken, or information about the data-
>> plane topology of the measured network.
>> 
>> 
>> This is an arguably editorial change. It makes it clear that work
>> targeted at IPPM that focuses on measurement on the spatial
>> characteristics of paths as context for metrics (e.g. parts of IOAM, but
>> also work like draft-amf-ippm-route, which attempts to answer the
>> question "what about traceroute" applied to the registry).
>> 
>> ---
>> 
>> We additionally propose to strike the (out of date) near term milestones
>> from the charter text; milestones are separately managed.
>> 
>> 
>> Beyond these updates, we propose to leave the charter unchanged. We've
>> got some time to have this discussion on the agenda in Prague, but in
>> the meantime please send your thoughts on this proposal to the mailing
>> list.
>> 
>> 
>> Many thanks, best regards,
>> 
>> Brian (for the chairs)
>