Re: [ippm] New Liaison Statement, "LS - Harmonization of IP Capacity and Latency Parameters: Revision of Draft Rec. Y.1540 on IP packet transfer performance parameters and New Annex A with Lab Evaluation Plan"

[Greg Mirsky <[e-mail]>]

Hi Al, and All,
I agree with the response to the liaison.

Regards,
Greg

On Thu, Mar 28, 2019 at 4:43 PM MORTON, ALFRED C (AL) <[e-mail]>
wrote:

> IPPM,
>
> Following a side-meeting discussion of the Subject Liaison,
> and several individuals indicating continuing interest
> in this work,  I would like to propose the brief text of
> a Liaison reply.
>
> The WG will need to reach consensus on the text of
> the reply as a next step.
>
> Thanks to all who have expressed interest in this work!
> regards,
> Al
>
> Proposed Body of Liaison Reply (to all recipients)
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>
> To: ITU-T SG12, ITU-T SG11, BBF ETSI TC STQ, ETSI TC INT
>
> IETF IPPM working group thanks you for your liaison describing results from
> your evaluation of IP Capacity and Latency Metrics and  methods of
> measurement.
> The harmonization of these measurements is welcome in our industry.
>
> We have the following comments on the preliminary results, and
> the evaluation plan:
>
> After noting the efficacy of UDP measurements in your results,
> we can state that UDP transport is the basis for most IPPM measurement
> implementations, and IPPM's measurement protocols.
>
> When assessing IP Capacity and Latency, there should be
> no attempt to determine the technology involved in the path,
> such as AQM. The method should measure the path as a black box.
>
> The presence of packet-marking-sensitive technologies, such as
> Diff-serv queues add complexity to IP Capacity and Latency.
>
> There is not yet a metric that characterizes the BW-Delay product
> in the IPPM-literature.
>
> Live network testing may be affected by Network operator policy,
> it could change from the lab measurements - the test traffic is
> part of the background traffic. OTOH, the operator could be
> prioritizing test traffic, especially their own authorized testing.
>
> We also suggest to include QUIC-based measurements in your Lab
> evaluation, if possible.
>
> Where a model or post-processing needs to be applied to the measurements,
> this aspect must also be specified, especially for transport with multiple
> connections (averaging the connection performance).
>
> Some of the IPPM WG participants who indicated their interest in
> November 2019 have begun to share their comments informally,
> as planned.
>
> Please keep the IPPM WG informed of your progress, especially if there are
> ways in which IPPM WG participants assist further.
>
> <signed by chairs of IPPM WG>
>
>
> > -----Original Message-----
> > From: Liaison Statement Management Tool [mailto:[e-mail]]
> > Sent: Wednesday, March 13, 2019 1:23 PM
> > To: Bill Cerveny <[e-mail]>; Brian Trammell <[e-mail]
> >;
> > Tommy Pauly <[e-mail]>
> > Cc: Mirja Kühlewind <[e-mail]>; Scott Mansfield
> > <[e-mail]>; Bill Cerveny <[e-mail]>;
> MORTON,
> > ALFRED C (AL) <[e-mail]>; Spencer Dawkins
> > <[e-mail]>; IP Performance Measurement Discussion
> > List <[e-mail]>; Brian Trammell <[e-mail]>; itu-t-
> > [e-mail]; Tommy Pauly <[e-mail]>
> > Subject: New Liaison Statement, "LS - Harmonization of IP Capacity and
> > Latency Parameters: Revision of Draft Rec. Y.1540 on IP packet transfer
> > performance parameters and New Annex A with Lab Evaluation Plan"
> >
> > Title: LS - Harmonization of IP Capacity and Latency Parameters: Revision
> > of Draft Rec. Y.1540 on IP packet transfer performance parameters and New
> > Annex A with Lab Evaluation Plan
> > Submission Date: 2019-03-13
> > URL of the IETF Web page:
> > https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__datatracker.ietf.org_liaison_1632_&d=DwIDaQ&c=LFYZ-
> >
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=uLz_oPVMGlxwhg8pC1vN9Gw4ypYFZ
> > k506Axq2RLOyo0&s=ynxvwzaXF2tlJ0iSuBqmqYiJfPueJhlEBFtiyfxfNcg&e=
> > Please reply by 2019-05-01
> > From: Judit Kiss <[e-mail]>
> > To: Brian Trammell <[e-mail]>,Bill Cerveny
> > <[e-mail]>,Tommy Pauly <[e-mail]>
> > Cc: Scott Mansfield <[e-mail]>,Bill Cerveny
> > <[e-mail]>,Brian Trammell <[e-mail]>,IP Performance
> > Measurement Discussion List <[e-mail]>,Mirja Kühlewind
> > <[e-mail]>,Spencer Dawkins <[e-mail]
> >,itu-
> > [e-mail],Tommy Pauly <[e-mail]>
> > Response Contacts: A. C. Morton <[e-mail]>
> > Technical Contacts:
> > Purpose: For action
> >
> > Body: As an update to our communications from the Q17/12 Interim meeting
> > (Darmstadt, 16-17 October 2018) and the Geneva, 27 November - 6 December
> > 2018 meeting with the full SG12, we now share further results from the
> > Q17/12 Interim meeting (Berlin, 5-7 March 2019).
> >
> > We have completed nearly all planned tests using the Phase 1 Laboratory
> > test bed that supports development our revised Rec. Y.1540 and new Annex
> > A/Y.1540 specifications of IP Capacity and Latency methods of
> measurement.
> > We have compared several existing measurement methods based on TCP and
> UDP
> > transport. The Phase 2 Network tests are just starting now.
> >
> > We continue to invite interested parties to join the testing efforts, and
> > to coordinate on the development of a new generation of harmonized
> > specifications of IP Capacity performance metrics and methods of
> > measurement, and other key performance parameters.
> >
> > Our key results and current findings from the Berlin meeting are:
> >
> > •     The text of the IP Capacity definition and Methods of measurement,
> > and the Phase 1 and Phase 2 sections of the test and evaluation plan were
> > edited and agreed during the meeting. In the agreed text, the Phase 2
> > study will be carried-out in production networks, and lab networks if
> they
> > are made available in a timely fashion. This material constitutes
> > revisions to Y.1540 sections and the new ANNEX A/Y.1540. The meeting
> > agreed to seek Consent on this material in May, 2018.
> > •     The meeting agreed to prepare the Preliminary Test Summary
> > (presented for the first time at this meeting) as a new informative
> > APPENDIX to Y.1540, to provide the supporting test data for the
> > conclusions represented as requirements in the body and Annex A.
> > •     Even without correction factors, the UDP measurements are
> > considerably closer to the calibrated shaper rate than TCP, despite using
> > the most favourable circumstances for TCP (no added delay or background
> > traffic). With correction for headers that are viewed by the traffic
> > shaper, UDP measurements using iPerf 2 are within 200ppm of the
> configured
> > shaper rate. UDP-based measurements are the benchmark for capacity,
> > accurately assessing the “ground truth” of the traffic shaper rate under
> > all tested conditions.
> > •     TCP measurements using iPerf2 underestimate the shaper rate with or
> > without correction factors. Typical round-trip delay and the presence of
> > competing/background traffic tend to make TCP-based estimates of
> available
> > capacity appreciably worse.
> > •     Another contribution updated the overall academic Survey, adding
> > material to one section on WiFi performance and one new section Encrypted
> > stream Network QoS parameters and their relationship to QoE. Both topics
> > introduce a set of KPIs/metrics prioritized through machine learning, and
> > models of QoE.  The relevance to active testing includes the possible
> > additional metrics collected, or input for stream design.
> > •     It is clear from the surveys presented that Internet subscriber use
> > of TCP protocol is declining. Video and browser traffic has been shifting
> > to UDP and higher-layer reliability mechanisms for years, with the most
> > significant growth in 2018 due to CDN adoption.
> > •     There was a demonstration of a new prototype measurement tool that
> > makes UDP-based Capacity, Latency, and Loss measurements while searching
> > for the peak Capacity that can be supported on the tested path. The
> > meeting provided useful feedback for the developer.
> >
> > Q17/12 and collaborating SDOs have covered many new areas of
> > investigation, and are approaching the point where development of
> > coordinated specifications can proceed.
> > All group members continue to seek additional SDO and individual support,
> > participation and/or constructive review, so that together, we can
> proceed
> > toward the next generation of IP performance metrics.
> >
> > Attachments:
> > –     SG12-TD775 with revised Y.1540 clauses and new Y.1540 Annex A;
> > –     SG12-TD776 with new Y.1540 Appendix, Phase 1 Test Summary.
> > Attachments:
> >
> >     TD775
> >     https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__www.ietf.org_lib_dt_documents_LIAISON_liaison-2D2019-2D03-2D13-2Ditu-
> > 2Dt-2Dsg-2D12-2Dippm-2Dls-2Dharmonization-2Dof-2Dip-2Dcapacity-2Dand-
> > 2Dlatency-2Dparameters-2Drevision-2Dof-2Ddraft-2Drec-2Dy1540-2Don-2Dip-
> > 2Dpacket-2Dtransfer-2Dperfo-2Dattachment-2D1.docx&d=DwIDaQ&c=LFYZ-
> >
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=uLz_oPVMGlxwhg8pC1vN9Gw4ypYFZ
> > k506Axq2RLOyo0&s=dAR1A1k2C0NFX677nghi88SGItQwZxXCUB3dnMtgshw&e=
> >
> >     TD776
> >     https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__www.ietf.org_lib_dt_documents_LIAISON_liaison-2D2019-2D03-2D13-2Ditu-
> > 2Dt-2Dsg-2D12-2Dippm-2Dls-2Dharmonization-2Dof-2Dip-2Dcapacity-2Dand-
> > 2Dlatency-2Dparameters-2Drevision-2Dof-2Ddraft-2Drec-2Dy1540-2Don-2Dip-
> > 2Dpacket-2Dtransfer-2Dperfo-2Dattachment-2D2.docx&d=DwIDaQ&c=LFYZ-
> >
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=uLz_oPVMGlxwhg8pC1vN9Gw4ypYFZ
> > k506Axq2RLOyo0&s=S0LEXw6CpSLFV3ME16BU0wA4iUoUEjs5gLFLs6ElumA&e=
>
> _______________________________________________
> ippm mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/ippm
>