[ippm] Liaison from Broadband Forum

[Brian Trammell <[e-mail]>]

Greetings, all, and happy (Gregorian) new year!

We have received a liaison from the Broadband Forum that consists of a series of questions to the IPPM WG and the experts therein. To facilitate discussion, the test of the liaison (provided in Microsoft Word format) is copied below; see http://datatracker.ietf.org/liaison/1221/ for the original liaison. Please reply to this message, answering questions inline; the chairs will prepare a response in mid-February from results of the mailing list discussion.

Best regards,

Brian


This liaison is to inform you of the progress of Working Text WT-304 Broadband Service Attributes and Performance Metrics and to request responses from the IPPM WG to a number of questions.
 
In our meeting of 3-7 December 2012, the Broadband Forum identified a number of performance metrics within RFCs that are of potential interest. We also identified potential standards gaps between the existing IP performance metrics and some current practices for large scale performance testing of broadband Internet access services, as exemplified by recent performance measurement trials conducted on behalf of the FCC and OFCOM. The questions below are related to these potential standards gaps and in general to the application of these metrics to performance testing of broadband Internet access services as envisioned by the WT-304 framework.
 
1. RFC 3148 addresses a framework for defining empirical bulk transfer capacity metrics over transport protocols like TCP. However, a number of factors may limit the applicability of the RFC with regard to broadband Internet access performance testing.

	1.a. It requires that a number of TCP behaviors (cwnd behavior; loss recovery; segment size; retransmission timer) be specified beyond the normal requirements applied to TCP implementations. Is RFC 3148 therefore appropriate for performance testing of broadband Internet access services that may use TCP implementations on a wide variety of existing platforms? Is it possible to quantify the variation associated with these behaviors, and are there ways to minimize or mitigate that variation without manipulating the TCP implementations?

	1.b. It does not discuss throughput limitations related to bandwidth-delay product or techniques (such as use of multiple TCP connections) commonly used to mitigate such issues. Are such techniques considered compatible with use of RFC 3148, and if so can you provide guidance on their use?

	1.c. It recommends the collection of a number of ancillary metrics in Section 3 that existing TCP implementations may or may not support. Are these metrics commonly retrievable in implementations that are not primarily intended as measurement tools? If they are not available, how significantly is the value of a “basic” throughput measurement diminished?
 

2. RFC 6349 addresses a framework for TCP throughput testing designed primarily for business class services. We note that this framework requires steps such as determination of path MTU, bottleneck bandwidth and RTT before actually performing throughput testing, which seems to make it unsuitable for testing of broadband Internet access services. Is this assessment correct?

 
3. Are there alternatives to RFC 3148 and RFC 6349 that we should consider for throughput performance testing of broadband Internet access services?

 
4. We are considering use of the following packet performance metrics for evaluating performance of residential Internet access services:

	4.a. Round trip packet delay per RFC 2681. Are there recommendations for achieving end-to-end clock synchronization in a residential Internet access environment that would enable measurement of one-way packet delay for these services?

	4.b. One way packet delay variation per RFC 3393, using the methodology discussed in Section 5 to use data from the measurements to correct errors due to unsynchronized clocks. We note that Section 5 is presented as a basis for discussion and that the considerations as listed require further investigation. Is the methodology discussed in Section 5 sufficiently complete to be considered in the context of WT-304 testing?
	4.c. Round trip packet loss per RFC 2680.

We welcome any comments you have on the packet performance metrics listed above.
 

5. RFC 2544 addresses device benchmarking in non-production networks, including non-TCP throughput testing that may be useful for capacity measurement. Is there an alternative we should consider for non-TCP capacity measurement for broadband Internet access services, i. e. production networks?


6. There are several areas where we are not aware of a solution documented within the IETF, but where we believe a solution would be useful. Is there any current or proposed work in the following areas?
	6.a. Measurement or estimation of performance relative to application-specific classes of traffic, such as:
        6.a.i.     Real time traffic such as VoIP; and
        6.a.ii.     Near-real time traffic such as streaming video.
	6.b. Measurement of DNS resolution time.
 

Thank you for your consideration of the above questions, and we look forward to your response. The BBF welcomes input on this project from the IETF, and we will continue to keep you informed of progress on the Working Text. For information, the upcoming quarterly Broadband Forum meetings are listed at the end of this liaison.