Re: [lmap] LMAP regulator use case

[Sørensen, Frode <[e-mail]>]

Dear all,

Thanks for your feedback on the regulator use cases, Phil. I support many of your suggestions for improving the text of the use cases, in particular, mentioning comparability in current section 2.2, but also mentioning fixed and mobile.

In my view, sub cases 4.2, 4.4 and 4.5 are relevant, but also important. I agree with Klaus' and James' comments, thank you both. I support the additions proposed by James.

I would also like to add a couple of other aspects regarding these sub cases:

4.2 Promoting end user empowerment
I agree with Phil that this is also aka end user use case. But the regulator is an important stakeholder in the use case. Klaus mentions some regulatory aspects. In addition, it is a fact that in some jurisdictions the regulator, or a third party on behalf of the regulator, operates a speed meter for example. (However, I do not think Ofcom (UK) does this.) An important reason for the regulator's engagement is to achieve comparability between operators.

4.4 Detecting degradation of the Internet access
4.5 Detecting degradation of specific applications

These use cases already exit as regulator use cases externally to LMAP. For example, the "BEREC Guidelines for quality of service in the scope of net neutrality" available at
http://berec.europa.eu/eng/document_register/subject_matter/berec/regulatory_best_practices/guidelines/?doc=1101
describes the background for these use cases in chapters 4 and 5 respectively:
4. Degradation of Internet access service as a whole
5. Degradation of Internet access service with regard to individual applications

Use case 4.4 is probably also relevant for FCC since they in their press release regarding the open Internet report & order say for example "We will closely monitor the robustness and affordability of broadband Internet access services, with a particular focus on any signs that specialized services are in any way retarding the growth of or constricting capacity available for broadband Internet access service."

It is also interesting that there already exist measurement tools that support use case 4.5, notable Glasnost which is available at http://www.measurementlab.net/tools/glasnost.

However, I agree with Phil that these two use cases are challenging to perform. All the more, large scale measurements are needed to perform them in an accurate and verifiable way. Therefore, I think these use cases should be kept.

Sorry that I am not able to attend this IETF meeting, but I am looking forward to join you at the next meeting,
Frode


Fra: James Miller [mailto:[e-mail]]
Sendt: 6. november 2013 07:47
Til: Nieminen Klaus
Kopi: [e-mail]; Sørensen, Frode; [e-mail]; Marc Linsner; Henning Schulzrinne
Emne: Re: [lmap] LMAP regulator use case

I would propose adding in section 2 or 4 some text describing a regulators need to ensure that policies intended to protect the privacy interests of a potential tester can be enforced using features available in the system.  Recognizing that http://www.ietf.org/id/draft-folks-lmap-framework-00.txt and in the lists there's a view that a single entity will be overseeing a large scale measurement, there is a clear need for a test infrastructure to include tools to enforce the privacy policies of the managing entity and coordinate collaborative activity.  A desire to protect the privacy of testers contributing to our mobile measurement effort lead our work on the project, and our decision to adopt changes to our fixed technical infrastructure and data model to support a totally anonymous collection at the MA and communications between controllers, and through the collectors and the test results collected.  I understand there has been discussion on this point in the past and we had included this in the Section 7 Security Considerations of our original I-D.  In any event I think the regulator use case benefits from calling the point out specifically.
http://tools.ietf.org/id/draft-schulzrinne-lmap-requirements-00.txt

Proposed addition in Section 2.2:
:: Regulators role in the development and enforcement of broadband Internet access service policies also require that the measurement approaches meet a high level of verifiability, accuracy and provider-independence to support valid and meaningful comparisons of Internet access service performance.   Regulators also require that policies intended to protect the privacy interests of potential testers can be enforced using features available in the system.


Naturally any use case is going to have some variation among the specific practices of the parties.  On Klaus and Frode's comments (and excellent draft contributions thanks!) I would chime in to add that "Transparency" is a fundamental feature of the rules intended to protect the Open Internet (or network neutrality in other parlance).  Our rule is intended to enable "consumers to make informed choices regarding use of such services" but also "for content, application, service, and device providers to develop, market, and maintain Internet offerings." 47 CFR 8.3 - Transparency.

Whether protecting consumers is moved I think you find the thought and regulatory use case solidly grounded in the US approach as well.  I would also suggest adding some text to emphasize that Transparency is also important to enabling development in the ecosystem.

Proposed addition in Section 4.1:
:: End users need effective transparency to be able to make informed choices throughout the different stages of their relationship with ISPs, when selecting Internet access service offers, and when considering switching service offer within an ISP or to an alternative ISP.  Transparency in Internet performance information is also important to content, application, service, and device providers' to ensure they are able to develop, market, and maintain Internet offerings.  Quality information about service offering can be provided based on performance measurements of the services provided in the market.


On Klaus' comments on what in US jurisdictions might be understand as "blocking" or "discriminatory" conduct, I'd add that US law includes similar provisions. 47 C.F.R.§§ 8.5, 8.7.  Recognizing that these topics including "Quality of Service" are evolving and differ in many respects across jurisdictions, a high level description focused on the value of performance information to regulators in this regard may be helpful in the section.

I would also breach another topic for future discussion on assessing quality on the test network itself.  To maintain comparability and quality of the collected data, the architecture itself must be monitored with auditing of whether the infrastructure is functioning correctly, e.g. server health and connectivity. MA side is much easier to keep a handle on with special purpose probes (such as with our fixed "whitebox" approach) but server side checks on and between controllers is also a very important issue to keep an eye on.  I don't have a specific proposal how to address this in an I-D right now but I'd note it as something any large scale measurement manager will need to address.
--
[e-mail]<mailto:[e-mail]>

"Wovon man nicht sprechen kann, darüber muß man schweigen."
(What we cannot speak about we must pass over in silence.  )
Ludwig Josef Johann Wittgenstein

On Wed, Nov 6, 2013 at 12:19 AM, Nieminen Klaus <[e-mail]<mailto:[e-mail]>> wrote:
Hi Philip, Frode and all,

As one of the  regulator friends that Frode mentioned, I would like to tell how we see these issues in Finland as I'm working for the Finnish telco regulator.

4.2 Promoting end user empowerment
I would like to see this chapter to be included. As said publishing transparent and comparable information is important but so it's also end user's ability to verify the performance of the Internet access service provided, comparing the measurement results achieved with the information given in the contract. If the contracted service quality level is not met, end users can use these results when complaining to the ISP to achieve improvement and compensation.

If you want to include it to end user requirements it is ok for me to reference it from the regulator part. However I do not want it to be totally deleted as it is also our requirement not only end user one.

4.4 Detecting degradation of the Internet access
4.5 Detecting degradation of specific applications
Yes I know that this is a hard problem how to measure NN. However it is a clear regulatory requirement at least in the EU. See Directive 2009/136/EC http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2009:337:0011:01:EN:HTML

According to Art 22(3), "in order to prevent the degradation of service and the hindering or slowing down of traffic over networks, Member States shall ensure that national regulatory authorities are able to set minimum quality of service requirements on an undertaking or undertakings providing public communications networks." The reach of this regulatory tool has been extensively examined by BEREC, which sees it as the third and last step of the regulatory approach to net neutrality breaches

So we do need tools for this purpose as Frode has described.

best regards,

- Klaus -
___________________________________________
KLAUS NIEMINEN
Communications Network Specialist
Finnish Communications Regulatory Authority
P.O. Box 313
FIN-00181 HELSINKI, FINLAND
tel.: [phone]<tel:%2B358%2029%205390%20528>
fax: [phone]<tel:%2B358%209%206966%20873>
e-mail: [e-mail]<mailto:[e-mail]>
www: http://www.ficora.fi

-----Original Message-----
From: [e-mail]<mailto:[e-mail]> [mailto:[e-mail]<mailto:[e-mail]>]
Sent: 6. marraskuuta 2013 4:42
To: [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
Subject: Re: [lmap] LMAP regulator use case

Frode
thanks very much for the comments - and it is excellent to have detailed text suggestions.

2.2
I think i'm basically OK with this replacing the current S2.2 (including the subsections) I suggest that it should mention comparability in the final para (as well as in the penultimate one) - personally I think this is the most important characteristic from a regulator's point of view, so that it's fair to compare measurements they make across the different ISPs.
the one thing in the current S2.2 (actually in S2.2.3) that I think is worth keeping is a mention of fixed and mobile. I think this could be handled by adding to the Intro (S1) something like: "Measurements will take place on both fixed and mobile access"

Happy to add a new chapter 4, but comments about some of your sub use cases

4.1 Promoting competition through transparency
>>For competition to successfully discipline operators' behaviour in the interests of their customers, end users must be fully aware of the characteristics of the ISPs' access offers.
"must be fully aware" over-states things.

>>The system can be operated by a regulator or a measurement provider.
add "on behalf of the regulator"

>>  However, the quality of an Internet access service is also determined by the connectivity to the rest of the Internet. Therefore test configurations which measure beyond the ISP's own network may also be relevant in some cases.
true. perhaps worth mentioning that the ISPs can influence this to some extent - how well they're connected to the Internet, how they cache popular content etc.

4.2 Promoting end user empowerment
i think this can be deleted. this is the end user use case, not the regulator use case.

4.3 Monitoring overall market development good for me

4.4 Detecting degradation of the Internet access
4.5 Detecting degradation of specific applications

I don't like these sections about testing for whether the ISP is being net neutral.
I think it would be very hard to design tests.
In part this is because net neutrality is more of a political concept than technical one (in my view)

i tihnk some of the stuff in S4.4 is really part of 4.3, as it's a market development issue (the internet has got slower, and doesnt meet the policy objective of Superfast broadband by Year 2015 - or whatever)-

best wishes
phil

________________________________________
From: [e-mail]<mailto:[e-mail]> [[e-mail]<mailto:[e-mail]>] On Behalf Of Sørensen, Frode [[e-mail]<mailto:[e-mail]>]
Sent: 30 October 2013 14:55
To: [e-mail]<mailto:[e-mail]>
Subject: [lmap] LMAP regulator use case

Marc, Phil, Trevor, all,

I have prepared a proposed description of a regulator use case for the LMAP use case ID. I noticed the "advertisement" for further input from regulators for the use case a couple of weeks ago. Finally I have managed to find sufficient time to contribute to the ID, supported by a couple of "regulator friends" from the Net Neutrality Working Group of the BEREC organization, representing European national regulators. The proposal consists of two parts; a modification of section 2.2 "Regulators", plus a new chapter after chapter 3 "Details of ISP use case", titled "Details of regulator use case".

Thanks,
Frode


Proposed update:

2.2 Regulators
-------------------
Regulators in jurisdictions around the world are responding to consumers' adoption of Internet access services for traditional telecommunications and media services by promoting competition among providers of electronic communications, to ensure that users derive maximum benefit in terms of choice, price, and quality.

Some jurisdictions have responded to a need for greater information about Internet access service performance in the development of regulatory policies and approaches for broadband technologies by developing large-scale measurement programs. Programs such as the U.S. Federal Communications Commission's Measuring Broadband America, European Commission's Quality of Broadband Services in the EU reports and a growing list of other programs employ a diverse set of operational and technical approaches to gathering data to perform analysis and reporting on diverse aspects of broadband performance.

While each jurisdiction responds to distinct consumer, industry, and regulatory concerns, much commonality exists in the need to produce datasets that are able to compare multiple Internet access service providers, diverse technical solutions, geographic and regional distributions, and marketed and provisioned levels and combinations of broadband Internet access services. In some jurisdictions, the role of measuring is provided by a measurement provider.

Measurement providers measure network performance from users towards multiple content and application providers, included dedicated test measurement servers, to show a performance of the actual Internet access service provided by different ISPs. Users need to know the performance that are achieving from their own ISP. In addition, they need to know the performance of other ISPs of same location as background information for selecting their ISP. Measurement providers will provide measurement results with associated measurement methods and measurement metrics.
_______________________________________________
lmap mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/lmap