Re: [Explicit-meas] Comparing Alternate Marking and Explicit Flow Measurements (Spin bit, ...)

["Lubashev, Igor" <[e-mail]>]

Thank you, Ian and Spencer.

Sorry for top posting (the thread could otherwise get a little long and I am using Outlook…).

Ian, we did consider privacy implications of the markings. Because all markings and internal counters are completely reset when there is a CID change, we did not see the markings compromise linkability during migrations. Otherwise, since the markings are minimal, we see them only disclosing the information they are meant to disclose – upstream/downstream loss.  We do discuss privacy-related implications of disclosing upstream/downstream loss in https://datatracker.ietf.org/doc/html/draft-ferrieuxhamchaoui-quic-lossbits-03#section-8. However, the discussion in the ID is theoretical and is not a result of large global study whose results are confirmed by independent third parties. I would be happy to collaborate on this with an interested PhD student or another researcher.

Spencer, I will review the PLUS minutes. In a setup when endpoints are sending information to unknown third parties, my immediate concern is less with the third parties being unknown and more with the extent of the information sent being unknown. After all, endpoints are already sending information to “unknown third parties” on path every time they communicate over the Internet. With the explicit measurements proposals, the set of “unknown third parties” is not changing. What endpoints must focus on is the information content of the signal. In any case, the abovementioned draft allows for endpoints to decide whether this signal is being sent AND ensures that a certain portion of all connections does not use this signaling (so connections explicitly opting out do not stand out).

Delivery of qlog to specific operators is possible, but it does not help much to locate the source of the loss/delay (upstream of the operator? downstream? in the operator’s own systems?) – the primary goal of this draft.

Very best,


  *   Igor

From: Spencer Dawkins at IETF <[e-mail]>
Sent: Monday, March 29, 2021 9:04 AM

Hi, Ian,

On Mon, Mar 29, 2021 at 7:01 AM Ian Swett <ianswett=[e-mail]<mailto:[e-mail]>> wrote:
Thanks for the maprg slides Igor, it's great to hear you'd enable this if it was standardized.

<Taking off my WG chair hat>

I didn't notice that you're an IPPM co-chair - congratulations to you, and to IPPM!

Has Akamai conducted any privacy analysis it would be willing to share with the IETF?  For this to be widely deployed, I think that will need to be done, similar to the spin bit.

Exactly, because (please see below)

On Sat, Mar 27, 2021 at 4:29 PM Lubashev, Igor <[e-mail]<mailto:[e-mail]>> wrote:
Thank you, Mauro.

I would like to point out to IPPM WG that some of the proposed measurement techniques have already been implemented and had been running on the Internet in 2019 and 2020.  Namely, Akamai has implemented Loss Bits (L+Q) and enabled them on a large portion of QIUC traffic served to Orange Telecom users in several countries, while Orange implemented observers that collected and analyzed the measurements.  We have discussed the measurements and techniques in MAPRG during IETF-105 (and at other WGs and meetings).

Here is our IETF-105 MAPRG presentation discussing the data https://datatracker.ietf.org/meeting/105/materials/slides-105-maprg-packet-loss-signaling-for-encrypted-protocols-01<https://urldefense.com/v3/__https:/datatracker.ietf.org/meeting/105/materials/slides-105-maprg-packet-loss-signaling-for-encrypted-protocols-01__;!!GjvTz_vk!Dujw9V18lqGrXL8GUOd79MFlesJpdVXn5R0Q-0m9rU-AQOyxjrwEflbtDIcQmfQ$>.

In short, we found that unidirectional observations of QUIC traffic with L+Q bits alone to be effective for measuring both upstream and downstream packet loss and characterizing the magnitude of packet reordering.

During the last meeting Ian asked whether Akamai would implement and enable these measurements techniques for QUIC on the large scale, if all the relevant drafts get standardized by IETF.  The answer is that, yes, we would.  We are very interested in doing what it takes to improve Internet performance for the users, and we would work hard to implement techniques that can help network operators to improve QOS on their networks.

- Igor

> -----Original Message-----
> From: Cociglio Mauro <mauro.cociglio=[e-mail]<mailto:[e-mail]>>
> Sent: Tuesday, March 16, 2021 9:48 AM
> To: IETF IPPM WG ([e-mail]<mailto:[e-mail]>) <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>
> Cc: [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
> Subject: [ippm] Comparing Alternate Marking and Explicit Flow
> Measurements (Spin bit, ...)
>
> Hi.
> Last Friday during the IPPM meeting,  after the "Explicit Flow Measurements"
> draft presentation
> (https://urldefense.com/v3/__https://tools.ietf.org/html/draft-mdt-ippm-<https://urldefense.com/v3/__https:/tools.ietf.org/html/draft-mdt-ippm->
> explicit-flow-measurements-01__;!!GjvTz_vk!CC-VjNQml5au1pXfCN50J-D-
> O99Zvrz686frIu4shWxPEipwQd7TTuOM3NTyNCU$ ), Greg Mirsky raised an
> issue that I think is very important.
> The question is about differences and similarities between the two types of
> production traffic packet marking for performance measurements, proposed
> in IETF and initiated in the IPPM WG: Alternate Marking and Explicit Flow
> Measurements.
>
> The first technique known as Alternate Marking or AM/PM (Alternate
> Marking Performance Monitoring) is defined, in general terms, in RFC8321
> (the point-to-point version) and RFC8889 (the multipoint version).
> It is essentially a Telco measurement, born to measure packet delay and loss
> between the input and output of a network, or between 2 internal points of
> the network, in order to identify and localize a problem. It is a network
> measure and it is the network operator that performs the marking by
> modifying packets on the fly.
> The strength of this technique comes from the decoupling of marking and
> measurement. We can mark all traffic, using a fixed marking interval (typically
> "big": from 1 second up to 5 minutes), then we decide what to measure
> based on the resources I want to use. In case of packet loss measurement
> we can start from a single packet counter for all traffic for each measurement
> point (possibility described in RFC8889), to have a network measurement, to
> arrive to a counter for each point-to-point connection you want to monitor
> (as described in RFC8321).
> In order to obtain the measurement it is necessary to compare the data
> collected from at least 2 measurement points (counters for packet loss,
> timestamps for delay). Then a "communication" between measurement
> points, or with a Network Measurement Center, is needed.
> There are already commercial implementations of this technique (also for
> IPv4) and IETF drafts that are standardizing it for various protocols (IPv6,
> MPLS, Segment Routing, BIER, ...).
> The Alternate Marking methodology is evolving into the draft "Big Data
> AltMark" (https://urldefense.com/v3/__https://tools.ietf.org/html/draft-<https://urldefense.com/v3/__https:/tools.ietf.org/html/draft->
> c2f-ippm-big-data-alt-mark-01__;!!GjvTz_vk!CC-VjNQml5au1pXfCN50J-D-
> O99Zvrz686frIu4shWxPEipwQd7TTuOM_yNIIgs$ ) that defines point-to-point
> flows measurements applying post processing to performance data collected
> by sampling a single network multipoint flow.
>
> The second marking technique for performance monitoring of packet
> networks has been called Explicit Flow Measurements (EFM), and is more
> recent because it's born with the Spin bit RTT measurement. And it came
> about primarily to have an end-to-end performance measure, from the
> terminal, on which an application is running, to the server at the opposite
> end of the network. EFM can be seen as complementary measures to
> Alternate Marking.

One question on this.  As a QUIC WG member I was under the impression the spin bit would enable measurement within networks, but I agree that it is best as an end-to-end measurement.  However, network endpoints can and do(ie: qlog) export metrics and traces to provide detailed information that's much richer than spin or loss bits, which makes me less clear on the value of EFM.  Igor indicated it worked as intended, but there is the separate question of whether it provides enough value on top of endpoint logging and Alternate Marking.

Brian Trammell and I have both been chatting with the TSV ADs about topics that the PLUS BOF was relevant to. My recollection from https://datatracker.ietf.org/doc/minutes-96-plus/<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/minutes-96-plus/__;!!GjvTz_vk!Dujw9V18lqGrXL8GUOd79MFlesJpdVXn5R0Q-0m9rU-AQOyxjrwEflbtMqEnAT0$> was that most of the concern expressed by the SEC types in the room was about endpoints sending information to network entities that the user was not aware of (and, truth be told, that one concern detailed PLUS forevermore). (*)

(*) corrections on that point are welcome, but I'd be surprised if anyone offered one. It wasn't a close BOF result.

It's obvious to me that relying on qlog plus delivery of qlog output to network elements would be useful, and a lot more likely to be accurate than inferences from a heavily encrypted byte stream, but please ask very early in the process whether there's a similar concern about unwitting (from the user's perspective) leakage to operators, or just bite the bullet and do the privacy analysis now.

And Good Luck.

Best,

Spencer

> It requires certain characteristics of the protocols to which it can be applied,
> which are client-server, and it is particularly convenient for protocols that
> prevent the marking of packets on the fly (e.g. QUIC), because the marking
> occurs only on the end-points of the connection.
> The disadvantage with respect to the previous technique is that it always
> works for client-server point-to-point connection, it is not possible to
> aggregate measurements saving on monitoring resources as described in
> RFC8889. The advantage is that it can also work with a single monitoring
> point, even if having more points enhances it and allows intradomain
> measurements. With a single measurement point you can obtain end-to-end
> measures (Spin bit, Delay bit for delay and Loss bit, rT loss bit for packet loss)
> or end-to-observer measures (sQuare bit and Reflection bit for packet loss).
> End-to-observer measurements and scalability considerations make it
> particularly convenient to place a measurement point on the client (see
> https://urldefense.com/v3/__https://tools.ietf.org/html/draft-cnbf-ippm-<https://urldefense.com/v3/__https:/tools.ietf.org/html/draft-cnbf-ippm->
> user-devices-explicit-monitoring-01__;!!GjvTz_vk!CC-VjNQml5au1pXfCN50J-
> D-O99Zvrz686frIu4shWxPEipwQd7TTuOMvZGbbWs$ ).
>
> Best Regards.
>
> Mauro
> _____________________
> Mauro Cociglio
> TIM - Telecom Italia
> Via G. Reiss Romoli, 274
> 10148 - Torino (Italy)
> Tel.: [phone]<tel:+39%20011%20228%205028>
> Mobile: [phone]<tel:+39%20335%20766%209751>
> _____________________
>
>
> TIM - Uso Interno - Tutti i diritti riservati.
>
> Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle
> persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla
> conoscenza di queste informazioni sono rigorosamente vietate. Qualora
> abbiate ricevuto questo documento per errore siete cortesemente pregati di
> darne immediata comunicazione al mittente e di provvedere alla sua
> distruzione, Grazie.
>
> This e-mail and any attachments is confidential and may contain privileged
> information intended for the addressee(s) only. Dissemination, copying,
> printing or use by anybody else is unauthorised. If you are not the intended
> recipient, please delete this message and any attachments and advise the
> sender by return e-mail, Thanks.
>
> Rispetta l'ambiente. Non stampare questa mail se non è necessario.
>
> _______________________________________________
> ippm mailing list
> [e-mail]<mailto:[e-mail]>
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ippm_<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/ippm_>
> _;!!GjvTz_vk!CC-VjNQml5au1pXfCN50J-D-
> O99Zvrz686frIu4shWxPEipwQd7TTuOMMXmeW4s$