Re: [Explicit-meas] Comparing Alternate Marking and Explicit Flow Measurements (Spin bit, ...)

[Spencer Dawkins at IETF <[e-mail]>]

Hi, Ian,

On Mon, Mar 29, 2021 at 7:01 AM Ian Swett <ianswett=
[e-mail]> wrote:

> Thanks for the maprg slides Igor, it's great to hear you'd enable this if
> it was standardized.
>
> <Taking off my WG chair hat>
>

I didn't notice that you're an IPPM co-chair - congratulations to you, and
to IPPM!


> Has Akamai conducted any privacy analysis it would be willing to share
> with the IETF?  For this to be widely deployed, I think that will need to
> be done, similar to the spin bit.
>

Exactly, because (please see below)


> On Sat, Mar 27, 2021 at 4:29 PM Lubashev, Igor <[e-mail]>
> wrote:
>
>> Thank you, Mauro.
>>
>> I would like to point out to IPPM WG that some of the proposed
>> measurement techniques have already been implemented and had been running
>> on the Internet in 2019 and 2020.  Namely, Akamai has implemented Loss Bits
>> (L+Q) and enabled them on a large portion of QIUC traffic served to Orange
>> Telecom users in several countries, while Orange implemented observers that
>> collected and analyzed the measurements.  We have discussed the
>> measurements and techniques in MAPRG during IETF-105 (and at other WGs and
>> meetings).
>>
>> Here is our IETF-105 MAPRG presentation discussing the data
>> https://datatracker.ietf.org/meeting/105/materials/slides-105-maprg-packet-loss-signaling-for-encrypted-protocols-01
>> .
>>
>> In short, we found that unidirectional observations of QUIC traffic with
>> L+Q bits alone to be effective for measuring both upstream and downstream
>> packet loss and characterizing the magnitude of packet reordering.
>>
>> During the last meeting Ian asked whether Akamai would implement and
>> enable these measurements techniques for QUIC on the large scale, if all
>> the relevant drafts get standardized by IETF.  The answer is that, yes, we
>> would.  We are very interested in doing what it takes to improve Internet
>> performance for the users, and we would work hard to implement techniques
>> that can help network operators to improve QOS on their networks.
>>
>> - Igor
>>
>> > -----Original Message-----
>> > From: Cociglio Mauro <mauro.cociglio=[e-mail]>
>> > Sent: Tuesday, March 16, 2021 9:48 AM
>> > To: IETF IPPM WG ([e-mail]) <[e-mail]>;
>> [e-mail]
>> > Cc: [e-mail]; [e-mail]
>> > Subject: [ippm] Comparing Alternate Marking and Explicit Flow
>> > Measurements (Spin bit, ...)
>> >
>> > Hi.
>> > Last Friday during the IPPM meeting,  after the "Explicit Flow
>> Measurements"
>> > draft presentation
>> > (
>> https://urldefense.com/v3/__https://tools.ietf.org/html/draft-mdt-ippm-
>> > explicit-flow-measurements-01__;!!GjvTz_vk!CC-VjNQml5au1pXfCN50J-D-
>> > O99Zvrz686frIu4shWxPEipwQd7TTuOM3NTyNCU$ ), Greg Mirsky raised an
>> > issue that I think is very important.
>> > The question is about differences and similarities between the two
>> types of
>> > production traffic packet marking for performance measurements, proposed
>> > in IETF and initiated in the IPPM WG: Alternate Marking and Explicit
>> Flow
>> > Measurements.
>> >
>> > The first technique known as Alternate Marking or AM/PM (Alternate
>> > Marking Performance Monitoring) is defined, in general terms, in RFC8321
>> > (the point-to-point version) and RFC8889 (the multipoint version).
>> > It is essentially a Telco measurement, born to measure packet delay and
>> loss
>> > between the input and output of a network, or between 2 internal points
>> of
>> > the network, in order to identify and localize a problem. It is a
>> network
>> > measure and it is the network operator that performs the marking by
>> > modifying packets on the fly.
>> > The strength of this technique comes from the decoupling of marking and
>> > measurement. We can mark all traffic, using a fixed marking interval
>> (typically
>> > "big": from 1 second up to 5 minutes), then we decide what to measure
>> > based on the resources I want to use. In case of packet loss measurement
>> > we can start from a single packet counter for all traffic for each
>> measurement
>> > point (possibility described in RFC8889), to have a network
>> measurement, to
>> > arrive to a counter for each point-to-point connection you want to
>> monitor
>> > (as described in RFC8321).
>> > In order to obtain the measurement it is necessary to compare the data
>> > collected from at least 2 measurement points (counters for packet loss,
>> > timestamps for delay). Then a "communication" between measurement
>> > points, or with a Network Measurement Center, is needed.
>> > There are already commercial implementations of this technique (also for
>> > IPv4) and IETF drafts that are standardizing it for various protocols
>> (IPv6,
>> > MPLS, Segment Routing, BIER, ...).
>> > The Alternate Marking methodology is evolving into the draft "Big Data
>> > AltMark" (
>> https://urldefense.com/v3/__https://tools.ietf.org/html/draft-
>> > c2f-ippm-big-data-alt-mark-01__;!!GjvTz_vk!CC-VjNQml5au1pXfCN50J-D-
>> > O99Zvrz686frIu4shWxPEipwQd7TTuOM_yNIIgs$ ) that defines point-to-point
>> > flows measurements applying post processing to performance data
>> collected
>> > by sampling a single network multipoint flow.
>> >
>> > The second marking technique for performance monitoring of packet
>> > networks has been called Explicit Flow Measurements (EFM), and is more
>> > recent because it's born with the Spin bit RTT measurement. And it came
>> > about primarily to have an end-to-end performance measure, from the
>> > terminal, on which an application is running, to the server at the
>> opposite
>> > end of the network. EFM can be seen as complementary measures to
>> > Alternate Marking.
>>
>
> One question on this.  As a QUIC WG member I was under the impression the
> spin bit would enable measurement within networks, but I agree that it is
> best as an end-to-end measurement.  However, network endpoints can and
> do(ie: qlog) export metrics and traces to provide detailed information
> that's much richer than spin or loss bits, which makes me less clear on the
> value of EFM.  Igor indicated it worked as intended, but there is the
> separate question of whether it provides enough value on top of endpoint
> logging and Alternate Marking.
>

Brian Trammell and I have both been chatting with the TSV ADs about topics
that the PLUS BOF was relevant to. My recollection from
https://datatracker.ietf.org/doc/minutes-96-plus/ was that most of the
concern expressed by the SEC types in the room was about endpoints sending
information to network entities that the user was not aware of (and, truth
be told, that one concern detailed PLUS forevermore). (*)

(*) corrections on that point are welcome, but I'd be surprised if anyone
offered one. It wasn't a close BOF result.

It's obvious to me that relying on qlog plus delivery of qlog output to
network elements would be useful, and a lot more likely to be accurate than
inferences from a heavily encrypted byte stream, but please ask very early
in the process whether there's a similar concern about unwitting (from the
user's perspective) leakage to operators, or just bite the bullet and do
the privacy analysis now.

And Good Luck.

Best,

Spencer


> > It requires certain characteristics of the protocols to which it can be
>> applied,
>> > which are client-server, and it is particularly convenient for
>> protocols that
>> > prevent the marking of packets on the fly (e.g. QUIC), because the
>> marking
>> > occurs only on the end-points of the connection.
>> > The disadvantage with respect to the previous technique is that it
>> always
>> > works for client-server point-to-point connection, it is not possible to
>> > aggregate measurements saving on monitoring resources as described in
>> > RFC8889. The advantage is that it can also work with a single monitoring
>> > point, even if having more points enhances it and allows intradomain
>> > measurements. With a single measurement point you can obtain end-to-end
>> > measures (Spin bit, Delay bit for delay and Loss bit, rT loss bit for
>> packet loss)
>> > or end-to-observer measures (sQuare bit and Reflection bit for packet
>> loss).
>> > End-to-observer measurements and scalability considerations make it
>> > particularly convenient to place a measurement point on the client (see
>> >
>> https://urldefense.com/v3/__https://tools.ietf.org/html/draft-cnbf-ippm-
>> > user-devices-explicit-monitoring-01__;!!GjvTz_vk!CC-VjNQml5au1pXfCN50J-
>> > D-O99Zvrz686frIu4shWxPEipwQd7TTuOMvZGbbWs$ ).
>> >
>> > Best Regards.
>> >
>> > Mauro
>> > _____________________
>> > Mauro Cociglio
>> > TIM - Telecom Italia
>> > Via G. Reiss Romoli, 274
>> > 10148 - Torino (Italy)
>> > Tel.: [phone] <+39%20011%20228%205028>
>> > Mobile: [phone] <+39%20335%20766%209751>
>> > _____________________
>> >
>> >
>> > TIM - Uso Interno - Tutti i diritti riservati.
>> >
>> > Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle
>> > persone indicate. La diffusione, copia o qualsiasi altra azione
>> derivante dalla
>> > conoscenza di queste informazioni sono rigorosamente vietate. Qualora
>> > abbiate ricevuto questo documento per errore siete cortesemente pregati
>> di
>> > darne immediata comunicazione al mittente e di provvedere alla sua
>> > distruzione, Grazie.
>> >
>> > This e-mail and any attachments is confidential and may contain
>> privileged
>> > information intended for the addressee(s) only. Dissemination, copying,
>> > printing or use by anybody else is unauthorised. If you are not the
>> intended
>> > recipient, please delete this message and any attachments and advise the
>> > sender by return e-mail, Thanks.
>> >
>> > Rispetta l'ambiente. Non stampare questa mail se non è necessario.
>> >
>> > _______________________________________________
>> > ippm mailing list
>> > [e-mail]
>> > https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ippm_
>> > _;!!GjvTz_vk!CC-VjNQml5au1pXfCN50J-D-
>> > O99Zvrz686frIu4shWxPEipwQd7TTuOMMXmeW4s$
>>
>