Re: [Explicit-meas] [EXT] Re: Explicit Flow Measurements implementations

"Bulgarella Fabio (Guest)" <fabio.bulgarella@guest.telecomitalia.it> Tue, 30 March 2021 13:41 UTC

Return-Path: <fabio.bulgarella@guest.telecomitalia.it>
X-Original-To: explicit-meas@ietfa.amsl.com
Delivered-To: explicit-meas@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 042003A12BC for <explicit-meas@ietfa.amsl.com>; Tue, 30 Mar 2021 06:41:58 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.989
X-Spam-Level:
X-Spam-Status: No, score=-1.989 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=guest.telecomitalia.it header.b=AyAugdCQ; dkim=pass (2048-bit key) header.d=guest.telecomitalia.it header.b=AyAugdCQ
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1KyAioQ6CWie for <explicit-meas@ietfa.amsl.com>; Tue, 30 Mar 2021 06:41:52 -0700 (PDT)
Received: from mx02.telecomitalia.it (mx02.telecomitalia.it [217.169.121.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4DEA43A12CA for <explicit-meas@ietf.org>; Tue, 30 Mar 2021 06:41:50 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; d=guest.telecomitalia.it; s=selector1; c=relaxed/relaxed; q=dns/txt; i=@guest.telecomitalia.it; t=1617111707; x=2481025307; h=MIME-Version:From:Date:Message-ID:Subject:To; bh=0Dukrc+Siiw+Yk8W+9pP7Psu5scDqPLQwCsIlJWZE3Q=; b=AyAugdCQtDS1ZWs3FxMc4GzR6cbpRLaUvq90IsJRXJgWvdWSr5pHAY5/y/y8CeXd 53pfxlYDvKqZ2uCZLgHS6eqzNYwuHLlm0IeWsDlojNyAcRFIn7+AYk5/svnQYSJc sOND057nG66fLzBZ227xlpTsKDjmivblPqiXuh9hWjpu/mNbyRl0Y7uCwKbqGFo3 CYjzagryzp6NDySRjfdxuIy3KRmOl9QdsHAE+fSnfBv/ty4tGoG0XlVIUAuvnDw/ Co5/QdL/d/939xhyxzjFFjf9JzBF2YAVd/XuoyFezT7uX9jl5g1b0ZrtxwWe3V1E 08rp4KyavwKCyqiJpOAm8A==;
Received: from mx05.telecomitalia.it ( [10.90.45.21]) (using TLS with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx02.telecomitalia.it () with SMTP id B1.44.08746.B9A23606; Tue, 30 Mar 2021 15:41:47 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; d=guest.telecomitalia.it; s=selector1; c=relaxed/relaxed; q=dns/txt; i=@guest.telecomitalia.it; t=1617111707; x=2481025307; h=MIME-Version:From:Date:Message-ID:Subject:To; bh=0Dukrc+Siiw+Yk8W+9pP7Psu5scDqPLQwCsIlJWZE3Q=; b=AyAugdCQtDS1ZWs3FxMc4GzR6cbpRLaUvq90IsJRXJgWvdWSr5pHAY5/y/y8CeXd 53pfxlYDvKqZ2uCZLgHS6eqzNYwuHLlm0IeWsDlojNyAcRFIn7+AYk5/svnQYSJc sOND057nG66fLzBZ227xlpTsKDjmivblPqiXuh9hWjpu/mNbyRl0Y7uCwKbqGFo3 CYjzagryzp6NDySRjfdxuIy3KRmOl9QdsHAE+fSnfBv/ty4tGoG0XlVIUAuvnDw/ Co5/QdL/d/939xhyxzjFFjf9JzBF2YAVd/XuoyFezT7uX9jl5g1b0ZrtxwWe3V1E 08rp4KyavwKCyqiJpOAm8A==;
X-AuditID: d9a97916-8215c2400000222a-ba-60632a9b2f0a
From: "Bulgarella Fabio (Guest)" <fabio.bulgarella@guest.telecomitalia.it>
To: Ian Swett <ianswett=40google.com@dmarc.ietf.org>
CC: "quic@ietf.org" <quic@ietf.org>, Dmitri Tikhonov <dtikhonov@litespeedtech.com>, "IETF IPPM WG (ippm@ietf.org)" <ippm@ietf.org>, "Lubashev, Igor" <ilubashe@akamai.com>, "alexandre.ferrieux@orange.com" <alexandre.ferrieux@orange.com>, Isabelle Hamchaoui <isabelle.hamchaoui@orange.com>, "explicit-meas@ietf.org" <explicit-meas@ietf.org>, Mirja Kuehlewind <mirja.kuehlewind@ericsson.com>, "emile.stephan@orange.com" <emile.stephan@orange.com>, Marcus Ihlar <marcus.ihlar@ericsson.com>
Thread-Topic: [Explicit-meas] [EXT] Re: Explicit Flow Measurements implementations
Thread-Index: AQHXH9hH/03O2GWJU0qc4UN67vaK+aqRgUYAgAq4Tz0=
Date: Tue, 30 Mar 2021 13:41:46 +0000
Message-ID: <1617111706701.30426@guest.telecomitalia.it>
References: <5e32f9e61db74620b4e08cb12b90311d@TELMBXC15BA020.telecomitalia.local> <CAKcm_gNE657KXC8P7pggYta99hFi4jPg076B4Ckfi1O2B1p4Uw@mail.gmail.com> <b9d62224070a432fb6fb518169db5746@TELMBXC15BA020.telecomitalia.local>, <CAKcm_gMojJwOSeuTpeUr1QiEJ=2y6m6BNZANR0KxeqwLaMnT5A@mail.gmail.com>
In-Reply-To: <CAKcm_gMojJwOSeuTpeUr1QiEJ=2y6m6BNZANR0KxeqwLaMnT5A@mail.gmail.com>
Accept-Language: it-IT, en-US
Content-Language: it-IT
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [93.66.96.106]
Content-Type: multipart/alternative; boundary="_000_161711170670130426guesttelecomitaliait_"
MIME-Version: 1.0
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrCIsWRmVeSWpSXmKPExsXCFaUrqjtbKznBYPp9M4slG84yOzB6LFny kymAMYrLJiU1J7MstUjfLoEr49UV3YK/Txkrnl5oZ2tg/HqfsYuRk0NCwERiedNk5i5GLg4h gW4miR13zrGCOCwCn1gkjq2bBVc1+/xT9i5GDg42AS+J2XeUQcIiQOElv9uYQOqZBb4xS7ye eYIdJCEsECyx6nwPI0RRiMS9yd9YIGwriWcn28HiLAKqEnd2n2AGsXkFLCR61txkh7hiKZPE tF2/2UASnAKBEu9XrgYrYhSQlZiwexFYM7OAuMSL6RDLJAQEJJbsOc8MYYtKvHz8jxXCNpDY unQfC4StILHxbyMTRG+cxJMV09ggFgtKnJz5hAVksYTAEw6J9Ydesk5gFJ+FZMcsJD2zkPTM AgYGs4CmxPpd+hAlihJTuh9ClWtItM6Zy44svoCRfRWjaG6FgZFeSWpOanJ+bmZJYk5mol5m ySZGYETeXFkptoNx9oY3eocYmTgYDzFKcDArifAKH0hMEOJNSaysSi3Kjy8qzUktPsQozcGi JM5bdQ0oJZCeWJKanZpakFoEk2Xi4JRqYKr095RI2nFodu9PvqOv9JctcPk870FccaDY/MlM YuF5U15y3fY8//jehhvL6xb114T8fqrXHmGQFsji4iNo+mG94eaCn/M2H5hZY30l48O0n3OX 7LqdrqHhfDYqV/ZlfWHJsjpXK87ZpsuYjkU7sQVe3P6Cq0NcR4nL58mydx+XXza4vmzL9NOM xw9NfLp8X97XeV27PZ1/20/hPzvhUM692UpCNoZK+6PXhL7cu+Vc573IrT9S2OJPc7sI6K80 fP2vJrPd/cCTs225Kx38W/UOR6/132tQvP/gvd4iRdsFr/qvHP7+u2/r5INJAmnHrxX63Z40 q1HuuInO7OYm7o7bGx+Wd4f9WnymOZPrluxNJZbijERDLeai4kQAAbPJvTcDAAA=
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrGIsWRmVeSWpSXmKPExsXCFaWtpztbKznB4NM2ZYtlP+Qtvkxgszi4 /weLxZINZ5ktHv78ymjR1LCC2aLnwTtmi0sHP7JbHHz5itWic+lrdoueBdwO3B6Tjyxg9jix 7Aqrx6+vV9k8liz5yeTxZNtxZo+WZyfZAtiiuGxSUnMyy1KL9O0SuDJeXdEt+PuUseLphXa2 Bsav9xm7GDk5JARMJGaff8rexcjFISSwilFixtPPbF2MHBxsAl4Ss+8og9SIANUs+d3GBFLD LPCNWeL1zBPsIAlhgWCJVed7GCGKQiTuTf7GAmFbSTw72Q4WZxFQlbiz+wQziM0rYCHRs+Ym 1LKlTBLTdv1mA0lwCgRKvF+5GqyIUUBWYsLuRWDNzALiEi+mQyyTEBCQWLLnPDOELSrx8vE/ VgjbQGLr0n0sELaCxMa/jUwQvXEST1ZMY4NYLChxcuYTlgmMIrOQjJ2FpGwWkrJZQP8zC2hK rN+lD1GiKDGl+yFUuYZE65y57MjiCxjZVzGK5lYYmOqVpOakJufnZpYk5mQm6mWWbGIER7Wu 6A7GbSve6B1iZOJgPMQowcGsJMIrfCAxQYg3JbGyKrUoP76oNCe1+BCjNAeLkjhv1TWglEB6 YklqdmpqQWoRTJaJg1OqgWna0cvPip60L4oNPC0f06C1bXe46YP75ifEbjR8VwjPtXsnIyvg XvOH9exDtc+HZ0fPUcq9tsrDYlVyVnLK1weHTimdm8Ga5ZrXkGx16NyzaQzFF3QZzZb+ddHj 72DdLX3Xcs1M1Yeyvxl0EnROvpKas+VlfUvci5Pda9znPc9ir3hybq9/R6VUMdclD4Wly96a fY+8qep/Zr6w4vMc2S8T73spnlv/7xzP2U0PAsPfdyllNwj2GBtuPt8VK/z5akfU95IHdifv BnBuZwi5GLYrevcrUdNSnsm3ezzSHx+tNeU8m/PH7d4Xzm9L1X9feSfbw8TNWpiZ9ejCPe6t 6gueNso2rOxK3DD9gizb1BNKLMUZiYZazEXFiQAtbxBkWQMAAA==
Archived-At: <https://mailarchive.ietf.org/arch/msg/explicit-meas/eeM6a7wSPjSQfaNham1ORHDigIE>
X-Mailman-Approved-At: Tue, 30 Mar 2021 06:43:46 -0700
Subject: Re: [Explicit-meas] [EXT] Re: Explicit Flow Measurements implementations
X-BeenThere: explicit-meas@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "This mailing list is intended for discussions relating to Explicit Flow Measurements Techniques." <explicit-meas.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/explicit-meas>, <mailto:explicit-meas-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/explicit-meas/>
List-Post: <mailto:explicit-meas@ietf.org>
List-Help: <mailto:explicit-meas-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/explicit-meas>, <mailto:explicit-meas-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 30 Mar 2021 13:41:58 -0000

Hi Ian, All,

here are some implementations of EFM techniques in quic-go if you want to have a look on them.


Spinbit: https://github.com/fabiobulgarella/quic-go/tree/v0.20.0-spinbit

Delaybit: https://github.com/fabiobulgarella/quic-go/tree/v0.20.0-delaybit

Loss Tbit (and old delaybit): https://github.com/fabiobulgarella/quic-go/tree/tim-ds-pl1-spin-free (note that this implementation is not updated to last quic-go tag as the others, as well as containing the implementation of the old version of the delaybit​; will be soon updated).

Loss QR (and spinbit together): https://github.com/fabiobulgarella/quic-go/tree/v0.20.0-spinbit_qr


About the observer, please consider the spindump (https://github.com/EricssonResearch/spindump) which includes logics for all the EFM techniques.


The Android implementation is quite experimental and for now closed source.


Here are some slides with some tests done in the android implementation and on linux using spindump:

https://github.com/fabiobulgarella/tim-materials/blob/main/TIMquic%20%26%20QuicGen%201.2.pdf

​

Best Regards,

Fabio and Massimo

​


________________________________
Da: Explicit-meas <explicit-meas-bounces@ietf.org> per conto di Ian Swett <ianswett=40google.com@dmarc.ietf.org>
Inviato: martedì 23 marzo 2021 14:31
A: Nilo Massimo
Cc: quic@ietf.org; Dmitri Tikhonov; IETF IPPM WG (ippm@ietf.org) Lubashev, Igor; alexandre.ferrieux@orange.com; Isabelle Hamchaoui; explicit-meas@ietf.org; Mirja Kuehlewind; emile.stephan@orange.com; Marcus Ihlar
Oggetto: Re: [Explicit-meas] [EXT] Re: Explicit Flow Measurements implementations

Great, thanks!

On Tue, Mar 23, 2021 at 7:31 AM Nilo Massimo <massimo.nilo=40telecomitalia.it@dmarc.ietf.org<mailto:40telecomitalia.it@dmarc.ietf.org>> wrote:
Hi Ian,
thank you for your reply.

As for our TIM activities, we will collect the material and in the next days we plan to share it via email in the IPPM WG.

Best Regards,
Massimo

____________________

TIM - Telecom Italia
Via G. Reiss Romoli, 274
10148 - Torino (Italy)
____________________

From: Ian Swett <ianswett=40google.com@dmarc.ietf.org<mailto:40google.com@dmarc.ietf.org>>
Sent: lunedì 22 marzo 2021 23:24
To: Nilo Massimo <massimo.nilo@telecomitalia.it<mailto:massimo.nilo@telecomitalia.it>>
Cc: IETF IPPM WG (ippm@ietf.org<mailto:ippm@ietf.org>) <ippm@ietf.org<mailto:ippm@ietf.org>>; quic@ietf.org<mailto:quic@ietf.org>; Dmitri Tikhonov <dtikhonov@litespeedtech.com<mailto:dtikhonov@litespeedtech.com>>; Lubashev, Igor <ilubashe@akamai.com<mailto:ilubashe@akamai.com>>; alexandre.ferrieux@orange.com<mailto:alexandre.ferrieux@orange.com>; Isabelle Hamchaoui <isabelle.hamchaoui@orange.com<mailto:isabelle.hamchaoui@orange.com>>; explicit-meas@ietf.org<mailto:explicit-meas@ietf.org>; Mirja Kuehlewind <mirja.kuehlewind@ericsson.com<mailto:mirja.kuehlewind@ericsson.com>>; emile.stephan@orange.com<mailto:emile.stephan@orange.com>; Marcus Ihlar <marcus.ihlar@ericsson.com<mailto:marcus.ihlar@ericsson.com>>
Subject: [EXT] Re: Explicit Flow Measurements implementations

Thanks for the summary.

Is there a plan to share(via email or presentation) what's been learned from those experiments with the IETF?  If they've been shared in the past and I forgot, please just add relevant links to this thread so they're easy for people to find.

Thanks, Ian

On Thu, Mar 18, 2021 at 7:57 AM Nilo Massimo <massimo.nilo=40telecomitalia.it@dmarc.ietf.org<mailto:40telecomitalia.it@dmarc.ietf.org>> wrote:
Hi All.

After the "Explicit Flow Measurements" draft presentation (https://datatracker.ietf.org/doc/html/draft-mdt-ippm-explicit-flow-measurements-01) in the IPPM WG on March 12th, it was asked by Ian Sweet and Martin Duke if there are any implementations of the EFM methodology on servers and clients.

Here is a list of some implementations made by the authors of the draft.

TIM/Telecom Italia has realized an implementation for delay measurement with:
- a client and a passive observer on mobile user devices, built in Android using Quic-Go (with gomobile binding) and spindump library.
  The observer executes delay measurements, both with Spin bit and Delay bit marking methods.
  The client app exchange QUIC traffic marked for Spin bit/Delay bit versus a server.

- a server built by mean of Quic-Go, generates marked traffic and supports delay measurements with Spin bit and Delay bit methods.

  (Draft https://datatracker.ietf.org/doc/html/draft-cnbf-ippm-user-devices-explicit-monitoring-01)
  This implementation have been tested on TIM production network, both with mobile and fixed access.

- Other implementations for loss algorithms (QR and T) have been tested in our lab with good results and we are planning to move them in production network for some services under our control.

Akamai and Orange have built a monitoring system with QL marking, specifically for loss measurements.
Their implementation has already been used in CDN servers to make EFM.
Litespeed Tech also implemented QL marking in their lsquic library.

Moreover, Ericsson is carrying out several monitoring initiatives using the SpinBit for delay EFM.

Best Regards,
Massimo and Fabio
 ____________________
TIM - Telecom Italia
Via G. Reiss Romoli, 274
10148 - Torino (Italy)
____________________


TIM - Uso Interno - Tutti i diritti riservati.

________________________________
[https://cdn.tim.ticdn.it/content/dam/flytoco-areapubblica-aemfe/tim_it/dem-assets/email_footer/Banner_mail_questa_e_tim_3012.jpg]<https://on.tim.it/banner-mail-dip>
Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione, Grazie.

This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorised. If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail, Thanks.

Rispetta l'ambiente. Non stampare questa mail se non è necessario.