Re: [Explicit-meas] [EXT] Re: Explicit Flow Measurements implementations

"Bulgarella Fabio (Guest)" <fabio.bulgarella@guest.telecomitalia.it> Tue, 30 March 2021 13:42 UTC

Return-Path: <fabio.bulgarella@guest.telecomitalia.it>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3DA7F3A12BC for <quic@ietfa.amsl.com>; Tue, 30 Mar 2021 06:42:00 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.989
X-Spam-Level:
X-Spam-Status: No, score=-1.989 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=guest.telecomitalia.it header.b=OZIx9khe; dkim=pass (2048-bit key) header.d=guest.telecomitalia.it header.b=AyAugdCQ
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HEya306_BRn9 for <quic@ietfa.amsl.com>; Tue, 30 Mar 2021 06:41:55 -0700 (PDT)
Received: from mx02.telecomitalia.it (mx02.telecomitalia.it [217.169.121.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 618963A12CE for <quic@ietf.org>; Tue, 30 Mar 2021 06:41:53 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; d=guest.telecomitalia.it; s=selector1; c=relaxed/relaxed; q=dns/txt; i=@guest.telecomitalia.it; t=1617111708; x=2481025308; h=MIME-Version:From:Date:Message-ID:Subject:To; bh=0Dukrc+Siiw+Yk8W+9pP7Psu5scDqPLQwCsIlJWZE3Q=; b=OZIx9kheUHLrbkfvdSDsMroju5rKkzDqsbqDuzqYMankhs4C2Cw6F7dKLqfNGL7N x7jbQwUIXEuQjAfewkhzLQsNFk9wqpX2oQ2QVIpEir3030Wr0+ku8B/YsC8v6cTI wVE0ZpoL00z0o9UJBC47lx4nuuOIbEm/A8IB+eLDRAu4wAc6IpNr/T76gvSxoViM bCVF0srTardpDrLGFClmHjOtHSJH7YCFlIqKA6QqL8WqRz9OY8Tmaq/4OcaYTgDo endkuMbJvvOHk2hQyxzZls8Chl6yHiFN/3zkIp/cXLkyPRdNDZmrltGoBq3jqYhP 55+2gPkiEff6mtnVixDkdA==;
Received: from mx05.telecomitalia.it ( [10.90.45.21]) (using TLS with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx02.telecomitalia.it () with SMTP id D1.44.08746.B9A23606; Tue, 30 Mar 2021 15:41:47 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; d=guest.telecomitalia.it; s=selector1; c=relaxed/relaxed; q=dns/txt; i=@guest.telecomitalia.it; t=1617111707; x=2481025307; h=MIME-Version:From:Date:Message-ID:Subject:To; bh=0Dukrc+Siiw+Yk8W+9pP7Psu5scDqPLQwCsIlJWZE3Q=; b=AyAugdCQtDS1ZWs3FxMc4GzR6cbpRLaUvq90IsJRXJgWvdWSr5pHAY5/y/y8CeXd 53pfxlYDvKqZ2uCZLgHS6eqzNYwuHLlm0IeWsDlojNyAcRFIn7+AYk5/svnQYSJc sOND057nG66fLzBZ227xlpTsKDjmivblPqiXuh9hWjpu/mNbyRl0Y7uCwKbqGFo3 CYjzagryzp6NDySRjfdxuIy3KRmOl9QdsHAE+fSnfBv/ty4tGoG0XlVIUAuvnDw/ Co5/QdL/d/939xhyxzjFFjf9JzBF2YAVd/XuoyFezT7uX9jl5g1b0ZrtxwWe3V1E 08rp4KyavwKCyqiJpOAm8A==;
X-AuditID: d9a97916-17aab7000000222a-bc-60632a9b7039
From: "Bulgarella Fabio (Guest)" <fabio.bulgarella@guest.telecomitalia.it>
To: Ian Swett <ianswett=40google.com@dmarc.ietf.org>
CC: "quic@ietf.org" <quic@ietf.org>, Dmitri Tikhonov <dtikhonov@litespeedtech.com>, "IETF IPPM WG (ippm@ietf.org)" <ippm@ietf.org>, "Lubashev, Igor" <ilubashe@akamai.com>, "alexandre.ferrieux@orange.com" <alexandre.ferrieux@orange.com>, Isabelle Hamchaoui <isabelle.hamchaoui@orange.com>, "explicit-meas@ietf.org" <explicit-meas@ietf.org>, Mirja Kuehlewind <mirja.kuehlewind@ericsson.com>, "emile.stephan@orange.com" <emile.stephan@orange.com>, Marcus Ihlar <marcus.ihlar@ericsson.com>
Subject: Re: [Explicit-meas] [EXT] Re: Explicit Flow Measurements implementations
Thread-Topic: [Explicit-meas] [EXT] Re: Explicit Flow Measurements implementations
Thread-Index: AQHXH9hH/03O2GWJU0qc4UN67vaK+aqRgUYAgAq4Tz0=
Date: Tue, 30 Mar 2021 13:41:46 +0000
Message-ID: <1617111706701.30426@guest.telecomitalia.it>
References: <5e32f9e61db74620b4e08cb12b90311d@TELMBXC15BA020.telecomitalia.local> <CAKcm_gNE657KXC8P7pggYta99hFi4jPg076B4Ckfi1O2B1p4Uw@mail.gmail.com> <b9d62224070a432fb6fb518169db5746@TELMBXC15BA020.telecomitalia.local>, <CAKcm_gMojJwOSeuTpeUr1QiEJ=2y6m6BNZANR0KxeqwLaMnT5A@mail.gmail.com>
In-Reply-To: <CAKcm_gMojJwOSeuTpeUr1QiEJ=2y6m6BNZANR0KxeqwLaMnT5A@mail.gmail.com>
Accept-Language: it-IT, en-US
Content-Language: it-IT
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [93.66.96.106]
Content-Type: multipart/alternative; boundary="_000_161711170670130426guesttelecomitaliait_"
MIME-Version: 1.0
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrMIsWRmVeSWpSXmKPExsXCFaUrqjtbKznBoH+RuUXPAm4HRo8lS34y BTBGcdmkpOZklqUW6dslcGW8uqJb8PcpY8XTC+1sDYxf7zN2MXJySAiYSGx7uJu5i5GLQ0ig m0nia/cjRhCHReATi8SxdbPgqmaff8rexcjBwSbgJTH7jjJIWAQovOR3GxNIPbPAN2aJ1zNP sIMkhAWCJVad72GEKAqRuDf5GwuEbSXx7GQ7WJxFQFXizu4TzCA2r4CFRM+am+wQVyxlkpi2 6zcbSIJTIFDi/crVYEWMArISE3YvAmtmFhCXeDEdYpmEgIDEkj3nmSFsUYmXj/+xQtgGEluX 7mOBsBUkNv5tZILojZN4smIaG8RiQYmTM5+wgCyWEHjCIbH+0EvWCYzis5DsmIWkZxaSnlnA wGAW0JRYv0sfokRRYkr3Q6hyDYnWOXPZkcUXMLKvYhTNrTAw0itJzUlNzs/NLEnMyUzUyyzZ xAiMx5srK8V2MM7e8EbvECMTB+MhRgkOZiURXuEDiQlCvCmJlVWpRfnxRaU5qcWHGKU5WJTE eauuAaUE0hNLUrNTUwtSi2CyTBycUg1MFdf+K+w0/mlfFTfdquxFxAIHtr29c5KfRq8sFDic nL/y4Yt9u45w+Yp6zcze/7R+b/RM12ufpnm+/jyXmUP02WkH1jXRinMZ78nUfD550OrQg/Xc 2e5cnK8+3U/jEn7BYbtV62P78wVeHQpi85Xij2Qd7/QqFH04T0X2IsOS6rSuZVlvuWVP/17H Un1i3rwTy2JOF7WxuTU9X5ZmIPV8HuO/q/t1rqaKPwnUWz2P42z0T5ZPwu/Wfbg52fJ3WGd4 Z6fD45Vza2bYLPut889VT/jsVaOlV/sve+8I9TEzTJll8WH3ReOQU76KBXcPLhUqNlSdpLDu btKzn9uLGLgvrXvq/9hpbtaBjdu2GaltUVBiKc5INNRiLipOBABdKqawNgMAAA==
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrGIsWRmVeSWpSXmKPExsXCFaWtpztbKznB4NM2ZYtlP+Qtvkxgszi4 /weLxZINZ5ktHv78ymjR1LCC2aLnwTtmi0sHP7JbHHz5itWic+lrdoueBdwO3B6Tjyxg9jix 7Aqrx6+vV9k8liz5yeTxZNtxZo+WZyfZAtiiuGxSUnMyy1KL9O0SuDJeXdEt+PuUseLphXa2 Bsav9xm7GDk5JARMJGaff8rexcjFISSwilFixtPPbF2MHBxsAl4Ss+8og9SIANUs+d3GBFLD LPCNWeL1zBPsIAlhgWCJVed7GCGKQiTuTf7GAmFbSTw72Q4WZxFQlbiz+wQziM0rYCHRs+Ym 1LKlTBLTdv1mA0lwCgRKvF+5GqyIUUBWYsLuRWDNzALiEi+mQyyTEBCQWLLnPDOELSrx8vE/ VgjbQGLr0n0sELaCxMa/jUwQvXEST1ZMY4NYLChxcuYTlgmMIrOQjJ2FpGwWkrJZQP8zC2hK rN+lD1GiKDGl+yFUuYZE65y57MjiCxjZVzGK5lYYmOqVpOakJufnZpYk5mQm6mWWbGIER7Wu 6A7GbSve6B1iZOJgPMQowcGsJMIrfCAxQYg3JbGyKrUoP76oNCe1+BCjNAeLkjhv1TWglEB6 YklqdmpqQWoRTJaJg1OqgWna0cvPip60L4oNPC0f06C1bXe46YP75ifEbjR8VwjPtXsnIyvg XvOH9exDtc+HZ0fPUcq9tsrDYlVyVnLK1weHTimdm8Ga5ZrXkGx16NyzaQzFF3QZzZb+ddHj 72DdLX3Xcs1M1Yeyvxl0EnROvpKas+VlfUvci5Pda9znPc9ir3hybq9/R6VUMdclD4Wly96a fY+8qep/Zr6w4vMc2S8T73spnlv/7xzP2U0PAsPfdyllNwj2GBtuPt8VK/z5akfU95IHdifv BnBuZwi5GLYrevcrUdNSnsm3ezzSHx+tNeU8m/PH7d4Xzm9L1X9feSfbw8TNWpiZ9ejCPe6t 6gueNso2rOxK3DD9gizb1BNKLMUZiYZazEXFiQAtbxBkWQMAAA==
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/C-bLkVIk1pIx7plUBz04yxh1fNg>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 30 Mar 2021 13:42:00 -0000

Hi Ian, All,

here are some implementations of EFM techniques in quic-go if you want to have a look on them.


Spinbit: https://github.com/fabiobulgarella/quic-go/tree/v0.20.0-spinbit

Delaybit: https://github.com/fabiobulgarella/quic-go/tree/v0.20.0-delaybit

Loss Tbit (and old delaybit): https://github.com/fabiobulgarella/quic-go/tree/tim-ds-pl1-spin-free (note that this implementation is not updated to last quic-go tag as the others, as well as containing the implementation of the old version of the delaybit​; will be soon updated).

Loss QR (and spinbit together): https://github.com/fabiobulgarella/quic-go/tree/v0.20.0-spinbit_qr


About the observer, please consider the spindump (https://github.com/EricssonResearch/spindump) which includes logics for all the EFM techniques.


The Android implementation is quite experimental and for now closed source.


Here are some slides with some tests done in the android implementation and on linux using spindump:

https://github.com/fabiobulgarella/tim-materials/blob/main/TIMquic%20%26%20QuicGen%201.2.pdf

​

Best Regards,

Fabio and Massimo

​


________________________________
Da: Explicit-meas <explicit-meas-bounces@ietf.org> per conto di Ian Swett <ianswett=40google.com@dmarc.ietf.org>
Inviato: martedì 23 marzo 2021 14:31
A: Nilo Massimo
Cc: quic@ietf.org; Dmitri Tikhonov; IETF IPPM WG (ippm@ietf.org); Lubashev, Igor; alexandre.ferrieux@orange.com; Isabelle Hamchaoui; explicit-meas@ietf.org; Mirja Kuehlewind; emile.stephan@orange.com; Marcus Ihlar
Oggetto: Re: [Explicit-meas] [EXT] Re: Explicit Flow Measurements implementations

Great, thanks!

On Tue, Mar 23, 2021 at 7:31 AM Nilo Massimo <massimo.nilo=40telecomitalia.it@dmarc.ietf.org<mailto:40telecomitalia.it@dmarc.ietf.org>> wrote:
Hi Ian,
thank you for your reply.

As for our TIM activities, we will collect the material and in the next days we plan to share it via email in the IPPM WG.

Best Regards,
Massimo

____________________

TIM - Telecom Italia
Via G. Reiss Romoli, 274
10148 - Torino (Italy)
____________________

From: Ian Swett <ianswett=40google.com@dmarc.ietf.org<mailto:40google.com@dmarc.ietf.org>>
Sent: lunedì 22 marzo 2021 23:24
To: Nilo Massimo <massimo.nilo@telecomitalia.it<mailto:massimo.nilo@telecomitalia.it>>
Cc: IETF IPPM WG (ippm@ietf.org<mailto:ippm@ietf.org>) <ippm@ietf.org<mailto:ippm@ietf.org>>; quic@ietf.org<mailto:quic@ietf.org>; Dmitri Tikhonov <dtikhonov@litespeedtech.com<mailto:dtikhonov@litespeedtech.com>>; Lubashev, Igor <ilubashe@akamai.com<mailto:ilubashe@akamai.com>>; alexandre.ferrieux@orange.com<mailto:alexandre.ferrieux@orange.com>; Isabelle Hamchaoui <isabelle.hamchaoui@orange.com<mailto:isabelle.hamchaoui@orange.com>>; explicit-meas@ietf.org<mailto:explicit-meas@ietf.org>; Mirja Kuehlewind <mirja.kuehlewind@ericsson.com<mailto:mirja.kuehlewind@ericsson.com>>; emile.stephan@orange.com<mailto:emile.stephan@orange.com>; Marcus Ihlar <marcus.ihlar@ericsson.com<mailto:marcus.ihlar@ericsson.com>>
Subject: [EXT] Re: Explicit Flow Measurements implementations

Thanks for the summary.

Is there a plan to share(via email or presentation) what's been learned from those experiments with the IETF?  If they've been shared in the past and I forgot, please just add relevant links to this thread so they're easy for people to find.

Thanks, Ian

On Thu, Mar 18, 2021 at 7:57 AM Nilo Massimo <massimo.nilo=40telecomitalia.it@dmarc.ietf.org<mailto:40telecomitalia.it@dmarc.ietf.org>> wrote:
Hi All.

After the "Explicit Flow Measurements" draft presentation (https://datatracker.ietf.org/doc/html/draft-mdt-ippm-explicit-flow-measurements-01) in the IPPM WG on March 12th, it was asked by Ian Sweet and Martin Duke if there are any implementations of the EFM methodology on servers and clients.

Here is a list of some implementations made by the authors of the draft.

TIM/Telecom Italia has realized an implementation for delay measurement with:
- a client and a passive observer on mobile user devices, built in Android using Quic-Go (with gomobile binding) and spindump library.
  The observer executes delay measurements, both with Spin bit and Delay bit marking methods.
  The client app exchange QUIC traffic marked for Spin bit/Delay bit versus a server.

- a server built by mean of Quic-Go, generates marked traffic and supports delay measurements with Spin bit and Delay bit methods.

  (Draft https://datatracker.ietf.org/doc/html/draft-cnbf-ippm-user-devices-explicit-monitoring-01)
  This implementation have been tested on TIM production network, both with mobile and fixed access.

- Other implementations for loss algorithms (QR and T) have been tested in our lab with good results and we are planning to move them in production network for some services under our control.

Akamai and Orange have built a monitoring system with QL marking, specifically for loss measurements.
Their implementation has already been used in CDN servers to make EFM.
Litespeed Tech also implemented QL marking in their lsquic library.

Moreover, Ericsson is carrying out several monitoring initiatives using the SpinBit for delay EFM.

Best Regards,
Massimo and Fabio
 ____________________
TIM - Telecom Italia
Via G. Reiss Romoli, 274
10148 - Torino (Italy)
____________________


TIM - Uso Interno - Tutti i diritti riservati.

________________________________
[https://cdn.tim.ticdn.it/content/dam/flytoco-areapubblica-aemfe/tim_it/dem-assets/email_footer/Banner_mail_questa_e_tim_3012.jpg]<https://on.tim.it/banner-mail-dip>
Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione, Grazie.

This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorised. If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail, Thanks.

Rispetta l'ambiente. Non stampare questa mail se non è necessario.