Re: [ippm] [Explicit-meas] [EXT] Re: Explicit Flow Measurements implementations

"Bulgarella Fabio (Guest)" <fabio.bulgarella@guest.telecomitalia.it> Tue, 30 March 2021 13:41 UTC

Return-Path: <fabio.bulgarella@guest.telecomitalia.it>
X-Original-To: ippm@ietfa.amsl.com
Delivered-To: ippm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2801C3A12BC for <ippm@ietfa.amsl.com>; Tue, 30 Mar 2021 06:41:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.989
X-Spam-Level:
X-Spam-Status: No, score=-1.989 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=guest.telecomitalia.it header.b=AyAugdCQ; dkim=pass (2048-bit key) header.d=guest.telecomitalia.it header.b=AyAugdCQ
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Y9MghgRhU9pf for <ippm@ietfa.amsl.com>; Tue, 30 Mar 2021 06:41:54 -0700 (PDT)
Received: from mx02.telecomitalia.it (mx02.telecomitalia.it [217.169.121.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 56C003A12C6 for <ippm@ietf.org>; Tue, 30 Mar 2021 06:41:53 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; d=guest.telecomitalia.it; s=selector1; c=relaxed/relaxed; q=dns/txt; i=@guest.telecomitalia.it; t=1617111707; x=2481025307; h=MIME-Version:From:Date:Message-ID:Subject:To; bh=0Dukrc+Siiw+Yk8W+9pP7Psu5scDqPLQwCsIlJWZE3Q=; b=AyAugdCQtDS1ZWs3FxMc4GzR6cbpRLaUvq90IsJRXJgWvdWSr5pHAY5/y/y8CeXd 53pfxlYDvKqZ2uCZLgHS6eqzNYwuHLlm0IeWsDlojNyAcRFIn7+AYk5/svnQYSJc sOND057nG66fLzBZ227xlpTsKDjmivblPqiXuh9hWjpu/mNbyRl0Y7uCwKbqGFo3 CYjzagryzp6NDySRjfdxuIy3KRmOl9QdsHAE+fSnfBv/ty4tGoG0XlVIUAuvnDw/ Co5/QdL/d/939xhyxzjFFjf9JzBF2YAVd/XuoyFezT7uX9jl5g1b0ZrtxwWe3V1E 08rp4KyavwKCyqiJpOAm8A==;
Received: from mx05.telecomitalia.it ( [10.90.45.21]) (using TLS with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx02.telecomitalia.it () with SMTP id C1.44.08746.B9A23606; Tue, 30 Mar 2021 15:41:47 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; d=guest.telecomitalia.it; s=selector1; c=relaxed/relaxed; q=dns/txt; i=@guest.telecomitalia.it; t=1617111707; x=2481025307; h=MIME-Version:From:Date:Message-ID:Subject:To; bh=0Dukrc+Siiw+Yk8W+9pP7Psu5scDqPLQwCsIlJWZE3Q=; b=AyAugdCQtDS1ZWs3FxMc4GzR6cbpRLaUvq90IsJRXJgWvdWSr5pHAY5/y/y8CeXd 53pfxlYDvKqZ2uCZLgHS6eqzNYwuHLlm0IeWsDlojNyAcRFIn7+AYk5/svnQYSJc sOND057nG66fLzBZ227xlpTsKDjmivblPqiXuh9hWjpu/mNbyRl0Y7uCwKbqGFo3 CYjzagryzp6NDySRjfdxuIy3KRmOl9QdsHAE+fSnfBv/ty4tGoG0XlVIUAuvnDw/ Co5/QdL/d/939xhyxzjFFjf9JzBF2YAVd/XuoyFezT7uX9jl5g1b0ZrtxwWe3V1E 08rp4KyavwKCyqiJpOAm8A==;
X-AuditID: d9a97916-156aa7000000222a-bb-60632a9bf93a
From: "Bulgarella Fabio (Guest)" <fabio.bulgarella@guest.telecomitalia.it>
To: Ian Swett <ianswett=40google.com@dmarc.ietf.org>
CC: "quic@ietf.org" <quic@ietf.org>, Dmitri Tikhonov <dtikhonov@litespeedtech.com>, "IETF IPPM WG (ippm@ietf.org)" <ippm@ietf.org>, "Lubashev, Igor" <ilubashe@akamai.com>, "alexandre.ferrieux@orange.com" <alexandre.ferrieux@orange.com>, "Isabelle Hamchaoui" <isabelle.hamchaoui@orange.com>, "explicit-meas@ietf.org" <explicit-meas@ietf.org>, Mirja Kuehlewind <mirja.kuehlewind@ericsson.com>, "emile.stephan@orange.com" <emile.stephan@orange.com>, Marcus Ihlar <marcus.ihlar@ericsson.com>
Thread-Topic: [Explicit-meas] [EXT] Re: Explicit Flow Measurements implementations
Thread-Index: AQHXH9hH/03O2GWJU0qc4UN67vaK+aqRgUYAgAq4Tz0=
Date: Tue, 30 Mar 2021 13:41:46 +0000
Message-ID: <1617111706701.30426@guest.telecomitalia.it>
References: <5e32f9e61db74620b4e08cb12b90311d@TELMBXC15BA020.telecomitalia.local> <CAKcm_gNE657KXC8P7pggYta99hFi4jPg076B4Ckfi1O2B1p4Uw@mail.gmail.com> <b9d62224070a432fb6fb518169db5746@TELMBXC15BA020.telecomitalia.local>, <CAKcm_gMojJwOSeuTpeUr1QiEJ=2y6m6BNZANR0KxeqwLaMnT5A@mail.gmail.com>
In-Reply-To: <CAKcm_gMojJwOSeuTpeUr1QiEJ=2y6m6BNZANR0KxeqwLaMnT5A@mail.gmail.com>
Accept-Language: it-IT, en-US
Content-Language: it-IT
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [93.66.96.106]
Content-Type: multipart/alternative; boundary="_000_161711170670130426guesttelecomitaliait_"
MIME-Version: 1.0
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrCIsWRmVeSWpSXmKPExsXCFaUrqjtbKznB4HqvuUXPg3fMDoweS5b8 ZApgjOKySUnNySxLLdK3S+DKeHVFt+DvU8aKpxfa2RoYv95n7GLk5JAQMJHoOTOFuYuRi0NI oJtJYs/+z0wgDovAJxaJY+tmwVXNPv+UvYuRg4NNwEti9h1lkLAIUHjJ7zawemaBb8wSr2ee YAdJCAsES6w638MIURQicW/yNxYI20ri2cl2sDiLgKrEnd0nmEFsXgELiZ41N9khrljKJDFt 1282kASnQKDE+5WrwYoYBWQlJuxeBNbMLCAu8WI6xDIJAQGJJXvOM0PYohIvH/9jhbANJLYu 3ccCYStIbPzbyATRGyfxZMU0NojFghInZz5hAVksIfCEQ2L9oZesExjFZyHZMQtJzywkPbOA gcEsoCmxfpc+RImixJTuh1DlGhKtc+ayI4svYGRfxSiaW2FgpFeSmpOanJ+bWZKYk5mol1my iREYkTdXVortYJy94Y3eIUYmDsZDjBIczEoivMIHEhOEeFMSK6tSi/Lji0pzUosPMUpzsCiJ 81ZdA0oJpCeWpGanphakFsFkmTg4pRqYDkwVe9V34ZNS5bY79Z9ELHb3thzv9xeUeLsw9ef5 lxKT50rLe8hn1k5wuVo3bfKG2glL202eHY/nak9YqdmoyfOy8dYzT/M5n9mst75jKlkqcM1L 4pzA/8Wfdp9kayw+t03NxvNkwWeZKPE1DZvUP0+8sztI9k3dloTmMvPeO9u+Tpo6bd3zvfPV n2rO37Zr9rzs7CuFfGt0svSczKKNk+54/zp38ndWvFPmLb2tu20X3rRtPZfisYNh/6vH166u f545XVCtp6Jp/6czRaxNJ8TOxtRYpfXWXT8kaPT/y6Vd2/nj0zqT5c6v8dy2v/Duu1+rN509 9dUjrd2PYa1T2q/s3OQuzi9HdvW9duVv267EUpyRaKjFXFScCAB60r5gNwMAAA==
X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrGIsWRmVeSWpSXmKPExsXCFaWtpztbKznB4NM2ZYtlP+Qtvkxgszi4 /weLxZINZ5ktHv78ymjR1LCC2aLnwTtmi0sHP7JbHHz5itWic+lrdoueBdwO3B6Tjyxg9jix 7Aqrx6+vV9k8liz5yeTxZNtxZo+WZyfZAtiiuGxSUnMyy1KL9O0SuDJeXdEt+PuUseLphXa2 Bsav9xm7GDk5JARMJGaff8rexcjFISSwilFixtPPbF2MHBxsAl4Ss+8og9SIANUs+d3GBFLD LPCNWeL1zBPsIAlhgWCJVed7GCGKQiTuTf7GAmFbSTw72Q4WZxFQlbiz+wQziM0rYCHRs+Ym 1LKlTBLTdv1mA0lwCgRKvF+5GqyIUUBWYsLuRWDNzALiEi+mQyyTEBCQWLLnPDOELSrx8vE/ VgjbQGLr0n0sELaCxMa/jUwQvXEST1ZMY4NYLChxcuYTlgmMIrOQjJ2FpGwWkrJZQP8zC2hK rN+lD1GiKDGl+yFUuYZE65y57MjiCxjZVzGK5lYYmOqVpOakJufnZpYk5mQm6mWWbGIER7Wu 6A7GbSve6B1iZOJgPMQowcGsJMIrfCAxQYg3JbGyKrUoP76oNCe1+BCjNAeLkjhv1TWglEB6 YklqdmpqQWoRTJaJg1OqgWna0cvPip60L4oNPC0f06C1bXe46YP75ifEbjR8VwjPtXsnIyvg XvOH9exDtc+HZ0fPUcq9tsrDYlVyVnLK1weHTimdm8Ga5ZrXkGx16NyzaQzFF3QZzZb+ddHj 72DdLX3Xcs1M1Yeyvxl0EnROvpKas+VlfUvci5Pda9znPc9ir3hybq9/R6VUMdclD4Wly96a fY+8qep/Zr6w4vMc2S8T73spnlv/7xzP2U0PAsPfdyllNwj2GBtuPt8VK/z5akfU95IHdifv BnBuZwi5GLYrevcrUdNSnsm3ezzSHx+tNeU8m/PH7d4Xzm9L1X9feSfbw8TNWpiZ9ejCPe6t 6gueNso2rOxK3DD9gizb1BNKLMUZiYZazEXFiQAtbxBkWQMAAA==
Archived-At: <https://mailarchive.ietf.org/arch/msg/ippm/_Jv0dOLSE_AvX1UkSgVifaXChq0>
X-Mailman-Approved-At: Mon, 05 Apr 2021 10:42:33 -0700
Subject: Re: [ippm] [Explicit-meas] [EXT] Re: Explicit Flow Measurements implementations
X-BeenThere: ippm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF IP Performance Metrics Working Group <ippm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ippm>, <mailto:ippm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ippm/>
List-Post: <mailto:ippm@ietf.org>
List-Help: <mailto:ippm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ippm>, <mailto:ippm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 30 Mar 2021 13:41:59 -0000

Hi Ian, All,

here are some implementations of EFM techniques in quic-go if you want to have a look on them.


Spinbit: https://github.com/fabiobulgarella/quic-go/tree/v0.20.0-spinbit

Delaybit: https://github.com/fabiobulgarella/quic-go/tree/v0.20.0-delaybit

Loss Tbit (and old delaybit): https://github.com/fabiobulgarella/quic-go/tree/tim-ds-pl1-spin-free (note that this implementation is not updated to last quic-go tag as the others, as well as containing the implementation of the old version of the delaybit​; will be soon updated).

Loss QR (and spinbit together): https://github.com/fabiobulgarella/quic-go/tree/v0.20.0-spinbit_qr


About the observer, please consider the spindump (https://github.com/EricssonResearch/spindump) which includes logics for all the EFM techniques.


The Android implementation is quite experimental and for now closed source.


Here are some slides with some tests done in the android implementation and on linux using spindump:

https://github.com/fabiobulgarella/tim-materials/blob/main/TIMquic%20%26%20QuicGen%201.2.pdf

​

Best Regards,

Fabio and Massimo

​


________________________________
Da: Explicit-meas <explicit-meas-bounces@ietf.org> per conto di Ian Swett <ianswett=40google.com@dmarc.ietf.org>
Inviato: martedì 23 marzo 2021 14:31
A: Nilo Massimo
Cc: quic@ietf.org; Dmitri Tikhonov; IETF IPPM WG (ippm@ietf.org); Lubashev, Igor; alexandre.ferrieux@orange.com; Isabelle Hamchaoui; explicit-meas@ietf.org; Mirja Kuehlewind; emile.stephan@orange.com; Marcus Ihlar
Oggetto: Re: [Explicit-meas] [EXT] Re: Explicit Flow Measurements implementations

Great, thanks!

On Tue, Mar 23, 2021 at 7:31 AM Nilo Massimo <massimo.nilo=40telecomitalia.it@dmarc.ietf.org<mailto:40telecomitalia.it@dmarc.ietf.org>> wrote:
Hi Ian,
thank you for your reply.

As for our TIM activities, we will collect the material and in the next days we plan to share it via email in the IPPM WG.

Best Regards,
Massimo

____________________

TIM - Telecom Italia
Via G. Reiss Romoli, 274
10148 - Torino (Italy)
____________________

From: Ian Swett <ianswett=40google.com@dmarc.ietf.org<mailto:40google.com@dmarc.ietf.org>>
Sent: lunedì 22 marzo 2021 23:24
To: Nilo Massimo <massimo.nilo@telecomitalia.it<mailto:massimo.nilo@telecomitalia.it>>
Cc: IETF IPPM WG (ippm@ietf.org<mailto:ippm@ietf.org>) <ippm@ietf.org<mailto:ippm@ietf.org>>; quic@ietf.org<mailto:quic@ietf.org>; Dmitri Tikhonov <dtikhonov@litespeedtech.com<mailto:dtikhonov@litespeedtech.com>>; Lubashev, Igor <ilubashe@akamai.com<mailto:ilubashe@akamai.com>>; alexandre.ferrieux@orange.com<mailto:alexandre.ferrieux@orange.com>; Isabelle Hamchaoui <isabelle.hamchaoui@orange.com<mailto:isabelle.hamchaoui@orange.com>>; explicit-meas@ietf.org<mailto:explicit-meas@ietf.org>; Mirja Kuehlewind <mirja.kuehlewind@ericsson.com<mailto:mirja.kuehlewind@ericsson.com>>; emile.stephan@orange.com<mailto:emile.stephan@orange.com>; Marcus Ihlar <marcus.ihlar@ericsson.com<mailto:marcus.ihlar@ericsson.com>>
Subject: [EXT] Re: Explicit Flow Measurements implementations

Thanks for the summary.

Is there a plan to share(via email or presentation) what's been learned from those experiments with the IETF?  If they've been shared in the past and I forgot, please just add relevant links to this thread so they're easy for people to find.

Thanks, Ian

On Thu, Mar 18, 2021 at 7:57 AM Nilo Massimo <massimo.nilo=40telecomitalia.it@dmarc.ietf.org<mailto:40telecomitalia.it@dmarc.ietf.org>> wrote:
Hi All.

After the "Explicit Flow Measurements" draft presentation (https://datatracker.ietf.org/doc/html/draft-mdt-ippm-explicit-flow-measurements-01) in the IPPM WG on March 12th, it was asked by Ian Sweet and Martin Duke if there are any implementations of the EFM methodology on servers and clients.

Here is a list of some implementations made by the authors of the draft.

TIM/Telecom Italia has realized an implementation for delay measurement with:
- a client and a passive observer on mobile user devices, built in Android using Quic-Go (with gomobile binding) and spindump library.
  The observer executes delay measurements, both with Spin bit and Delay bit marking methods.
  The client app exchange QUIC traffic marked for Spin bit/Delay bit versus a server.

- a server built by mean of Quic-Go, generates marked traffic and supports delay measurements with Spin bit and Delay bit methods.

  (Draft https://datatracker.ietf.org/doc/html/draft-cnbf-ippm-user-devices-explicit-monitoring-01)
  This implementation have been tested on TIM production network, both with mobile and fixed access.

- Other implementations for loss algorithms (QR and T) have been tested in our lab with good results and we are planning to move them in production network for some services under our control.

Akamai and Orange have built a monitoring system with QL marking, specifically for loss measurements.
Their implementation has already been used in CDN servers to make EFM.
Litespeed Tech also implemented QL marking in their lsquic library.

Moreover, Ericsson is carrying out several monitoring initiatives using the SpinBit for delay EFM.

Best Regards,
Massimo and Fabio
 ____________________
TIM - Telecom Italia
Via G. Reiss Romoli, 274
10148 - Torino (Italy)
____________________


TIM - Uso Interno - Tutti i diritti riservati.

________________________________
[https://cdn.tim.ticdn.it/content/dam/flytoco-areapubblica-aemfe/tim_it/dem-assets/email_footer/Banner_mail_questa_e_tim_3012.jpg]<https://on.tim.it/banner-mail-dip>
Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione, Grazie.

This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorised. If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail, Thanks.

Rispetta l'ambiente. Non stampare questa mail se non è necessario.