Re: [tcpm] [EXTERNAL] Re: Hystart and delay jitter

Ingemar Johansson S <ingemar.s.johansson@ericsson.com> Fri, 19 March 2021 09:16 UTC

Return-Path: <ingemar.s.johansson@ericsson.com>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5B8C03A07A4 for <tcpm@ietfa.amsl.com>; Fri, 19 Mar 2021 02:16:20 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.349
X-Spam-Level:
X-Spam-Status: No, score=-2.349 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.248, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JHID2x1Lqs0A for <tcpm@ietfa.amsl.com>; Fri, 19 Mar 2021 02:16:18 -0700 (PDT)
Received: from EUR05-VI1-obe.outbound.protection.outlook.com (mail-vi1eur05on2056.outbound.protection.outlook.com [40.107.21.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 12CC93A0743 for <tcpm@ietf.org>; Fri, 19 Mar 2021 02:16:17 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=QvyE7wVJFHxVNUbe2fBoiHxDKB4/oYy90iNTRzilerbLfHRPIrRa8AFlqOjqfW71aRCHj2eyeP190K3R717gb+MWPpQYhVdO0T8l3lIkyGzByz9wRpLbXUQV3LTuVm+xBjff/TQGcTVFLPAYSMadNfMBDdcirs2eIMlRwjObljnuVo+pDp0zXexExX9KjQmnYrR1yni50kmyTxQvoJOC8pz18enZOkcBzK1muK41MQfZA19tnOQYEeDP9V56e8kBRw0Ose/LyAnzqoQESVHN8nAfvSV7YGJtuMavOJHW7HeDYLAX8HMYCdG9RrR2KjE9WuD/xaAslNhfSwmRQMEEUw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZTuUecmUbayfpG4RnOkorFODlBki/zYUM4ZazyLUhe8=; b=AWk2GP+qIeI0tsJKxiiJVoCiNx9MVrh4+EM5gLkN9J+n/6TJj6WUmb+d4jHCbMNWwP/15a9rC1sjV8cKL7cejD7fgFLP6LsTjGjngQHVAEyekCysy7YqwWZId9DRu7rXO0zJb8h88EYNXx9WHR59keqhfD5sqjdcYKCmYdblkHcF7qE3CRzX5SCoQ/3w5WeNGXmWb3VwxyT5Ja257wJ9/sTbRYpRRNxS5SC4M8nH8jtrppdOsoDexbWgMStLrhBsEE7zhHM1sdciKCsWZNynQFoATwErEHMnfp0u3BHVWxFqIT6KHpkgbz+qCEWSLwS+Awk7aAL4PigZJ4MeSMUiDA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZTuUecmUbayfpG4RnOkorFODlBki/zYUM4ZazyLUhe8=; b=oqw6RuHtB+taSzK5mS5vYiJSRoHXqa9bDZRejQAOc2n0+FjR3t61JkChsPoZVFCTkg6XI/3L74KlMofKy/nMOOMW5gUcRJeCxV240oFICDGsN6Fe9Jh5WRBwVryUc/MwC1DL6TiypI8f7yhEBmJe1NvBRcPN6dbdrQeaW41cdX8=
Received: from HE1PR0701MB2299.eurprd07.prod.outlook.com (2603:10a6:3:6c::8) by HE1PR0701MB2844.eurprd07.prod.outlook.com (2603:10a6:3:54::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3977.9; Fri, 19 Mar 2021 09:16:13 +0000
Received: from HE1PR0701MB2299.eurprd07.prod.outlook.com ([fe80::a087:95cb:e76b:d57]) by HE1PR0701MB2299.eurprd07.prod.outlook.com ([fe80::a087:95cb:e76b:d57%10]) with mapi id 15.20.3977.009; Fri, 19 Mar 2021 09:16:13 +0000
From: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
To: Praveen Balasubramanian <pravb=40microsoft.com@dmarc.ietf.org>, "ncardwell=40google.com@dmarc.ietf.org" <ncardwell=40google.com@dmarc.ietf.org>, Christian Huitema <huitema@huitema.net>
CC: "tcpm@ietf.org" <tcpm@ietf.org>, Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
Thread-Topic: [tcpm] [EXTERNAL] Re: Hystart and delay jitter
Thread-Index: AQHXF43Ijixf/iHizESn/C9J9cynIqqKXUpggACy2HA=
Date: Fri, 19 Mar 2021 09:16:13 +0000
Message-ID: <HE1PR0701MB229938341C5A909CB5664085C2689@HE1PR0701MB2299.eurprd07.prod.outlook.com>
References: <376bdc9f-4774-bfc8-1736-6c94fb24953c@huitema.net> <CADVnQymN6UH+XTgdkwdX16TsDTeeTu+S-=O1nVjQFWYbpDT24Q@mail.gmail.com> <CY1PR00MB01700C2168E1B4200BE23B34B6699@CY1PR00MB0170.namprd00.prod.outlook.com>
In-Reply-To: <CY1PR00MB01700C2168E1B4200BE23B34B6699@CY1PR00MB0170.namprd00.prod.outlook.com>
Accept-Language: sv-SE, en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=09b6e272-b5ca-4c22-a421-e5ad318ef3b2; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2021-03-18T22:29:46Z; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;
authentication-results: dmarc.ietf.org; dkim=none (message not signed) header.d=none;dmarc.ietf.org; dmarc=none action=none header.from=ericsson.com;
x-originating-ip: [83.227.122.88]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 46485c98-c9cd-46eb-661c-08d8eab7a4a6
x-ms-traffictypediagnostic: HE1PR0701MB2844:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <HE1PR0701MB284421C8BB1BD546D0E5E65EC2689@HE1PR0701MB2844.eurprd07.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: wujEVrY1l0g4c7fAthZ/UUk57bZYLvdzsaKvXTuzmTuOOGdsyOGtSBH2KzFDumm1LyUV+v64HHO5WvyvKOD1dPCRZu+4eNe+OTanDz+aL/ngZwKmGhh87tRUmFb62JyhulwM/qG7LONMmpXo7JsmuQVGg/oEwB48BoX0y0PZ5x7yGv2R/m1sXZkgatzCM+C7dLImnPFO1AJ35+1I9ShmfuR1+M/Hkupd5UJbNK3gySGS5oZ0AsFDIN/wy5Wo2FLc96LfV5pHcI1KUizwrRSr5uXlywT1Fn80obpVzfVwdVPvWpkNzlbM5/q0ZVKM2Ppa/IlrmzdRDFoAtkGVIHvi7MCMmuxtixf8ax3tMZddvKS+yndISevqz+UZvkp4lBcFelwRG58SJW4y5VKTyz235xGeHppXvYN2/IoWF+tZPww5LLYfXzUJGm/IQ3gFR5BjsfPfPtNunfpuCXgeavyn/CO032o9tEF/y6uSO19ba1z9lb+EutCI6sZtGHTN2PHfkYB20zlhDRDyC22mh051mBd4u9pA88me5YQK6FQ1Air3SozQPObzOiJywpWcSvfGSvQXxWij8ZpgfuBKehxLps6Bk5Ra1Ylm7AIL+HYWe8FLpxHOmQ0suROGiR8j0AAnHiyCvCLVNBfY//T7W4twFRLC5cgtL3oDFv5OvwTh0Op823AkkOv8yvEaj1BoPWxndsIoxueLyMLHpyXIb5PgzU5PwnxzJFj3ycmK3odAGeA=
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:HE1PR0701MB2299.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(366004)(39860400002)(396003)(136003)(346002)(45080400002)(186003)(71200400001)(83380400001)(478600001)(66446008)(2906002)(316002)(64756008)(66476007)(38100700001)(966005)(86362001)(33656002)(52536014)(5660300002)(99936003)(55016002)(107886003)(76116006)(7696005)(4326008)(6506007)(53546011)(54906003)(8676002)(8936002)(9686003)(110136005)(66556008)(66946007)(66616009)(26005); DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata: =?us-ascii?Q?Z7jTni0TNTdcLUgfD1PtW/WejiodKCDqZrjFLuiNrls7U7XaIujefIEUWQbz?= =?us-ascii?Q?uE20YOhbwhNQEa7hWfKv1Xj6U/vMV9Z+WeWxjb/HvgBDBR9z0YvLlMagYUil?= =?us-ascii?Q?ojO+X2TMB31n/8b8m+ltKm0bQwSanlsSzpSyVWoBhpEMKDKtC2tKxOiFUWQH?= =?us-ascii?Q?IPGVGWu4nw/bqMjtIkLGKjPjbZtGjGNE6o/rEhtlSVBLOUQoXgK11JgRMdJH?= =?us-ascii?Q?Vn4Q9BbCUVFCYdGeWSaFNz8Ketz6Hv2Xmn9mbZepUFvCMMtBS4kFzS94R0MH?= =?us-ascii?Q?lm144m5vSyqRJxGYYw9BHoFklWwwlvctMqZPIQo9B1jDYr+6dPErL8iucX8b?= =?us-ascii?Q?WKzdVi6hnVHQf306/ew1r8OGRqVo/e4FsmuRplSNQUncz3zkil5ffapM8pqX?= =?us-ascii?Q?7NJFEpZQ/HBO6x1L7Cxb8yDjPyS7LLCF5Ze8E48TlgUv5I8E8FSsDR7aIzfe?= =?us-ascii?Q?APLwcyfeCamksJc6VqCBdjWWKAq4Fa/hkRaY1UuxPew/qt/FjaR16gTdElwF?= =?us-ascii?Q?oKhH1SNmLpVPmrM+LKYYecNMnpulc7U5YFI02nSpXVVaIa+BEOQY4ODXUaX/?= =?us-ascii?Q?Iz1+7NYqvB/FmpcEiEQY5xpGa+XYmPsJPWZ0x4Z87JTZj86o9w8wK3XvN+3Z?= =?us-ascii?Q?WyBhAtZTUgYZxP1mipxqiVFjUXOcJrmpcQ2bmmTE8LX6oiflpJkg1XO0AIjK?= =?us-ascii?Q?eDpTdRkrGo8zgZsKXEhOBnVuibEONoB5o47jdOn5g1f4pP0nPawkNapWWeKe?= =?us-ascii?Q?bubfl5CeyVx+t5HBHK3iKiIJm0ZmBA6MfmB+qrSd3v1tsHp6wQv8YbhTzeOt?= =?us-ascii?Q?lI28yVhAqlrukm8pSMibrqrp7CA3n+tAU0R3vvoFh7Gt0r+lgyKGeZ/4gBbL?= =?us-ascii?Q?lRGoibMtbYBHIGdfxWz2u8i+KFU08Bp9n6olUX2FINsyUqvC/PdvIdD0rhBf?= =?us-ascii?Q?IIP8IY6LRaNPZi7fjb7U5SfqjNlI5v4FLNsfxtjiTIGc1GEz4yJUz9wTqEZM?= =?us-ascii?Q?cxCuz7ShD2BfSkrpBSq9ECqfOqID5skp3WUWc6m414sfKL8w9inZsKw8j81V?= =?us-ascii?Q?N8t3XyHbW106DCpxITI4aUY3DQFrULBEjjpcH6bje6OBbeSu/0PTvfL3oj47?= =?us-ascii?Q?B6fzGLh6bHntl3Nv1PeVY4Wu988W7g4gnuSuAGnJcRiZ9gARSYSuNVo+4wTR?= =?us-ascii?Q?1mzy7Oy60hZMrV+qWgU8p3FlcGFjiKg8S/OkskYmS8Yrf/fAK5BpQ5p05SO8?= =?us-ascii?Q?rzQ9qLczHGvZ7DpmwQlpn6i2qYLrEpmanhxdy/h9ardngLw0lLbkLOL07wI2?= =?us-ascii?Q?3yAsSWCIDPVzrGjcxC6jbAIn?=
Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg=SHA1; boundary="----=_NextPart_000_014C_01D71CA8.E2EA7A60"
MIME-Version: 1.0
X-OriginatorOrg: ericsson.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: HE1PR0701MB2299.eurprd07.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 46485c98-c9cd-46eb-661c-08d8eab7a4a6
X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Mar 2021 09:16:13.3669 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: RjBvBDEA5wINhFa3EQSKwwzn6AbsHzcHEWEOcKKv16MKf5J7gwDq9fhrpuylKZnUvBSxOulWhAwZbRVz0uKuSdOSwH1S048Hv5HUZtgoy4IU1yQd6iYLlyoaLMdkxTgq
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0701MB2844
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/UOCD7XXjKmE0QbMjA6fja2th3oE>
Subject: Re: [tcpm] [EXTERNAL] Re: Hystart and delay jitter
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 19 Mar 2021 09:16:20 -0000

Hi

My finding is that it is a bit problematic to rely on RTT estimates for path
capacity especially with cellular access . It was summarized in section 4 in
https://tools.ietf.org/html/draft-johansson-cc-for-4g-5g-02 and also
presented in
https://www.ietf.org/proceedings/96/slides/slides-96-iccrg-1.pdf

Regards
Ingemar

> -----Original Message-----
> From: tcpm <tcpm-bounces@ietf.org> On Behalf Of Praveen
> Balasubramanian
> Sent: den 18 mars 2021 23:37
> To: ncardwell=40google.com@dmarc.ietf.org; Christian Huitema
> <huitema@huitema.net>
> Cc: tcpm@ietf.org
> Subject: Re: [tcpm] [EXTERNAL] Re: Hystart and delay jitter
> 
> Thanks Christian for the suggestions! We will include a discussion on
jitter in
> the next draft update and also experiment with the min of max function.
> 
> -----Original Message-----
> From: tcpm <tcpm-bounces@ietf.org> On Behalf Of Neal Cardwell
> Sent: Friday, March 12, 2021 2:18 PM
> To: Christian Huitema <huitema@huitema.net>
> Cc: tcpm@ietf.org Extensions <tcpm@ietf.org>
> Subject: [EXTERNAL] Re: [tcpm] Hystart and delay jitter
> 
> On Fri, Mar 12, 2021 at 4:29 PM Christian Huitema <huitema@huitema.net>
> wrote:
> >
> > Back in November 2019, when adding Cubic and Hystart to my
> > implementation of QUIC, I noticed that Hystart was sensitive to delay
> > jitter. Hystart detects the buildup of queues by monitoring the RTT.
> > Some links experience delay jitter, caused for example by access
> > protocol for shared radio links or possibly by link-local ARQ protocols.
> > The delay jitter can cause Hystart to make the wrong decision, in two
ways:
> >
> > 1) Delay jitter during a previous period could cause some packets to
> > be delivered "faster than usual", causing Hystart to under-estimate
> > the min RTT for that period.
> >
> > 2) Delay jitter during the currently measured period can cause packets
> > to be delivered "slower than usual",  causing Hystart to over-estimate
> > the min RTT for that period.
> >
> > The combination of these two issues may cause Hystart to make the
> > wrong decisions, and exit slow start at levels well below link capacity.
> 
> Yes, we have found in both our production experience and controlled
> experiments that the Hystart-Delay algorithm is very susceptible to
spurious
> triggering from jitter, particularly in LTE and wifi paths.
> 
> We discussed this a bit in Spring 2017 in the comparison of BBR's
bandwidth-
> based mechanism for exiting startup, vs Hystart-Delay's delay-based
> mechanism:
>   https://protect2.fireeye.com/v1/url?k=6c35f204-33aecb47-6c35b29f-
> 861fcb972bfc-c8f94340e007b7bd&q=1&e=fd13218f-2e70-4b39-a879-
> e9b53c01447d&u=https%3A%2F%2Fnam06.safelinks.protection.outlook.com
> %2F%3Furl%3Dhttps%253A%252F%252Fwww.ietf.org%252Fproceedings%25
> 2F98%252Fslides%252Fslides-98-iccrg-an-update-on-bbr-congestion-control-
> 00.pdf%2523page%253D8%26amp%3Bdata%3D04%257C01%257Cpravb%2540
> microsoft.com%257Cbc3399870c0d4105299408d8e5a4cbaa%257C72f988bf86f
> 141af91ab2d7cd011db47%257C1%257C0%257C637511843745826011%257CUn
> known%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBT
> iI6Ik1haWwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3D6SQWDb1
> mFH0cgrayEwiRYeccYvP0dv%252BAJ1eokyJF%252BKs%253D%26amp%3Brese
> rved%3D0
> 
> > The draft-ietf-tcpm-hystartplusplus-01 does have some protection
> > against the second issue, because currentRoundMinRTT is computed on at
> > least N_RTT_SAMPLE. If that number is large enough, computing the min
> > over N samples should filter out "slower than usual" anomalies.
> > However, the draft does not include a protection against "faster than
> usual"
> > anomalies happening in the previous period. In my implementation, I
> > protected against that by computing a "min of max" function: compute a
> > rolling "MAX over N_RTT_SAMPLE", then compute the MIN value of that
> > during the reference period, and use that to set the reference value
> > "lastRoundMinRTT".
> >
> > I think it would be good to add a discussion of the effect of jitter
> > to the hystart++ draft. In addition, we may also want to mention
> > timestamps. The jitter on RTT may be caused by jitter on either
> > direction of transmission -- data path or ACK path. The effect of
> > jitter on the ACK path can be minimized if time stamps can be used to
> > monitor the variation of one-way delays. This is not discussed in the
> > current draft. Maybe it should be.
> 
> A discussion of the effect of jitter sounds like a great idea.
> 
> neal
> 
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://protect2.fireeye.com/v1/url?k=032901f3-5cb238b0-03294168-
> 861fcb972bfc-7a6ccb41bd90a846&q=1&e=fd13218f-2e70-4b39-a879-
> e9b53c01447d&u=https%3A%2F%2Fnam06.safelinks.protection.outlook.com
> %2F%3Furl%3Dhttps%253A%252F%252Fwww.ietf.org%252Fmailman%252Flis
> tinfo%252Ftcpm%26amp%3Bdata%3D04%257C01%257Cpravb%2540microsof
> t.com%257Cbc3399870c0d4105299408d8e5a4cbaa%257C72f988bf86f141af91a
> b2d7cd011db47%257C1%257C0%257C637511843745826011%257CUnknown%
> 257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1ha
> WwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3Doqte6wCaTBUvrw
> gXVRMcD13YgtspMJyIWMMLlwNuYdY%253D%26amp%3Breserved%3D0
> 
> _______________________________________________
> tcpm mailing list
> tcpm@ietf.org
> https://www.ietf.org/mailman/listinfo/tcpm