Re: [tcpPrague] Pacing IW

Ingemar Johansson S <ingemar.s.johansson@ericsson.com> Fri, 05 February 2021 10:48 UTC

Return-Path: <ingemar.s.johansson@ericsson.com>
X-Original-To: tcpprague@ietfa.amsl.com
Delivered-To: tcpprague@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A32843A0C22 for <tcpprague@ietfa.amsl.com>; Fri, 5 Feb 2021 02:48:37 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.351
X-Spam-Level:
X-Spam-Status: No, score=-2.351 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KVUIjTtzaR7d for <tcpprague@ietfa.amsl.com>; Fri, 5 Feb 2021 02:48:35 -0800 (PST)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60083.outbound.protection.outlook.com [40.107.6.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 51CA63A0BBA for <tcpPrague@ietf.org>; Fri, 5 Feb 2021 02:48:34 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=SG54YndVGDp50VWxieEovqJpgmEkxtmGUL5Ca5akgMSQPSR/DNqKSbtWfVE0QAnWiuxJK/PeuqwOdbCYWJhX48uF2TmtZ0hS8y7QI0AYPuZOytZ9VNJMV3B+xD96bYA5rJvnjLdgODewvfl0bE34ndmPsxxMnidgiDsAIZZJp42GdDvrR8xLtigLN0stO1QR89O+dbqV12q2sXEFiGHCW8YA8DgzVqzxbTLpZhJXu7/UvUKNOZT4jLVNvPblS8rBNeHUpHf4FLoOieiOK2VnwCcVkIvab83LKbKsKNjPSys83m4Mu1DuWx/P/mP+s/s5mECc2aTC7XPQxIlr9CFUTg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=D++sTDNSRPPejv/tf8bKX0D0jlivU3D5tWTQjrF/g7s=; b=cH6X5ph7BLagNcjGEZBoYEe4kWHfS6xCPhdPVMuRzDqWkWV002qN5prrjnHHk5kBzUckM3iJ4RfWCHME+KFfVDMDIzU4CetXY5yHmFDEQ7GMkNgPKbaxwY11EkuCCObfNJ0cIDyc7egcJ3HG47wU+eiQ2yQ0ny6g5DR8sCnwGf/AskF3CNWG944gxTt32c0KLcpudFG32mQnMIaRNRcMYrKkMa7pL3cejaaIL1SMQ9lcBQxRC4HF+oSQ3QS9xobDbBsxUZ+an3wSeoHrC90bZsDZpQJQYms/r3JEvfTz3efRBSnMvgbvj5py8MAa0ljoldciHvRhRo588hxpiMDaMg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=D++sTDNSRPPejv/tf8bKX0D0jlivU3D5tWTQjrF/g7s=; b=Dj3PWisIowu30orGCCNmzpNE3UAQq2Ham9RyCEfG8brk8VILKAlJx4wTokV5DXOmvuk6nKXZM9j9BYHCoWJLm0TP3M28LStjTymZ95wZectpXrTzGny8KF+lYM4qattmGodRSxXEMciz0KQNQE5aLm8xhhxSI6dths9uSf9F+eo=
Received: from (2603:10a6:3:6c::8) by HE1PR0702MB3579.eurprd07.prod.outlook.com (2603:10a6:7:7e::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.12; Fri, 5 Feb 2021 10:48:31 +0000
Received: from HE1PR0701MB2299.eurprd07.prod.outlook.com ([fe80::494d:6160:61fd:5b1]) by HE1PR0701MB2299.eurprd07.prod.outlook.com ([fe80::494d:6160:61fd:5b1%9]) with mapi id 15.20.3825.020; Fri, 5 Feb 2021 10:48:31 +0000
From: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
To: "Tilmans, Olivier (Nokia - BE/Antwerp)" <olivier.tilmans@nokia-bell-labs.com>, Bob Briscoe <research@bobbriscoe.net>
CC: TCP Prague List <tcpPrague@ietf.org>, Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
Thread-Topic: [tcpPrague] Pacing IW
Thread-Index: AQHW+Z4LM/xHQu3tVkq5ODsEOVqBUqpJYn1g
Date: Fri, 05 Feb 2021 10:48:30 +0000
Message-ID: <HE1PR0701MB22997B76CFFA0762558E6D0DC2B29@HE1PR0701MB2299.eurprd07.prod.outlook.com>
References: <1ddb42c7-596b-031c-6215-84e675d08b7d@bobbriscoe.net> <AM8PR07MB7521C068DC045E27A99C0022E0B59@AM8PR07MB7521.eurprd07.prod.outlook.com>
In-Reply-To: <AM8PR07MB7521C068DC045E27A99C0022E0B59@AM8PR07MB7521.eurprd07.prod.outlook.com>
Accept-Language: sv-SE, en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
authentication-results: nokia-bell-labs.com; dkim=none (message not signed) header.d=none;nokia-bell-labs.com; dmarc=none action=none header.from=ericsson.com;
x-originating-ip: [83.227.122.88]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 133a7d3c-eeea-4a63-7d6e-08d8c9c393ea
x-ms-traffictypediagnostic: HE1PR0702MB3579:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <HE1PR0702MB35799FA3BCF37F1D4A1C9424C2B29@HE1PR0702MB3579.eurprd07.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:10000;
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: aPKvquRAPYVYpsO6OBnaLAEr7ZH5QFEhWHDFuP6XvvXibcnCmPWDoSL+qn/CoBjOLjYRG2ZIAvmGjrYB6e7Daf4eY5SPJe20E8ZgxQ9VS21AuaOEFPZMFQmHbfjacS036DfwhvtaQ9BG7YN3gZYgXQZCvFQ4Xg2uAp8/vsIYviOdMUCdEbs80HfZaGEShv4CajAb4bYt12U5jPiiI07IUJGBytD2t4GoQ36SV/fXJt3IbI8CaO/gllh1DuPHIOHU0e8DXpTbbepU1kWw72QFlE37/+Vd+261jFUsniCHwHJJbDIn69JoYQ7z6iWQN6R6GA7OhcKB2vwJRxhxjntySrk3zXuI6t3xpEg58RmNJNCBnkD3x7v1SF1TT+Nlb8A8+/pXPc8i7m90iYcbVPsnsxvjEGuUu9G5TWQuH7e353H1BUQ1eoDIoBDKzd0IqARg3xUyQw4v3loo9rG6d0qELOReQx7PAnsqdX2Towb75ceN2pLHzXR49aUUz94tiKiOkyISUFEB2k+rYcx7I2ZxxRQZDGc48JhHw5a9utV2RjgDWs4/2nSbFrIa8V9eOEJvTbJdF8UtartdXqCeJ9XoXDesIGTuwffD6qAhrY9Z9oaCh1R9qpYjGTQUNDj+RD6Vz6ogh+B/GjCqVhJ38YLIGPN8DibTfYLFt+MCHVR8Csh6D6RHwDok086ILLAp6r+6
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:HE1PR0701MB2299.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(54906003)(110136005)(2906002)(53546011)(6506007)(966005)(9686003)(5660300002)(55016002)(107886003)(86362001)(33656002)(66556008)(64756008)(4326008)(66616009)(66476007)(83380400001)(66946007)(8676002)(186003)(66446008)(52536014)(26005)(71200400001)(7696005)(8936002)(99936003)(498600001)(76116006)(336755003)(18886065003); DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata: 7Jz0v3lMYE3e+YWiOetqY9zZQsZMSAHR0Y5Z+kHwABAFk3smJ24Y+nD5WDAnUlr9nDQSYIR8hsFwBsEubViSFElYUCdMQIGdmQS8h8ckpO42laVRHQPvNErgy12iyJmOW+kUCeQCWRjqBx3h586a+4WPyx+Zdr/D8RORVbzIPNwnb3y6ZQXmLJAqYsvdfaygu+5RLXz9/jSaKwzN8lc40dMnFcE5wKn/OhMhBsdLf9HR2vqLo7zzpHoIxul5WiNJ+XKSsyHFRMedfZhzJk4PI1xwYp5WQzM1iJscUrQm6Ze1CYYGzLSQmHbNj/UUBH44lFVtO+vCTlGAC2YSs315Pmj7G8BHOF+RZOy8CZX++RNy0VSiZ2tEj2lrCV9f+z4a4bHOJZEhuOAxN8IST1R/ktb8HsiJTZECmrYrRlWE6VMlrxqzX00W888/QmEcFQzdEp3TG8rh1E0qwfKkB6CviPsYlfxW6EssieCdIP859j8v24HtqyptoRde/q2J5gd7WgCGEEFR2eihMSTXdCY0A8+ZQANfcrpNd5MiJxDiqs+TO60qWDDxTCsQ+NFI5Kl3z0qaZdfLcR2cgPmEnapBkev69KuesCZNM/e0PXZxi+hD1sW5gVYs6PQHqNiGemxSWHU4Vv2+RbLL/rKtrh3jmA6NsaH3jfhrPjjsirWUsFrB6mJscz9H/Ns+MNrfdXcR0uCt+f6Js4eHPpaWHJnxdQ8nddvFDur+n7ILLxEmrNIg+QKtHfNZuN9mwbLqDPAgl/q+v8swYiYR7c7/Pq9bIZgAkmkoghDRPWXcop8eZTIGVzSYs53T21zMwokETm0zKrJ1Y4kvUWWADqWfwh3n28qTJxEXRtcYOfAvHhorDMZquKzBySY4XqK/UxKUNrFS8jyYESp4/SN7Yn+VYqgIteO0ici8QkIC74Gwn7RRyDha9V9T7kX9szPNvgIDJGAp9I5QgACga7tQAuJwNuYFyh1jH43VeOU1BCJ+MlXu60EDLvGu13ypTQIRg/QAINBb5i2fn7qWpULit+1p2k3443ZnGp5Xy5t0newFes9vtVoK+P2ZZbumyAixm2RM1YqJe8KhGLkR5TkNrt3hPMHEialA4VMCyfL33F1+Syi6L0Y2SFE+ZXmdvWcDK3mp4Fxtcyy+ewM/z2nTVo0DwhaBhTiqBGRw3sahG13BwIZ7bdiw4H4/HPwJQqwSSHIN1lbCsIvWILO6jGt15adyj4j23LiDEi/VXAv2HK+ykfCPLUHRaB4mk+1AJXXCgcwrHKUaYK5ClZfxZdv6jsLHLomTd+nZJIXUaIpgfHY+1QwY8QvgDTX/6Ry0J4Kn8Fuz5BFy
Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg="SHA1"; boundary="----=_NextPart_000_021A_01D6FBB4.D25C55C0"
MIME-Version: 1.0
X-OriginatorOrg: ericsson.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: HE1PR0701MB2299.eurprd07.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 133a7d3c-eeea-4a63-7d6e-08d8c9c393ea
X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Feb 2021 10:48:30.9069 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ud9Duw2wWyuVWe2B1TSMnmxuElOdsjS5/Ox8Iv1MKLiP0DPn11cSA6Q+bxbWEQ3M/NQoVAnIxzOgEtHWkNuFMkJkU+zRUiYQyZoA4EO8hxiOaj9AvyKnOhGE2AGKE0UW
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0702MB3579
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpprague/1nGydZbYTm-1aLCFB4cfZZXWAJE>
Subject: Re: [tcpPrague] Pacing IW
X-BeenThere: tcpprague@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." <tcpprague.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpprague>, <mailto:tcpprague-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpprague/>
List-Post: <mailto:tcpprague@ietf.org>
List-Help: <mailto:tcpprague-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpprague>, <mailto:tcpprague-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 05 Feb 2021 10:48:38 -0000

On the pacing matters based on (initial RTT).

It is true that an initial RTT based on e.g. 3WHS can give values that are 
higher than the actual values when for instance cellular access is used.
There are a few reasons to this :
1) DRX (Discontinous reception) : A battery saving feature that makes the cell 
phone (a.k.a UE) sleep periodically. This feature is noticeable e.g. when ping 
measurements are done
2) UL scheduling : To transmit in uplink, an UE must first send a scheduling 
request (SR) to the base station, the base station then transmits a grant a 
short while after. The grant may however be too small, in which case the UE 
must piggy back a buffer status report (BSR) to the transmitted data.

All this gives the potential that the RTT may become higher than the true 
value

/Ingemar

> -----Original Message-----
> From: Tilmans, Olivier (Nokia - BE/Antwerp) <olivier.tilmans@nokia-bell-
> labs.com>
> Sent: den 2 februari 2021 14:08
> To: Bob Briscoe <research@bobbriscoe.net>
> Cc: TCP Prague List <tcpPrague@ietf.org>
> Subject: Re: [tcpPrague] Pacing IW
>
> Hi Bob,
>
>  > I notice you've pushed code to TCP Prague, to optionally pace out IW over
> > the estimated rtt (srtt).
>  >
> https://github.com/L4STeam/linux/commit/aea3ff36380bec82804eba8e3f99
> e5f14
>  > 342978c
>
> Note that this is disabled by default: this approach is quite fragile as you
> rightfully point later on; it enables quick experimentation. I'd expect
> someone willing to use this in production to have a finer-grained approach,
> e.g., using the destination cache to get recent srtt/cwnd values.
>
>  > [...]
>  > - If the initial srtt estimate is too low, there will be a queuing delay 
>  > spike,
> but it will never be worse than just sending IW at line rate.
>
> Agreed--I would expect this to happen, e.g., with PEPS terminating a
> connection.
>
>  > - If srtt is too high, a short flow would take longer to complete because 
>  >
> the first ACK will return before the IW has finished sending.
>
> Agreed. More generally, that tradeoff is extremely application dependent,
> so applications sensitive to this extra delay would either not use this or
> control the pacing rate used themselves by overwriting the socket max
> pacing rate.
>
>  > [...]
>  > So, possible solutions (either or both):
>  >
>  > 1. In tcp_prague.c, multiply the rate returned from  >
> prague_update_pacing_rate() by N (where N=2, say). So IW is at least  >
> paced over the first half of the RTT on average. Then, if the initial  > 
> srtt
> estimate is longer than reality, the error has to be 2x before IW  > will be
> delayed more than a whole round.
>
> This is already the case, given the connection is seen as in slowstart (
> ssthresh=infinity), the pacing rate will be (2 * IW/srtt)
> https://github.com/L4STeam/linux/blob/testing/net/ipv4/tcp_prague.c#L30
> 6
>
>  > 2. Introduce a third sysctl option (tcp_pace_iw = 2) that only paces IW 
>  > if
> srtt was initialized using the destination cache?
>
> I'd happily merge any addition or better ways to do this.
>
>  > Altho this is in TCP Prague now, I guess eventually it could be taken up 
>  >
> by any transport with any CC.
>
> Indeed, this was the quickest way to get it to work. It can definitely be
> altered to use the destination cache, and/or to use sk_max_pacing_rate in a
> smarter way (e.g.,!= infinity indicates the application explicitly requested 
> it
> before queuing data for its IW, which could/should be preferred to any
> generic in-kernel computation/assumption), and/or to be CCA independent.
>
>
> Best,
> Olivier
>
>  >
>  > Cheers
>  >
>  >
>  > Bob
>  >
>  > PS. Defensive coding nit: In tcp_output.c, I noticed you've kept the  >
> condition
>  >     if ( ... || tp->data_segs_out >= 10 )
>  > Surely this shouldn't have been a hard-coded '10' in the first place?
>  >
>  >
>  > --
>  >
> __________________________________________________________
> ______
>  > Bob Briscoe                               http://bobbriscoe.net/