[tcpPrague] Pacing IW

Bob Briscoe <research@bobbriscoe.net> Tue, 02 February 2021 12:18 UTC

Return-Path: <research@bobbriscoe.net>
X-Original-To: tcpprague@ietfa.amsl.com
Delivered-To: tcpprague@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id 2D8D03A1A3C for <tcpprague@ietfa.amsl.com>; Tue, 2 Feb 2021 04:18:45 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=bobbriscoe.net
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id 1qljDM7Z5bV2 for <tcpprague@ietfa.amsl.com>; Tue, 2 Feb 2021 04:18:43 -0800 (PST)
Received: from ssdrsserver2.hosting.co.uk (ssdrsserver2.hosting.co.uk []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EC9A23A1A3B for <tcpPrague@ietf.org>; Tue, 2 Feb 2021 04:18:39 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bobbriscoe.net; s=default; h=Content-Type:MIME-Version:Date:Message-ID: Subject:From:Cc:To:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=3qNLH4qjfFAAFIu4AO1+ruI2u2ojIbOOxyRQ6ZobTYY=; b=Xa5gRR1KcoQbM2F2buPh7fET3m fNbaOm5GwIZ6yMrU5EriMtArmua4w2eXbEHNEtdZSqNvVgSCJDk1hj0AIvRpEfJjvBXOH9j3vskbw jHHYw47DFbn1gm7cN0+34wMTcBC96I621ArRsYvVXFD2oS96PZX6imxO6a/HOOAibz3RvBWBOXt9G QVnLpBdkkikGXv75rnuscAbEEUfC1rdp0PyVm3Z+1qgt/nCv7TepGy4qCIRasO0IL0nJV88olk8CF zBoWCA5s4km7shIkQDRxkVg22ZfigH+h7lz1i4my2nxQgRBOVrv0yPpZgp+ZgKH7E1DpdFTQq8qCw kHF2ZCag==;
Received: from ([]:45874 helo=[]) by ssdrsserver2.hosting.co.uk with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from <research@bobbriscoe.net>) id 1l6udh-0000Wp-Hc; Tue, 02 Feb 2021 12:18:37 +0000
To: "Tilmans, Olivier (Nokia - BE/Antwerp)" <olivier.tilmans@nokia-bell-labs.com>
Cc: TCP Prague List <tcpPrague@ietf.org>
From: Bob Briscoe <research@bobbriscoe.net>
Message-ID: <1ddb42c7-596b-031c-6215-84e675d08b7d@bobbriscoe.net>
Date: Tue, 02 Feb 2021 12:18:36 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="------------ABDF2AAAF8722BCCC1BCBB0C"
Content-Language: en-GB
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - ssdrsserver2.hosting.co.uk
X-AntiAbuse: Original Domain - ietf.org
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - bobbriscoe.net
X-Get-Message-Sender-Via: ssdrsserver2.hosting.co.uk: authenticated_id: in@bobbriscoe.net
X-Authenticated-Sender: ssdrsserver2.hosting.co.uk: in@bobbriscoe.net
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpprague/4kyi0jUf5rqCv3k-NCy3FXpgDrk>
Subject: [tcpPrague] Pacing IW
X-BeenThere: tcpprague@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." <tcpprague.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpprague>, <mailto:tcpprague-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpprague/>
List-Post: <mailto:tcpprague@ietf.org>
List-Help: <mailto:tcpprague-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpprague>, <mailto:tcpprague-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 02 Feb 2021 12:18:45 -0000


I notice you've pushed code to TCP Prague, to optionally pace out IW 
over the estimated rtt (srtt).
Thanks for this. Some thoughts:

srtt can tend to be rather wildly wrong at the start of a connection. 
I'd say a high estimate will be more concerning than a low, by this 

- If the initial srtt estimate is too low, there will be a queuing delay 
spike, but it will never be worse than just sending IW at line rate.
- If srtt is too high, a short flow would take longer to complete 
because the first ACK will return before the IW has finished sending.

I recall Mark Handley presenting data showing longer RTT outliers at the 
start of connections (cause was unknown, but could have been a CDN 
getting its act together to serve the connection from an optimal 
location). Sorry, can't find the paper - it was perhaps 5-10 years ago. 
Whatever, in general, an over-high initial estimate is surely more 
likely than over-low, given the initial handshake might be delayed more 
than other packets, but it's unlikely to find some worm-hole that gets 
it back home faster than the speed of light.

So, possible solutions (either or both):

1. In tcp_prague.c, multiply the rate returned from 
prague_update_pacing_rate() by N(where N=2, say). So IW is at least 
paced over the first half of the RTT on average. Then, if the initial 
srtt estimate is longer than reality, the error has to be 2x before IW 
will be delayed more than a whole round.
2. Introduce a third sysctl option (tcp_pace_iw = 2) that only paces IW 
if srtt was initialized using the destination cache?

Altho this is in TCP Prague now, I guess eventually it could be taken up 
by any transport with any CC.



PS. Defensive coding nit: In tcp_output.c, I noticed you've kept the 
     if ( ... || tp->data_segs_out >= 10 )
Surely this shouldn't have been a hard-coded '10' in the first place?

Bob Briscoe                               http://bobbriscoe.net/