[iccrg] Sub-packet congestion window

Bob Briscoe <research@bobbriscoe.net> Tue, 19 November 2019 02:05 UTC

Return-Path: <research@bobbriscoe.net>
X-Original-To: iccrg@ietfa.amsl.com
Delivered-To: iccrg@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id 4A834120232 for <iccrg@ietfa.amsl.com>; Mon, 18 Nov 2019 18:05:49 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.998
X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=bobbriscoe.net
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id rM7kw5DBRnpW for <iccrg@ietfa.amsl.com>; Mon, 18 Nov 2019 18:05:44 -0800 (PST)
Received: from server.dnsblock1.com (server.dnsblock1.com []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CF35C12004E for <iccrg@irtf.org>; Mon, 18 Nov 2019 18:05:43 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bobbriscoe.net; s=default; h=Content-Type:MIME-Version:Date:Message-ID:Cc: Subject:From:To:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=wm4neeoQhTZFNMhQsSFKG/6vkZMLIMnsz6lBiWoJ+Zo=; b=IWdftCmdet6ZUCDmKVGgngk9Bj MRiOuHdiYH2PQV7eSQaCK8TI/31vWBUPV0c06u5scRtMqPX4bKijUAR7HLeeYbMHodnHEwv1JJNpX q3qnHh8flCaRr8ztwkqPNxRJ/SEKHTLz+cBV/RM3UiUce9IfkP5bnRYZRczGruP4pqz9VHIKWbYsi K1hH/Tv1Y81WLOfVMW7yDQt7oazz5uovCZ/faOhpTQC7t9iExfoXusF7bdltxTPvDDS49qd72cLUu 5uSOHxvpl/nutO86o/7D+mBv30+Ns4nvOwlX0/GpbyRrbwNx/IPyWFCqTMvuG4ULIaSc3j3VHwrA5 VinS404Q==;
Received: from [] (port=35812 helo=[]) by server.dnsblock1.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.92) (envelope-from <research@bobbriscoe.net>) id 1iWsth-0000YN-J6; Tue, 19 Nov 2019 02:05:41 +0000
To: iccrg IRTF list <iccrg@irtf.org>
From: Bob Briscoe <research@bobbriscoe.net>
Cc: Asad Sajjad Ahmed <me@asadsa.com>
Message-ID: <ca2ae733-b63a-6031-0839-a15221931ca7@bobbriscoe.net>
Date: Tue, 19 Nov 2019 10:05:38 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="------------BD5109C94E8B1EBB015E8BFD"
Content-Language: en-GB
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - server.dnsblock1.com
X-AntiAbuse: Original Domain - irtf.org
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - bobbriscoe.net
X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net
X-Authenticated-Sender: server.dnsblock1.com: in@bobbriscoe.net
Archived-At: <https://mailarchive.ietf.org/arch/msg/iccrg/jKZE5SqiurlOSqlYyEtrudC0ppk>
Subject: [iccrg] Sub-packet congestion window
X-BeenThere: iccrg@irtf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussions of Internet Congestion Control Research Group \(ICCRG\)" <iccrg.irtf.org>
List-Unsubscribe: <https://www.irtf.org/mailman/options/iccrg>, <mailto:iccrg-request@irtf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/iccrg/>
List-Post: <mailto:iccrg@irtf.org>
List-Help: <mailto:iccrg-request@irtf.org?subject=help>
List-Subscribe: <https://www.irtf.org/mailman/listinfo/iccrg>, <mailto:iccrg-request@irtf.org?subject=subscribe>
X-List-Received-Date: Tue, 19 Nov 2019 02:05:49 -0000


You may recall Neal outlined the problem of a sub-packet congestion 
window in his talk on BBRv2 yesterday.


As promised in the iccrg session yesterday, here's the recent work on 
this he mentioned was expected soon:

  * Ahmed, Asad, S., "Extending TCP for Low Round Trip Delay
    Master Thesis Thesis submitted to: University of Oslo (August 2019)
  * Modification to Linux TCP stack
    <https://bitbucket.org/asadsa/kernel420/src/submss/> to work with a
    sub-packet window (research prototype)

I've also linked to Asad's Master's thesis and code off the L4S landing 

  * L4S: Ultra-Low Queuing Delay for All
      o papers <https://riteproject.eu/dctth/#papers>
      o code <https://riteproject.eu/dctth/#code>

A more permanent archival URL for the thesis should appear very soon 
(the Uni's library takes 90 days to make a thesis available, which is 
nearly up).

      ==More Info==

If time, we'll write a paper on it - a thesis isn't the most accessible 
medium. We'll prepare a talk on this too - I've requested a slot in the 
next iccrg, but no promises 'cos Jana wants to make it a BBR special 

Asad is in cc if you have questions, or he's here in Singapore (except 
he was taken ill yesterday, but I hope he'll have recovered soon).


It's not specific to TCP Prague: Asad modified the base TCP stack and 
tested it with Reno and DCTCP. It uses a modified AIMD called LS-AIMD 
(logarithmically scaled AIMD), so more work would be needed to modify 
congestion control modules that are not pure AIMD, e.g. Cubic

The patch was fairly small in the end, although the design process we 
went through was tortuous, given we were breaking some fairly basic 
assumptions in TCP's design:

  * First, of course, Asad had to represent sub-packet cwnd. To minimize
    impact on the code, he used the usual Linux variable in segment
    units complemented by an integer part in bytes.
  * When sending less than 1 packet per RTT, one has to pace, of course;
  * Then the additive increase becomes larger than the multiplicative
    decrease - solved by logarithmically scaling the additive increase
    dependent on ssthresh, rather than a constant of 1 segment per RTT.
  * Then we had to fix Proportional Rate Reduction
  * Then we found bursts echoed for multiple round trips, which we
    solved by re-basing the pacing timer whenever we received an ACK,
    instead of always basing send times off each other.
  * Then we had all the implementation problems you get when you don't
    have a frequent enough ACK event to trigger the code to do stuff. So
    we had to adjust certain per-ACK actions mathematically so the
    outcome was as if they had been repeated per RTT (e.g. maintenance
    of DCTCP's EWMA).
  * Also had to produce patches for bugs in the original code on the way

It all worked nicely in the end. Lots more testing to do tho (see 
thesis) - ran out of time, as is traditional .

As I said in iccrg, there are other ways to approach this, but with this 
considerable experience under out belts, we are better placed to judge 
the pros and cons.



Bob Briscoe                               http://bobbriscoe.net/