[tcpPrague] TCP Prague under-utilising capacity when RTT scaling turned on

Ashutosh Srivastava <as12738@nyu.edu> Thu, 28 May 2020 19:20 UTC

Return-Path: <as12738@nyu.edu>
X-Original-To: tcpprague@ietfa.amsl.com
Delivered-To: tcpprague@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A63CA3A0765 for <tcpprague@ietfa.amsl.com>; Thu, 28 May 2020 12:20:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Level:
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_IMAGE_ONLY_32=0.001, HTML_IMAGE_RATIO_08=0.001, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=nyu.edu header.b=TEPXJziI; dkim=pass (2048-bit key) header.d=nyu-edu.20150623.gappssmtp.com header.b=uIqeQN4Y
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rugDs6h_WbMR for <tcpprague@ietfa.amsl.com>; Thu, 28 May 2020 12:20:55 -0700 (PDT)
Received: from mx0b-00256a01.pphosted.com (mx0a-00256a01.pphosted.com [148.163.150.240]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EC7BE3A0746 for <tcpprague@ietf.org>; Thu, 28 May 2020 12:20:54 -0700 (PDT)
Received: from pps.filterd (m0142700.ppops.net [127.0.0.1]) by mx0b-00256a01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 04SJ6Wvr030799 for <tcpprague@ietf.org>; Thu, 28 May 2020 15:20:54 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nyu.edu; h=mime-version : from : date : message-id : subject : to : content-type; s=20180315; bh=q93MczlnzMEoq4J7ZeMRrXpj0Gr1zV1cs7d6hRhVyWE=; b=TEPXJziIEOPsXaTHEuwp6Ax2Ef7EW9XuHUnF8HdAOPZZtiHB2yJflE5G/uewv4coP1c+ pAzzRwjVqcAUNH2mRGYEA5H6uX8SSyT8aL3Jlj6r0lXVBIJ3NTPCggbCwZyeSxgjr1Ml sLMHSq9DQf924kiWeGXtWFBnfiD2JV+ibPa9RhlDxal1ojtvrKEsilhXu66sYpiVc+eT hSXWM/vVW0KqN603PkrcEvopt2sjBBcbexVlI/ZEeS5LoE/4OeuaBFG4z4aMS4EmzZTS vHp23u25DZTTLPDZrhiHWszKOwnrMSx/hATfCraZGJCqtkjQQ/l0uIPfeoWAWhD5TERM 4A==
Received: from mail-il1-f197.google.com (mail-il1-f197.google.com [209.85.166.197]) by mx0b-00256a01.pphosted.com with ESMTP id 31a72t2scs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for <tcpprague@ietf.org>; Thu, 28 May 2020 15:20:53 -0400
Received: by mail-il1-f197.google.com with SMTP id w65so161386ilk.14 for <tcpprague@ietf.org>; Thu, 28 May 2020 12:20:53 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nyu-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=q93MczlnzMEoq4J7ZeMRrXpj0Gr1zV1cs7d6hRhVyWE=; b=uIqeQN4YNNyfi9mvACc0Yv6U9Gr8CShJ64yMqpSBBzTV5Xy1+wSv51FqhQhlGeOgws CrzAveAy8Gz+aKRJjP7IOX27n3NHlWxECASja9puNjJL2RjnIV6OMwbLUlbRAilNjIrB 1y6tQkt635bJm2aCBsT+1iuzBaxsTM8uoHzY4RIoaAdsUUeTcmR4tadykAfwXbwTE3Y1 5QIJgs9+4rm86SSyrH0Ls9VaEePOheQx82Ee8kGpt8whAnTsoFJ4tBUqNNLGMBgsLYfy iM1Ksb6Qp+f3+fnV8Ec+GD5JPa62I//BMvoOOvh1+UefqWnZOCuJjrtWjhE4jaHl40VE G/qA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=q93MczlnzMEoq4J7ZeMRrXpj0Gr1zV1cs7d6hRhVyWE=; b=IFF5rIa3/pWT9Hl7jyg35ApqAW/5OJhrk7/LkFl1yNSHqvVpR2FstC9f6ptRMUyy7d 4vUoQx68r1mD9F1RomoMdRt37OiFr61+VUT+jbgHOFnMRR8DgOsTeUSi5gkCMEJ1h/Q9 DVwi6c/GRROVchZjjIXPMR6uE+GBsKFGo2kz9PBpnN4icjTU4XEKtSfdG0/CNJuXL0cW pTkLVU+Uff2AaU9KcqUtu/CDRoqdGqv0ebVEK8P5JRGKyHuWfxbKo0pfu8R3onaAEnKx TpW9vYr0Qh4K4atn6qwGq2oPdczhLi6aTURQ/BnV8NPVsfERfkocJoyLFituzz1v0pAO nT7g==
X-Gm-Message-State: AOAM533smDWcHCkBD9VpjDtqLW8vxz0NWT+nbiCeZKst//VQMBK/4sri FizKx/p6X6h2U/z///4fSSYM1C1J2TbuWqpVEg+/oEBp5gvW7Z4CujHVYoCon9lUnoSRqYs15vp e0nXGGCgTBpvrLLtspFHgUtZNCg==
X-Received: by 2002:a5d:9598:: with SMTP id a24mr3651209ioo.182.1590693652723; Thu, 28 May 2020 12:20:52 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJxA0gMICv1jw9P/orsa013aL7h8irF5JkR3rD5j4+US8r+D8b4P/IEKg5Bfsw2iWUHrfzXeXd6zcheLGtpQg2o=
X-Received: by 2002:a5d:9598:: with SMTP id a24mr3651142ioo.182.1590693651702; Thu, 28 May 2020 12:20:51 -0700 (PDT)
MIME-Version: 1.0
From: Ashutosh Srivastava <as12738@nyu.edu>
Date: Thu, 28 May 2020 15:20:40 -0400
Message-ID: <CAJyCXaYXbMwrejcNTLv6bxSmf3L0hGekqF4Ddui8=yxba2b-iA@mail.gmail.com>
To: tcpprague@ietf.org
Content-Type: multipart/related; boundary="0000000000000ea96005a6ba3ad7"
X-Orig-IP: 209.85.166.197
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 spamscore=0 priorityscore=1501 impostorscore=0 lowpriorityscore=0 malwarescore=0 suspectscore=0 cotscore=-2147483648 clxscore=1015 phishscore=0 adultscore=0 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2004280000 definitions=main-2005280127
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpprague/y1Sli4N_1cvaGk1BxpA3ISK4_G8>
Subject: [tcpPrague] TCP Prague under-utilising capacity when RTT scaling turned on
X-BeenThere: tcpprague@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." <tcpprague.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpprague>, <mailto:tcpprague-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpprague/>
List-Post: <mailto:tcpprague@ietf.org>
List-Help: <mailto:tcpprague-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpprague>, <mailto:tcpprague-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 28 May 2020 19:20:57 -0000

Hi everyone,

This is about an issue I found with the recently added RTT scaling feature
of TCP Prague linux kernel implementation. This was first observed with
commit number 7f267bd
<https://github.com/L4STeam/linux/commit/7f267bd591c249697f63f81b21a1755202e86cc2#diff-38ce93325583f02d790276f5cafd1c42>
(
Mar 16, 2020 ) where RTT scaling was turned on by default and also with
future commits when we explicitly enable RTT scaling.

The plot below shows the throughput of a single TCP Prague flow. Although
the available capacity was 1Gbps, TCP Prague is not able to reach that
point anytime during the experiment.
[image: Screen Shot 2020-05-28 at 3.05.57 PM.png]
The experiment settings were as follows

   - This experiment was done on the Cloudlab
<https://www.cloudlab.us/> testbed
   with a 3-node topology ( source, router, receiver).
   - The bottleneck between the router and receiver was a 1 Gbps wired link
   ( 10 Gig interfaces , capacity restricted to 1Gbps using linux traffic
   shaping tools (tc) ).
   - The flow was sent using iperf3.
   - The AQM at the router was a FQ qdisc with a single bucket and was
   marking packets with ECN at a marking threshold of 5 ms. You can use the
   following parameters with the tc-fq qdisc to replicate this setting :  fq
   limit 5000p flow_limit 5000p orphan_mask 0 ce_threshold 5ms
   - The "ECN fallback on detection of classic ECN AQM" feature of TCP
   Prague were disabled for this set of experiments.
   - The propagation / base delay of the setup was very low ( around 0.4 ms
   ) and no delay was added on top.

If interested, you can look at the ss data plots for these experiments at :
https://drive.google.com/drive/folders/1O6uEngxrDX5ipY71sjqr36lXCQBoZV--?usp=sharing

Looking forward to your comments and questions regarding these results.

Thank you,

Ashutosh Srivastava
First year PhD student,
Department of Electrical and Computer Engineering
NYU Tandon School of Engineering