Re: [tcpPrague] TCP Prague under-utilising capacity when RTT scaling turned on
Ashutosh Srivastava <as12738@nyu.edu> Wed, 17 June 2020 03:10 UTC
Return-Path: <as12738@nyu.edu>
X-Original-To: tcpprague@ietfa.amsl.com
Delivered-To: tcpprague@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 08D9B3A0DF0 for <tcpprague@ietfa.amsl.com>; Tue, 16 Jun 2020 20:10:23 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.099
X-Spam-Level:
X-Spam-Status: No, score=-2.099 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=nyu.edu header.b=lbcgM+oA; dkim=pass (2048-bit key) header.d=nyu-edu.20150623.gappssmtp.com header.b=jpc8BYxm
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id s16I-hTURaXk for <tcpprague@ietfa.amsl.com>; Tue, 16 Jun 2020 20:10:21 -0700 (PDT)
Received: from mx0b-00256a01.pphosted.com (mx0b-00256a01.pphosted.com [67.231.153.242]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4BEBE3A0D0F for <tcpprague@ietf.org>; Tue, 16 Jun 2020 20:10:21 -0700 (PDT)
Received: from pps.filterd (m0119691.ppops.net [127.0.0.1]) by mx0b-00256a01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 05H3AK2l057498 for <tcpprague@ietf.org>; Tue, 16 Jun 2020 23:10:20 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nyu.edu; h=mime-version : references : in-reply-to : from : date : message-id : subject : to : content-type; s=20180315; bh=H5rB0huhLu7F5RWFTl+/ju7UCTO9+5SIRxX/i5Hnoh4=; b=lbcgM+oACPXsmCcYqhpMuOv7WIlryAatGoLFLAOTylOD6dQYlUF+7IYmPotVLFev6LTv joccVVxh9vjrC2log5N8ia7vz7ITE/M7AGI2ze5Y1WFThZd+v9SMghLkvdMD4k2/9ziH amInoa75Kd4iurM2L0C0gyojY4AVm++YSzy3k8ICZ7N1JI9PUThr6IBIJ/6GmhTlG0Qn BBV4zVcFagEcZWgsgD1DuHLiScmhDOLVEFNEmqPCjPX/RcUbbFwz97uVX6ZkFJpCk/I/ MXU8OpaiS2PN6Mbl3kiti4e6CGwF0wnL+4gCfU3+WDhsngw2m6NHWdcxSyz0Y1bj7v0o Fw==
Received: from mail-io1-f71.google.com (mail-io1-f71.google.com [209.85.166.71]) by mx0b-00256a01.pphosted.com with ESMTP id 31q64ju4vg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for <tcpprague@ietf.org>; Tue, 16 Jun 2020 23:10:20 -0400
Received: by mail-io1-f71.google.com with SMTP id b30so847657ioc.8 for <tcpprague@ietf.org>; Tue, 16 Jun 2020 20:10:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nyu-edu.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=H5rB0huhLu7F5RWFTl+/ju7UCTO9+5SIRxX/i5Hnoh4=; b=jpc8BYxmuBO8ipEW+95yTuCiIO8wxwnMmsFX9rUDhfS131Vfov5ob4uCYTj03B+8l4 ha4vRAML3ca4FfDbrR/wURn/yik0RpX3Uxe4p9P2oWxwdySCYo5Mt2YCt5aj/4o9ZHcy jYFAsHB/GJyJBH1Dzxu9lyX8YGAQAsDk8S+TQUAESYflqaNKbG+ADR0Q4NGjp47VOZ9P IBgZQ3WCv0xswCzWMGAgAI8n33N2wNqxIZmBJ7lzyj7QtmPyuxGWtZa17NIsvORWvmtC A0g+LzqHJfQ45jbF8H2SclZMkq/vMx645VtJYhXOENb2n73zA4/fVz8uci4hEzIltRxr Y9ug==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=H5rB0huhLu7F5RWFTl+/ju7UCTO9+5SIRxX/i5Hnoh4=; b=NXzitc0LhDJYVJgXShCj8xFgQx4EIke6xZ1YoKl5UunCCnpwpC8p6/HnrtGHZfomeG DoO7+DxTf6FAiCAIgJ0Hi04ZLsCAKAVsju+zTakMFjF55S/pKJdeG6Fp/migK/fbjdKc uTYY/eZYCeetGTqy7z+pzSOsJv5J7VtJoXKEws1cSGKAh4jnVgiEjmXebGIdvkIt08NK bFZQSDYjsi0xT33hMM3vsvcrIZPCohV1G/4Jnj8ZIpmt/9PXPM3lziMR79lt5t73+iub +jK37GTSO4U+sA5ZQ7s6HQLoQO8gf7IFYvu3CN/8vZSBeqQkNi7GPVhBT7oSewlgITf2 CfrQ==
X-Gm-Message-State: AOAM530DAdS7yoL2kGUn+ONG2qs8dZNUFdbrtFz7uWNPMdrnSpQIFEAk LFY0gwQh7IKCryH/fiLtRS0rIBmDatAjIl9/hDEkYRfliJDkbYgB7LWnw3LwZi3qAX3X4By4f6j qKbdXtJJ1mtD4jaBhtP1vWOuQtg==
X-Received: by 2002:a02:380d:: with SMTP id b13mr28770126jaa.107.1592363418843; Tue, 16 Jun 2020 20:10:18 -0700 (PDT)
X-Google-Smtp-Source: ABdhPJxoOGrl8zc8Z79mgBiyNj1V7KUlAF9/8bcfhXkaO16jlrwnZQKLl8SbJKsHSgpqcK3b3+RgqIZQL+6GGgvtQZc=
X-Received: by 2002:a02:380d:: with SMTP id b13mr28770104jaa.107.1592363418393; Tue, 16 Jun 2020 20:10:18 -0700 (PDT)
MIME-Version: 1.0
References: <CAJyCXaYXbMwrejcNTLv6bxSmf3L0hGekqF4Ddui8=yxba2b-iA@mail.gmail.com> <AM0PR07MB46414BF56843D476FCB4FA12E0830@AM0PR07MB4641.eurprd07.prod.outlook.com>
In-Reply-To: <AM0PR07MB46414BF56843D476FCB4FA12E0830@AM0PR07MB4641.eurprd07.prod.outlook.com>
From: Ashutosh Srivastava <as12738@nyu.edu>
Date: Tue, 16 Jun 2020 23:10:07 -0400
Message-ID: <CAJyCXabTZpi7zRw9erdUtJUpuSSs4Ho8MiSUNT_0u=Y_3yD9PQ@mail.gmail.com>
To: "Tilmans, Olivier (Nokia - BE/Antwerp)" <olivier.tilmans@nokia-bell-labs.com>, tcpprague@ietf.org
Content-Type: multipart/alternative; boundary="000000000000e855ef05a83effd4"
X-Orig-IP: 209.85.166.71
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 mlxlogscore=999 spamscore=0 suspectscore=0 mlxscore=0 adultscore=0 bulkscore=0 clxscore=1015 impostorscore=0 cotscore=-2147483648 malwarescore=0 priorityscore=1501 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2004280000 definitions=main-2006170025
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpprague/qOs2Dgga3_fiemL12Ai_Vrx_p-A>
Subject: Re: [tcpPrague] TCP Prague under-utilising capacity when RTT scaling turned on
X-BeenThere: tcpprague@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." <tcpprague.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpprague>, <mailto:tcpprague-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpprague/>
List-Post: <mailto:tcpprague@ietf.org>
List-Help: <mailto:tcpprague-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpprague>, <mailto:tcpprague-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 17 Jun 2020 03:10:23 -0000
Hi Olivier, > > Could you confirm that this is also happening with the tip of the current > Branch? I.e., 3b63cc0 is the "future commits" you are referring to above? > Yes, this is also happening with the tip of the current branch. > > In addition to the ss plots, could you: > - Confirm that this happens with both the internal TCP pacing and the fq > qdisc as pacer on the data sender > Yes, this happens with both internal TCP pacing and with the fq qdisc as pacer at the sender. > - Confirm that this happens both with and without gro/gso/tso/lro on the > endhosts (data sender and receiver, and also the aqm node as fq does not do > gro splitting prior to enqueue)? > There is still underutilisation after I turn off offloading (gro/gso/tso) on all interfaces on all hosts but it does not look as bad as earlier. - Confirm that the problem persists if you increase the base RTT--a simple > netem qdisc on the reverse path to add 2-5ms should be sufficient, do not > forget to disable gro/gso as that poorly interacts with netem. > At higher RTTs, there is some underutilisation at the beginning but TCP Prague slowly reaches full bandwidth and operates as expected thereafter. - Log the reported CE marks by the AQM, as well as the > delivered_ce/received_ce counters throughout the experiment on the data > sender/receiver? > I do not observe any CE marks / delivered_ce packets as the link always remains underutilised. - Log eventual drops (dropped counter on AQM, and retransmission > counters on the data sender). > > I have uploaded additional data for experiments with commit 3b63cc0 on the google drive I shared earlier : https://drive.google.com/drive/folders/1O6uEngxrDX5ipY71sjqr36lXCQBoZV--?usp=sharing The columns of the AQM csv file are : timestamp, dropped packets, queue backlog (bytes), queue backlog (packets) Columns of the ss.csv file at the sender are : timestamp, ID, cwnd, SRTT (ms), delivered, delivered_ce, pacing rate (Mbps) I have also added the raw data files. The experiment settings were the same as in my last mail. Thank you, Ashutosh.
- [tcpPrague] TCP Prague under-utilising capacity w… Ashutosh Srivastava
- Re: [tcpPrague] TCP Prague under-utilising capaci… Tilmans, Olivier (Nokia - BE/Antwerp)
- Re: [tcpPrague] TCP Prague under-utilising capaci… Ashutosh Srivastava