[tsvwg] FCT tests of Prague in shared RFC3168 queues

Pete Heist <pete@heistp.net> Wed, 10 March 2021 12:52 UTC

Return-Path: <pete@heistp.net>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 10F773A0A1E for <tsvwg@ietfa.amsl.com>; Wed, 10 Mar 2021 04:52:34 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.967
X-Spam-Level:
X-Spam-Status: No, score=-1.967 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, URI_DOTEDU=0.132] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=heistp.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ois51Bp_tNk3 for <tsvwg@ietfa.amsl.com>; Wed, 10 Mar 2021 04:52:32 -0800 (PST)
Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 084D33A09F6 for <tsvwg@ietf.org>; Wed, 10 Mar 2021 04:52:31 -0800 (PST)
Received: by mail-wm1-x331.google.com with SMTP id r15-20020a05600c35cfb029010e639ca09eso10953290wmq.1 for <tsvwg@ietf.org>; Wed, 10 Mar 2021 04:52:31 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=heistp.net; s=google; h=message-id:subject:from:to:date:user-agent:mime-version :content-transfer-encoding; bh=McE+KDUyp+VAZWgqZh4c2GYiG/kGAo/FN9sJBB6RI6A=; b=IAyUJs8dh8Pg2OENkAzQPoHzz0MHF4IVx99kC/WLDSbAH86lGkvGBIPYqR1pO64kF4 ISBbpSm58AZokovazYBUIo+hg5mCA9RVBrJk/m+0MM+5Ak8j1NSChVe15ZcWY/DG/HXE AWruw9lkVcpRgtr67Hdvnj9QMtB2pzFNGyfDNTaWFzX7NqcldZX9n6OJ81Df4lByFswL e6khiVjXvHWIXJgxC90e9SXv1ckK2geFXwdCm7/P/c7loWohWkqeImJHmTuuztqzFz/d Dzt7bcrClQzkUy7Wz9HEHvdmR2DBAbE8jaD0NyifOQULHKyqESqjSkJy/OyafErkWMW6 PkrQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:date:user-agent :mime-version:content-transfer-encoding; bh=McE+KDUyp+VAZWgqZh4c2GYiG/kGAo/FN9sJBB6RI6A=; b=bQePPxkaVPyd+ozS5B1qQAiUVQrrCiAQIfjhIEZCmR2dS9TkiV3riycSTroe24eEY0 6w2BBKIyTzzWyxMmrS43R7ft8H+i6g+wOKMhQziK0dKJwUjYPit8JBt86WsGToIZDqo3 WX77RWxq0VVATtRwcwtsBKjbe9n07KCG2JTXeStZrTboK77QTSTguNbtBpPIOsgWyJ8T ZaCrrq0mP8vEuUVOCqWF/jsTtfeDt6VfSPDoM4rID+Pg86leZjBVfnOsqt+AnVRuJs1H UMSLPLxNmEk5gnVRAOPto3GQdy9niFh/9Zhn5lRAMsus94Gv2tk9SHAs3FqZ7E1G5UI+ 4TAQ==
X-Gm-Message-State: AOAM531YBy6jah8CssqhIFYe46tfN/TzGimDaQYtscn4bkk4YozZCmKH OuQue9OVHGS6mXFn+bmG1vxVG7mK1V/QXQ==
X-Google-Smtp-Source: ABdhPJy06usuemC6eR/lqh53fkEv8eTw/FzG0v7Rup7pNgXzZvpEALu4KYlalJnYDHf4ag4CIo2ywA==
X-Received: by 2002:a7b:cdf7:: with SMTP id p23mr3295226wmj.26.1615380747356; Wed, 10 Mar 2021 04:52:27 -0800 (PST)
Received: from [10.72.0.88] (h-1169.lbcfree.net. [185.193.85.130]) by smtp.gmail.com with ESMTPSA id c35sm8276680wmp.3.2021.03.10.04.52.26 for <tsvwg@ietf.org> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Mar 2021 04:52:27 -0800 (PST)
Message-ID: <56ba28929ef26015e6c1a5e9798f5403622036d8.camel@heistp.net>
From: Pete Heist <pete@heistp.net>
To: tsvwg IETF list <tsvwg@ietf.org>
Date: Wed, 10 Mar 2021 13:52:26 +0100
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.38.4
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/ICN0ZaxSe4NtAfzyZYLXWoVPIxA>
Subject: [tsvwg] FCT tests of Prague in shared RFC3168 queues
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 10 Mar 2021 12:52:34 -0000

We've run tests of flow completion time (FCT) for CUBIC flows, in
competition with single CUBIC or Prague flows in a shared RFC3168
queue. The results are in the l4s-tests repo, and below.

https://github.com/heistp/l4s-tests/#harm-to-flow-completion-time

TL;DR
In shared RFC3168 queues with the chosen parameters, median and P95 FCT
increase significantly across RTTs with Prague as the competition,
compared to CUBIC.

Flow arrival times use an exponential distribution, and flow lengths a
lognormal distribution, with the parameters below. "Harm" calculations
(https://www.cs.cmu.edu/~rware/assets/pdf/ware-hotnets19.pdf) are added
for each metric. The results are just text, so I'll include them here.


Test Parameters:
----------------
CCAs under test:  cubic, prague
RTTs:             10ms, 20ms, 40ms, 80ms, 160ms
Bandwidth:        50Mbps
Qdisc:            fq_codel flows 1
Slow start delay: 20s

FCT Workload Parameters:
------------------------
Server URL:        localhost:8188
CCA:               cubic
Duration:          3m0s
Flows:             900
Mean arrival time: 200ms
Est. bandwidth:    25.83Mbps
Flow lengths (lognormal distribution):      
|- P5:             65536
|- Mean:           645683
|- P95:            2097152

RTT    CCA     GeoMean (Harm)    Median (Harm)     P95 (Harm)
---    ---     --------------    -------------     ----------
10ms   -       176.2ms           154.3ms           1061.3ms
10ms   cubic   339.3ms (0.481)   314.9ms (0.510)   2002.7ms (0.470)
10ms   prague  1171.3ms (0.850)  1131.7ms (0.864)  7305.2ms (0.855)
20ms   -       243.0ms           211.5ms           1155.3ms
20ms   cubic   376.6ms (0.355)   351.7ms (0.399)   1986.0ms (0.418)
20ms   prague  1394.1ms (0.826)  1385.3ms (0.847)  8454.8ms (0.863)
40ms   -       369.2ms           317.8ms           1634.7ms
40ms   cubic   475.7ms (0.224)   423.9ms (0.250)   2179.9ms (0.250)
40ms   prague  1643.8ms (0.775)  1637.7ms (0.806)  9879.3ms (0.835)
80ms   -       599.9ms           554.5ms           2135.6ms
80ms   cubic   906.7ms (0.338)   846.9ms (0.345)   3709.4ms (0.424)
80ms   prague  2384.8ms (0.748)  2228.4ms (0.751)  12159.9ms (0.824)
160ms  -       976.6ms           976.4ms           3465.1ms
160ms  cubic   1127.7ms (0.134)  1043.2ms (0.064)  4207.8ms (0.177)
160ms  prague  2471.9ms (0.605)  2174.7ms (0.551)  14418.8ms (0.760)