[tsvwg] sce vs l4s comparison plots?

Dave Taht <dave@taht.net> Sat, 09 November 2019 23:06 UTC

Return-Path: <dave@taht.net>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id 5AC32120120; Sat, 9 Nov 2019 15:06:02 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id XLBG-XzxzgdU; Sat, 9 Nov 2019 15:06:00 -0800 (PST)
Received: from mail.taht.net (mail.taht.net []) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 362D512006B; Sat, 9 Nov 2019 15:06:00 -0800 (PST)
Received: from dancer.taht.net (c-73-162-29-198.hsd1.ca.comcast.net []) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.taht.net (Postfix) with ESMTPSA id 6314D21425; Sat, 9 Nov 2019 23:05:57 +0000 (UTC)
From: Dave Taht <dave@taht.net>
To: tsvwg IETF list <tsvwg@ietf.org>, tcpm@ietf.org
References: <742142FB-6233-4048-931B-EE2DD9024454@gmx.de>
Date: Sat, 09 Nov 2019 15:05:54 -0800
In-Reply-To: <742142FB-6233-4048-931B-EE2DD9024454@gmx.de> (Sebastian Moeller's message of "Sat, 9 Nov 2019 23:32:09 +0100")
Message-ID: <87mud4ejl9.fsf@taht.net>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/hNFKzsh50bwXOH_HnxdzwCUdoz8>
Subject: [tsvwg] sce vs l4s comparison plots?
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 09 Nov 2019 23:06:02 -0000

It is really wonderful to have these two groups using the
flent tool and producing comparable data on the same test suites in
a few cases now, and the testing so far found a couple bugs.


I do feel rather strongly that any major change to how the internet does
congestion control requires a maximum of publically repeatable
benchmarks using open, shared source code. [1]

I do keep hoping for a rrul test on a 15 to 20x1 asymmetric network
simulation like how most cablemodems and dsl boxes are configured, and
more of the rtt_fair_* tests, and what the heck - the tcp_square_wave
test, and any of a dozen others. tcp_nup with 32 flows... heck, hundreds
of flows... go to town! Burn some cycles!

A few packet caps would be nice to have also, to look
over relative mark rates in tcptrace and scetrace.

Anyway... l4steam....

In looking over the l4s.cablelabs site today though, I see the data is
only available as .png files, and that the associated flent.gz data
is not apparently being published.

.svgs help. Flent can also be run at a higher sample rate... but

There are a few things about having *.flent.gz files available that
would help us look at some things in more detail. Me, I tend to look at
comparison cdf plots more than anything else, and secondly bar plots,
and lacking that it's hard to make a distinction betwen a whole bunch of
tiny lines at the bottom about overall queue depths and latency
distributions, among other things, without squinting, and doing tricks
in photoshop to make the two or more datasets "line up".

(there's dozens of plot types in the flent tool, and comparison plots
are a snap)

By default (when run without -x) flent captures very little metadata
about the system it is run on (IP addresses, a couple sysctls, and
qdiscs) but it's helpful to have. One example that would be in that
metadata, is that I'm unsure if the ns3 data is using an IW4 or IW10?

It sounds like you are rate limiting with htb? (to what quantum?)

Another example, in more "native" environments running at a simulated
line rate, BQL is quite important to have in the simulation
also. there's been a couple papers published on BQL's benefits vs a raw
txring, thus far, there's a good plot of what it does in fig 6 of:



So far as I know ns(X) does not correctly simulate GSO/TSO even when
run in DCE mode, but I could be out of date on that. TBF (and cake)
do break apart superpackets, htb (+ anything, like fq_codel or dualq)
do not.

I also have completely forgotten at this point, to what extent the
ns3 model of fq_codel matches the deployed reality. It was such a long
time ago.... as best as I recall ns3 didn't even have ecn support then!

I'm taking apart
as I write.

thx for the data!

[1] If anyone hasn't read my related rant yet, it's here: