Re: [tsvwg] [tcpm] sce vs l4s comparison plots?

["Rodney W. Grimes" <[e-mail]>]

Dave,
	Thank you for the links, some short responses inline.

Regards,
Rod

> 
> It is really wonderful to have these two groups using the
> flent tool and producing comparable data on the same test suites in
> a few cases now, and the testing so far found a couple bugs.
> 
> https://github.com/heistp/sce-l4s-bakeoff
> https://l4s.cablelabs.com/
> 
> I do feel rather strongly that any major change to how the internet does
> congestion control requires a maximum of publically repeatable
> benchmarks using open, shared source code. [1]
> 
> I do keep hoping for a rrul test on a 15 to 20x1 asymmetric network
> simulation like how most cablemodems and dsl boxes are configured, and
> more of the rtt_fair_* tests, and what the heck - the tcp_square_wave
> test, and any of a dozen others. tcp_nup with 32 flows... heck, hundreds
> of flows... go to town! Burn some cycles!

I believe Pete has some work scheduled for the (near?) future
in this area.

> 
> A few packet caps would be nice to have also, to look
> over relative mark rates in tcptrace and scetrace.
I do believe all you seek for Petes data is there,
https://github.com/heistp/sce-l4s-bakeoff/#test-output
specifically items:
*.flent.gz- standard Flent output file, may be used to view original data or re-generate plots
*.tcpdump_*.pcap.bz2- pcap file compressed with bzip2 -9

> 
> Anyway... l4steam....
> 
> In looking over the l4s.cablelabs site today though, I see the data is
> only available as .png files, and that the associated flent.gz data
> is not apparently being published.
> 
> ..svgs help. Flent can also be run at a higher sample rate... but
> 
> There are a few things about having *.flent.gz files available that
> would help us look at some things in more detail. Me, I tend to look at
> comparison cdf plots more than anything else, and secondly bar plots,
> and lacking that it's hard to make a distinction betwen a whole bunch of
> tiny lines at the bottom about overall queue depths and latency
> distributions, among other things, without squinting, and doing tricks
> in photoshop to make the two or more datasets "line up".
> 
> (there's dozens of plot types in the flent tool, and comparison plots
> are a snap)
> 
> By default (when run without -x) flent captures very little metadata
> about the system it is run on (IP addresses, a couple sysctls, and
> qdiscs) but it's helpful to have. One example that would be in that
> metadata, is that I'm unsure if the ns3 data is using an IW4 or IW10?
> 
> It sounds like you are rate limiting with htb? (to what quantum?)
> 
> Another example, in more "native" environments running at a simulated
> line rate, BQL is quite important to have in the simulation
> also. there's been a couple papers published on BQL's benefits vs a raw
> txring, thus far, there's a good plot of what it does in fig 6 of:
> 
> http://sci-hub.tw/10.1109/LANMAN.2019.8847054
> 
> Lastly...
> 
> So far as I know ns(X) does not correctly simulate GSO/TSO even when
> run in DCE mode, but I could be out of date on that. TBF (and cake)
> do break apart superpackets, htb (+ anything, like fq_codel or dualq)
> do not.
> 
> I also have completely forgotten at this point, to what extent the
> ns3 model of fq_codel matches the deployed reality. It was such a long
> time ago.... as best as I recall ns3 didn't even have ecn support then!

And apparently is still missing some ECN (RFC3168) support as your
following link contains some patches:
	6.  cd back one level to the top-level ns-3 directory,
	and patch the ns-3.30.1 release with a patch provided
	in the l4s-evaluation/patches/ directory. This patch
	adds support for ECE handling and DCTCP in the base
	TCP code, and adds ECN handling to CoDel and FqCoDel models.

> 
> I'm taking apart
> https://gitlab.com/tomhend/modules/l4s-evaluation/blob/master/README.md
> as I write.
> 
> thx for the data!
> 
> [1] If anyone hasn't read my related rant yet, it's here:
> https://conferences.sigcomm.org/sigcomm/2014/doc/slides/137.pdf
> 
> _______________________________________________
> tcpm mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/tcpm
> 

-- 
Rod Grimes                                                 [e-mail]