Re: [aqm] [iccrg] Comments on draft-irtf-iccrg-tcpeval

[Dave Taht <[e-mail]>]

adding aqm

On Tue, Dec 9, 2014 at 8:45 AM, Toke Høiland-Jørgensen <[e-mail]> wrote:
> At the Hawaii meeting I promised to review this document. Here are my
> comments:
>
> - Overall I think it is quite good. The document provides a nice
>   baseline for evaluating TCP behaviour, and having an implementation
>   available is definitely a plus. I'm not sufficiently familiar with NS2
>   and Tmix to comment on the implementation details (and in particular
>   on whether or not they achieve the stated goals of mixing and quickly
>   attaining steady state), so I'm just going to assume that that is the
>   case. :) An implementation that runs on real machines would be nice.

+10. Jeeze, guys, the internet stopped looking like sims a couple
decades back....

> - I note there is some overlap between this and other efforts to produce
>   test suites. I'm aware of at least the AQM and the BMWG groups'
>   efforts. Pursuing deduplication of effort might be worthwhile.

Well, additive would be a nicer outcome.

>   Cross-referencing relevant documents might also be. Not sure I have
>   enough of an overview of this to recommend particular things to refer
>   to / incorporate / etc, so I won't.
>
> - I'm not sure it would be possible to perform the tests specified in
>   the document without using the trace files (but see above regarding my
>   lack of experience with the tools). I'm not sure if this is a problem
>   or not (but could not find any information about the licensing of the
>   trace files either); if it is a problem, perhaps providing, say, a
>   table of TCP flow distributions to use as a fall-back, or something
>   like that, would be worthwhile?

I would argue for a large set of trace files generated yearly from
multiple vantage points.

>
> - Several of the tests specify that they should be run "with and
>   without" the TCP feature under test. Section 6.1 specifies SACK but no
>   ECN as a baseline for that particular test; why is it only for that
>   test? Explicitly specifying a baseline for the whole thing might be a
>   good idea. I'd argue that such a baseline should at least encourage to
>   also test against CUBIC (but will freely admit to being biased towards
>   Linux environments).

I think timestamps are well over 70% of all tcp traffic now. Sack is
way up, also. ECN can be negotiated with 60+% of the alexa top 1m...

I am not sure cubic is well defined anymore. Notably the addition of
sch_fq + pacing + changes to the hystart algorithm in linux have made
it a whole new ballgame, and there are too many other changes to linux
tcp in the last 4 years in particular to list here.

> - I applaud the inclusion of an asymmetrical link in one of the
>   scenarios (the satellite link in section 4.4.4). However, I'm a little
>   concerned that effects in that test are going to be dominated by the
>   long RTT, and so would like to see an asymmetrical link scenario with
>   lower RTT (in the "earth surface internet" range, so ~100 ms or less).
>   I've definitely seen weird things happen on asymmetrical links.

Well, as I am now commonly seeing asymmetric links deployed in the
field with a 10x1 down/up ratio (examples include many DSL setups, and
my new 120/12mbit cable connection), I would like to see this emulated
and explored a lot further.

Also wifi is commonly asymmetric (antennas on the station are much
worse than those on the AP, stations are in a "taxi stand" topology.

> - As far as metrics is concerned, I'd argue for emphasising
>   distributions more. Meaning that for the steady-state tests (section
>   4.5), the variance measures should not in "other metrics of general
>   interest" but be part of the core metric.

+1

>   For the transient tests, and for latency measures in particular, this
>   becomes even more important. In my view the phrase "ideally it would
>   be better to have more complete statistics" (section 5.1.3) is quite
>   the understatement: It is quite essential to include the distribution
>   characteristics, and in particular the outliers and the duration of
>   spikes in latency.

+10. Also the Tracy-widom distribution has become rather interesting
of late as a means to measure transitional behavior.

...

One of the things that bother me about most published simulations is
that they tend to run at low rates, where today we have networks going
at 10GigE+. A lot of papers focus overmuch on packet loss at low rates
below 5mbit, (where indeed it tends to be high with modern tcps) where
at higher rates minimal packet loss is generally observed, and
desirable.

Furthermore packet loss of certain kinds of packets (acks) tend to be
inconsequential and should be noted somehow. We can drop a lot more
acks than we do....

In looking over my current netperf-wrapper data set (120mbit/12mbit)

http://snapon.lab.bufferbloat.net/~d/comprehensive_with_ecn.tgz

I was struck by the low packet loss rates at these speeds, and by the
differences in average packet size. (cake2 is an all
singing/all_dancing fq_codel based shaper)

What other statistics could I be gathering from the real world that
would be useful here?

Inbound packet loss percentage: .02%
Average packet size: 2823 (offloads are in use) (so this percentage is
relative to the actual packet size to some extent)
Details:

qdisc cake2 801e: root refcnt 2 bandwidth 115Mbit besteffort flows
 Sent 861811656 bytes 612322 pkt (dropped 89, overlimits 556131 requeues 0)
 backlog 0b 0p requeues 0

          Class 0
  rate     115Mbit
  target     6.2ms
  delay        0us
  maxpkt         0
  pkts      305338
  bytes  862069036
  drops         89
  marks          0

...

Outbound packet loss percentage: .3%
Average packet size: 403 (offloads are in use)

Details:

qdisc cake2 801d: root refcnt 9 bandwidth 12Mbit besteffort flows
 Sent 90295734 bytes 252230 pkt (dropped 922, overlimits 434160 requeues 1)
 backlog 0b 0p requeues 1
          Class 0
  rate      12Mbit
  target     6.2ms
  delay        0us
  maxpkt         0
  pkts      230721
  bytes   93019420
  drops        922
  marks          0


>
> - Section 5.1 mentions AQM algorithms in passing. Since the work of the

Same test, with ecn. zero packet loss, near perfect utilization and
low latency, but honestly we could shoot a lot more acks.

qdisc cake2 8012: root refcnt 2 bandwidth 115Mbit besteffort flows
 Sent 861639003 bytes 610822 pkt (dropped 0, overlimits 534608 requeues 0)
 backlog 0b 0p requeues 0

  Class 0
  rate     115Mbit
  target     6.2ms
  delay        0us
  maxpkt         0
  pkts      292907
  bytes  861639003
  drops          0
  marks         87

qdisc cake2 8011: root refcnt 9 bandwidth 12Mbit besteffort flows
 Sent 90205372 bytes 246249 pkt (dropped 0, overlimits 419887 requeues 0)
 backlog 0b 0p requeues 0
          Class 0
  rate      12Mbit
  target     6.2ms
  delay        0us
  maxpkt         0
  pkts      222494
  bytes   90205372
  drops          0
  marks        923


>   AQM working group has progressed somewhat now, I believe it would be a
>   good idea to upgrade this to at the very least *recommend* testing the
>   behaviour when AQM and fairness queueing algorithms are introduced on
>   the (bottleneck) links.

:)

>
> -Toke
>
> _______________________________________________
> iccrg mailing list
> [e-mail]
> https://www.irtf.org/mailman/listinfo/iccrg



-- 
Dave Täht

thttp://www.bufferbloat.net/projects/bloat/wiki/Upcoming_Talks