[ippm] Fwd: How should capacity measurement interact with shaping?

[Matt Mathis <[e-mail]>]

Ok, moving the thread to IPPM

Some background, we (Measurement Lab) are testing a new transport (TCP)
performance measurement tool, based on BBR-TCP.   I'm not ready to talk
about results yet (well ok, it looks pretty good).    (BTW the BBR
algorithm just happens to resemble the algorithm described
in draft-morton-ippm-capcity-metric-method-00.)

Anyhow we noticed some interesting performance features for number of ISPs
in the US and Europe and I wanted to get some input for how these cases
should be treated.

One data point, a single trace saw ~94.5 Mbit/s for ~4 seconds,
fluctuating performance ~75 Mb/s for ~1 second and then stable
performance at ~83Mb/s for the rest of the 10 second test.    If I were to
guess this is probably a policer (shaper?) with a 1 MB token bucket and
a ~83Mb/s token rate (these numbers are not corrected for header overheads,
which actually matter with this tool).  What is weird about it is that
different ingress interfaces to the ISP (peers or serving locations)
exhibit different parameters.

Now the IPPM measurement question:   Is the bulk transport capacity of this
link ~94.5 Mbit/s or ~83Mb/s?   Justify your answer....?

Thanks,
--MM--
The best way to predict the future is to create it.  - Alan Kay

We must not tolerate intolerance;
       however our response must be carefully measured:
            too strong would be hypocritical and risks spiraling out of
control;
            too weak risks being mistaken for tacit approval.

Forwarded Conversation
Subject: How should capacity measurement interact with shaping?
------------------------

From: Matt Mathis <[e-mail]>
Date: Thu, Aug 15, 2019 at 8:55 AM
To: MORTON, ALFRED C (AL) <[e-mail]>


We are seeing shapers  with huge bucket sizes, perhaps as larger or larger
than 100 MB.

These are prohibitive to test by default, but can have a huge impact in
some common situations.  E.g. downloading software updates.

An unconditional pass is not good, because some buckets are small.  What
counts as large enough to be ok, and what "derating" is ok?

Thanks,
--MM--
The best way to predict the future is to create it.  - Alan Kay

We must not tolerate intolerance;
       however our response must be carefully measured:
            too strong would be hypocritical and risks spiraling out of
control;
            too weak risks being mistaken for tacit approval.


----------
From: MORTON, ALFRED C (AL) <[e-mail]>
Date: Mon, Aug 19, 2019 at 5:08 AM
To: Matt Mathis <[e-mail]>
Cc: CIAVATTONE, LEN <[e-mail]>, [e-mail] <
[e-mail]>


Hi Matt, currently cruising between Crete and Malta,

with about 7 days of vacation remaining – Adding my friend Len.

You know Rüdiger. It appears I’ve forgotten how to typs in 2 weeks

given the number of typos I’ve fixed so far...



We’ve seen big buffers on a basic DOCSIS cable service (downlink >2 sec)

but,

  we have 1-way delay variation or RTT variation limits

  when searching for the max rate, that don’t many packets

  queue in the buffer



  we want the status messages that result in rate adjustment to return

 in a reasonable amount of time (50ms + RTT)



  we usually search for 10 seconds, but if we go back and test with

  a fixed rate, we can see the buffer growing if the rate is too high.



  There will eventually be a discussion on the thresholds we use

  in the search // load rate control algorithm. The copy of

  Y.1540 I sent you has a simple one, we moved beyond that now

  (see the slides I didn’t get to present at IETF).



  There is value in having some of this discussion on IPPM-list,

  so we get some **agenda time at IETF-106**



We measure rate and performance, with some performance limits

built-in.  Pass/Fail is another step, de-rating too (made sense

with MBM “target_rate”).



Al


----------
From: <[e-mail]>
Date: Mon, Aug 26, 2019 at 12:05 AM
To: <[e-mail]>
Cc: <[e-mail]>, <[e-mail]>


Hi Al,



thanks for keeping me involved. I don’t have a precise answer and doubt,
that there will be a single universal truth.



If the aim is only to determine the IP bandwidth of an access, then we
aren’t interested in filling a buffer. Buffering events may occur, some of
which are useful and to be expected, whereas others are not desired:



   - Sender shaping behavior may matter (is traffic at the source CBR or is
   it bursty)
   - Random collisions should be tolerated at the access whose bandwidth is
   to be measured.
   - Limiting packet drop due to buffer overflow is a design aim or an
   important part of the algorithm, I think.
   - Shared media might create bursts. I’m not an expert in the area, but
   there’s an “is bandwidth available” check in some cases between a central
   sender using a shared medium and the receivers connected. WiFi and may be
   other wireless equipment buffers packets also to optimize wireless resource
   optimization.
   - It might be an idea to mark some flows by ECN, once there’s a guess on
   a sending bitrate when to expect no or very little packet drop. Today, this
   is experimental. CE marks by an ECN capable device should be expected
   roughly once queuing starts.



Practically, the set-up should be configurable with commodity hard- and
software and all metrics should be measurable at the receiver. Burstiness
of traffic and a distinction between queuing events which are to be
expected and (undesired) queue build up are the events to be distinguished.
I hope that can be done with commodity hard- and software. I at least am
not able to write down a simple metric distinguishing queues to be expected
from (undesired) queue build up causing congestion. The hard- and software
to be used should be part of the solution, not part of the problem (bursty
source traffic and timestamps with insufficient accuracy to detect queues
are what I’d like to avoid).



I’d suggest to move discussion to the list.



Regards,



Rüdiger


----------
From: MORTON, ALFRED C (AL) <[e-mail]>
Date: Thu, Sep 19, 2019 at 7:01 AM
To: [e-mail] <[e-mail]>
Cc: CIAVATTONE, LEN <[e-mail]>, [e-mail] <
[e-mail]>


I’m catching-up with this thread again, but before I reply:



*** Any objection to moving this discussion to IPPM-list ?? ***



@Matt – this is a question to you at this point...



thanks,

Al



*From:* [e-mail] [mailto:[e-mail]]
*Sent:* Monday, August 26, 2019 3:05 AM
*To:* MORTON, ALFRED C (AL) <[e-mail]>
*Cc:* CIAVATTONE, LEN <[e-mail]>; [e-mail]
*Subject:* AW: How should capacity measurement interact with shaping?



Hi Al,



thanks for keeping me involved. I don’t have a precise answer and doubt,
that there will be a single universal truth.



If the aim is only to determine the IP bandwidth of an access, then we
aren’t interested in filling a buffer. Buffering events may occur, some of
which are useful and to be expected, whereas others are not desired:



-        Sender shaping behavior may matter (is traffic at the source CBR
or is it bursty)

-        Random collisions should be tolerated at the access whose
bandwidth is to be measured.

-        Limiting packet drop due to buffer overflow is a design aim or an
important part of the algorithm, I think.

-        Shared media might create bursts. I’m not an expert in the area,
but there’s an “is bandwidth available” check in some cases between a
central sender using a shared medium and the receivers connected. WiFi and
may be other wireless equipment buffers packets also to optimize wireless
resource optimization.

-        It might be an idea to mark some flows by ECN, once there’s a
guess on a sending bitrate when to expect no or very little packet drop.
Today, this is experimental. CE marks by an ECN capable device should be
expected roughly once queuing starts.