Re: [tsvwg] Traffic protection as a hard requirement for NQB

[Greg White <[e-mail]>]

Hi Steven,

See my responses (marked [GW2]).

-Greg

From: Steven Blake <[e-mail]>
Date: Sunday, September 8, 2019 at 9:50 AM
To: Greg White <[e-mail]>
Cc: "[e-mail]" <[e-mail]>
Subject: Re: [tsvwg] Traffic protection as a hard requirement for NQB

On Fri, 2019-09-06 at 19:53 +0000, Greg White wrote:

[GW] comments below

Greg,

I don't know how to reconcile any of this below with the text of your draft (specifically Secs. 1 & 3), which talks about low-bandwidth applications, specifically those over UDP. Are you contending that the sum of these adds up to a significant fraction of traffic on any link?

[GW2] While it’s not difficult to come up with scenarios where the NQB traffic adds up to a significant percentage of link capacity, yet remains NQB, the point I was trying to make was that there isn’t a need to artificially limit NQB traffic to some small fraction of link capacity, and doing so just makes the problem harder.  Using a work-conserving DRR scheduler with 50/50 weighting between the queues is sufficient.  Like Bob Briscoe indicated in one of his recent posts, throw out your preconceived notions of how QoS has traditionally been implemented in networks.  This is different.

From: tsvwg <[e-mail]> on behalf of Steven Blake <[e-mail]>

The objective as I understand it is to offer low queueing latency to non-admissioned-controlled/non-capacity-seeking traffic (NQB). First, this is only feasible if NQB traffic is naturally self-limiting to a small fraction of total traffic (e.g., < 5%). Second, existence of this NQB service should not noticeably impact the QoS of capacity-seeking (QB) traffic. Third, implementations of this NQB service should try to ensure that capacity-seeking traffic trying to utilize the NQB service gets worse service with very high probability (so that it doesn't even bother to probe).

[GW] If the WRR weight is 0.5 for each queue, then it is feasible for the NQB PHB to provide low queueing latency to NQB traffic up to 50% of the total link capacity (and more than 50% when QB traffic is not filling up the other 50%).

That claim is technically false. You cannot achieve 100% utilization on any link (or link slice) at low queueing delay for uncoordinated, non-admissioned controlled traffic. I'll bet you cannot find a carrier anywhere that allocates 50% (or even 20%) of capacity on their metro or long-haul networks to admissioned-controlled & tarriffed EF traffic. So I don't see them doing so for non-admissioned controlled/non-tarriffed traffic.

[GW2] I think you’re missing my point.  The point was that it isn’t necessary that the aggregate of NQB traffic remains below 5% of link capacity.  Also, as the draft discusses, the intended application of the NQB PHB is on access network links, not long-haul or metro links.  Even so, you can’t compare historical configurations of EF prioritization to NQB.   If you need an analogy, imagine a link that supports an FQ-like qdisc, where flows are hashed into two queues, and there is a work-conserving DRR scheduler with equal weighting between the two queues.  While this hypothetical system wouldn’t give a whole lot benefit, I hope you can see that it wouldn’t be objectionable either.   Maybe you get lucky and the queue your flow gets hashed into is less loaded than the other, but maybe it isn’t.


[GW] On the second point, you are correct, the existence of the NQB service will not noticeably impact the QoS of capacity-seeking QB traffic.  For example, if the NQB traffic did happen to be smooth traffic (CBR) at exactly 50% of link capacity, then with NQB, the QB traffic flows have 50% of link capacity to compete with each other for. Let’s say that competition results in (e.g.) 0.1% packet drop/mark.  On the other hand, without the NQB service, and in the presence of the same CBR flow, the capacity-seeking traffic flows similarly compete for the remaining 50% of link capacity, and settle in to the same 0.1% packet drop/mark rate (yet the CBR flow is unfortunately also subject to high queuing delay as well as 0.1% drop if the link doesn’t support ECN).  Well, being picky, if the link didn’t support ECN then I guess the capacity seeking traffic would compete for 50.1% of link capacity without the NQB PHB, as opposed to 50% with the NQB PHB, but I would argue that isn’t a “noticeable impact”.

If I am writing a capacity seeking application, and I believe that there is a lightly utilized link slice of 20%-50% of capacity where I can grab more than my fair share, I would be highly tempted to setup two connections, one-NQB marked and one not, and let them each grab as much capacity as possible. Which benefits my app until everyone else does the same, and then we are back to square one.

[GW2] You could be setting up two connections today, even without NQB.  Heck, you could set up 200.  Or you could roll your own congestion controller that cuts its congestion window by 5% in response to a congestion signal (under the assumption that all the other suckers will back off first).  It is possible for the internet to be abused. I still find it useful.
[GW2] As a more specific response to this, what would you use for your congestion controller on the NQB queue?  In the case of DOCSIS, you’ll need to (more-or-less blindly) guarantee that you can maintain sub-millisecond queuing delay while seeking the link capacity in order to succeed. Why not just use TCP Prague or BBRv2 and make your life easier?

[GW] On your third point, I agree in concept, and it is an aspect that we’ve discussed privately several times.  At the end of the day, we could not convince ourselves of an appropriate “punishment” for mismarked QB traffic, other than high packet loss in the case where QP is not implemented or potential for out-of-order delivery when QP is implemented.

Given this, using a WFQ-ish scheduler with a high NQB scheduler weight to achieve low latency is the wrong hammer to hit this nail, IMHO. A small queue by itself is insufficient to prevent capacity-seeking traffic from trying to grab that large & isolated share of capacity. If I were implementing this (and only this) I would consider putting NQB traffic in a bounded strict priority queue, feeding it priority credits at say 2-4% of link capacity. I would also put a policer in front of that queue set at a rate of 4-6% link capacity, with a large enough token bucket to let small NQB bursts through. That is a very simple implementation, available in lots of existing hardware, that achieves the stated objectives.

[GW] But priority isn’t necessary, and it results in the need for limitations on NQB bandwidth.  For example, in your implementation, what would happen when the arrival rate at the link is only 20% of link capacity but all of it is NQB?  I guess the policer would unnecessarily discard a large percentage of that traffic.

DoS-ing NQB traffic in this implemention is "easier" than DoS-ing it in your proposed scheduling approach, but a targeted NQB DoS attack will have less impact on capacity-seeking traffic, and DoS-ing residential access links using only best-effort service is already a thing, so I'm not sure it really matters.

How do you estimate how much NQB traffic there might be? How would you tune this? My assumption is that this whole scheme only survives if the amount of NQB traffic is always so small relative to any link's capacity (< 5%) that you just allocation a small fraction of link capacity to NQB and forget about it. In which case, WRR scheduling doesn't give you good latency because your HOL NQB packet might have to wait for >= 20 MTU/link rate to get serviced.

[GW2] The whole idea is that estimating NQB bandwidth and tuning isn’t required.


Regards,

// Steve