Re: [tsvwg] Fwd: New Version Notification for draft-white-tsvwg-lld-00.txt

[Greg White <[e-mail]>]

I think that is a mischaracterization.

I don’t believe that (at least on cable broadband links) non-responsive, link-saturating traffic is “currently a big chunk of Internet traffic that requires low latency”.   Perhaps on very low speed links this may happen, but what are the applications that are streaming at (e.g.) 10 Mbps without any congestion control?

The current Internet places expectations on data sources that they implement congestion control, and that they respond in a fairly consistent way to congestion signals (usually packet drops).  Not all of them do, but the majority do, and it has not resulted in a tragedy of the commons.  The dual queue mechanism continues that expectation.

I also don’t agree that the dual queue system lacks incentives.  For an application to get the benefit of sub millisecond queuing delay, it needs to ensure that it is not the cause of the queuing delay.  It can do that in a couple of ways.  One is that it sends at a low and smooth enough rate that it is highly unlikely to cause queuing, the other is that it can respond to fast congestion signals (new ECN marks).  The applications that do this can enjoy sub millisecond queuing delay.  If that isn’t an incentive, that’s fine. Some NQB traffic may not care about latency performance, and so would be equally happy in the classic queue.

-Greg




From: Luca Muscariello <[e-mail]>
Date: Tuesday, March 19, 2019 at 2:53 AM
To: Greg White <[e-mail]>
Cc: "Tilmans, Olivier (Nokia - BE/Antwerp)" <[e-mail]>, tsvwg IETF list <[e-mail]>
Subject: Re: [tsvwg] Fwd: New Version Notification for draft-white-tsvwg-lld-00.txt

Got it, thanks.

Consider that one concern that I have is on what you Greg consider a pathological case.
That case is currently a big chunk of Internet traffic that requires low latency.

The current scheduler makes substantial assumptions about the behaviour and misbehaviour of data sources.
The thing the I find uncomfortable is that this proposal assumes that every source conforms to a given profile,
including explicit marking.

The current queueing system instead of creating incentives to well behave creates a mechanism that punishes
sources that do not conform. Even  sources that well behave can be punished just because they do not conform.

My understanding is that this might be difficult to accept for many applications and, most important, people.

Luca






On Tue, Mar 19, 2019 at 4:31 AM Greg White <[e-mail]<mailto:[e-mail]>> wrote:
Hi Luca, see below [GW].

From: Luca Muscariello <[e-mail]<mailto:[e-mail]>>

On Mon, Mar 18, 2019 at 1:21 PM Tilmans, Olivier (Nokia - BE/Antwerp) <[e-mail]<mailto:[e-mail]>> wrote:
Hi Luca,

Just to make sure, did I answer your question?

maybe. Let me ask a few more clarification questions below.


If not the behavior of putting NQB flows in the L4S queue is dependent on the
AQM used, hence the answer would be to look at
draft-ietf-tsvwg-aqm-dualq-coupled for a discussion of the effects of doing so.
DOCSIS adds on top of the AQM a protection scheme to sanction flows in the
L4S that cause queue build up, and uses a WRR to control how packets get
dequeued.

The final answer would then be: yes, you can put non-ECN controlled flows in
the L4S queue provided those are sparse enough. Otherwise, they would need
to be wrapped in a scalable CC and/or interpret the received ECN feedback (as
 the 2015 demo showing live video streaming zooming/zoomout did).

This is where things become unclear or just unspecified in the documents I read so far.
Sometimes maths or code is simpler to read that a thousand words.

My interpretation of what you say is that the time sharing between the two queues
depends on a closed loop mechanism that relies on classification first and a compliant sender that
is sensitive to ECN according  to the mechanism described in draft-ietf-tsvwg-l4s-arch.

Assuming that the control theoretic equilibrium you get requires a given WRR weight,
anytime non conformant traffic is allowed into the low latency queue this equilibrium can be
perturbed. Which is fine as long as the perturbation is small enough.

[GW] When both queues are in use by responsive traffic, the classic queue AQM drives the congestion signal for both queues, and gives them both a congestion signal that the two sender types interpret in a consistent way, leading toward per-flow fairness.  Just as with a single queue AQM, the consistent (flow unaware) application of drop probability (and presumed consistent interpretation of those congestion signals by senders) leads toward per-flow fairness.  The equilibrium is not strongly related to the WRR weight.  As long as the weight is high enough, the relationship between the congestion signals (drop in the case of the classic queue and ECN mark in the case of the low latency queue) drives the equilibrium.  Just as an example of a simple case, if all of the flows happened to be long-running responsive flows in congestion avoidance, then the congestion signals will drive the equilibrium as long as the WRR weight given to the low latency queue is greater than or equal to the share of the flows that are occupying that queue.  In the DOCSIS spec the default value for the weight is 230/256 (about 90%, see section C.2.2.7.17.6 in the DOCSIS spec), so as long as less than 90% of the simultaneous flows are occupying the low latency queue (for example if there are four L4S flows and one CUBIC flow), per-flow fairness will be maintained.   If greater than 90% of the simultaneous flows are in the low latency queue (e.g. twenty-four L4S flows vs. two CUBIC flows), the ~10%  weight given to the classic queue will result in the classic flows each getting somewhat more than their share of bandwidth (e.g. they will each get 5% of the capacity, while the L4S flows will each get 3.75%).  If we could assume that ALL flows are well-behaved responsive flows, then strict priority could be used, and per-flow fairness would be maintained regardless of the mix.  Since we can’t make that assumption, the non-100% weight protects some bandwidth in the Classic queue.

Non controlled traffic can generate a busy period in the low latency queue that can grow
w/o necessarily generating large standing queues. That should not trigger the queue protection
mechanism Greg mentioned.
From what I understand WRR would  then make the equilibrium drift to another value.
This value should be that the classic queue gets lower service time compared to what
the original configuration was meant to provide.

[GW] Non-responsive traffic in the low latency queue will consume some of that queue’s WRR weight (similarly, non-responsive traffic in the classic queue will consume some of that queue’s weight), with the result being that for the remaining congestion controlled traffic the range of flow mixes over which per-flow fairness is achieved is shifted.  As an example, if non-responsive traffic in the low latency queue consumed 50% of the total capacity, and the rest of the traffic was congestion controlled, the system would behave as if (for the remaining 50% of capacity available to the congestion controlled flows) the WRR weight was 80% (90% minus 50%, divided by 50%).   So per-flow fairness would only be achieved when the low latency queue is carrying less than or equal to 80% of the flows, otherwise the classic flows will get somewhat more than their share.    As another example, if non-responsive traffic in the classic queue were consuming 50% of the total capacity, and the rest of the traffic was congestion controlled, the system would provide per-flow fairness (with the remaining 50% of the capacity) across ALL mixes of L4S/Classic.

[GW] Yes, there are pathological cases where high-bandwidth non-responsive flows are consuming the entire channel capacity.  In those cases, the WRR weight will result in the flows in the low latency queue getting more aggregate bandwidth than the ones in the classic queue.


Do I get this right?

Thanks
Luca



Best,
Olivier


> -----Original Message-----
> From: Luca Muscariello <[e-mail]<mailto:[e-mail]>>
> Sent: Monday, March 18, 2019 12:21 PM
> To: Tilmans, Olivier (Nokia - BE/Antwerp) <olivier.tilmans@nokia-bell-
> labs.com<http://labs.com>>
> Cc: Greg White <[e-mail]<mailto:[e-mail]>>; tsvwg IETF list <[e-mail]<mailto:[e-mail]>>
> Subject: Re: [tsvwg] Fwd: New Version Notification for draft-white-tsvwg-lld-
> 00.txt
>
>
>
> On Mon, Mar 18, 2019 at 9:56 AM Tilmans, Olivier (Nokia - BE/Antwerp)
> <[e-mail]<mailto:[e-mail]> <mailto:olivier.tilmans@nokia-bell-<mailto:olivier.tilmans@nokia-bell->
> labs.com<http://labs.com>> > wrote:
>
>
>       > Is there an assumption that the low latency queue is only used by
> rate
>       > controlled flows
>       > that are sensitive to ECN feedback?
>
>       The L4S definitions in draft-ietf-tsvwg-l4s-arch state that L4S flow
> have a
>       scalable CC, in order to perform bandwidth pooling (fairly) between
> the
>       L4S flows and the classic ones.
>       This can however be relaxed a bit, see below.
>
>       > I would assume that a low latency queue is open to flows that are
> non
>       > necessarily controlled by ECN but just exogenously rate limited by
> say a
>       > codec rate.
>
>       This is what draft draft-white-tsvwg-nqb discusses, namely also
> adding
>       non-queue building flows in the L4S queue (e.g., non-capacity
> seeking
>       flows such as gaming, DNS traffic, pings, sparse CBR flows).
>
>       Note that the presence of such unresponsive flows in the L4S queue
>       implicitly decreases the throughput of the scalable L4S flows (but not
>       their classic counterpart), as they have to back off on behalf of the
>       non-responsive flows (but can do so without fiddling with a magic
> WRR
>       number). This can be accounted for in the coupling factor of the
> queue.
>
>
>
> Right, this is what I was trying to understand.
> I checked draft-white-tsvwg-nqb but the mechanism to share traffic in the
> two queues
> is not described. This document refers back to draft-ietf-tsvwg-l4s-arch so I
> cycled back to
> my starting point as there is no description of how the queueing system
> works when
> non marked ECNed traffic is sent  to the low latency queue.
>
> I thought that that was described in the DOCSIS spec Greg shared, but did
> not find an answer there either.
>
>
>
>
>
>
>
>       Best,
>       Olivier
>
>
>       > But now it seems like this is not the case.
>       >
>       > Thanks
>       > Luca
>       >
>       > On Sun 17 Mar 2019 at 18:32, Tilmans, Olivier (Nokia - BE/Antwerp)
>       > <[e-mail]<mailto:[e-mail]>
> <mailto:[e-mail]<mailto:[e-mail]>>  <mailto:olivier.tilmans@nokia-<mailto:olivier.tilmans@nokia->
> bell- <mailto:olivier.tilmans@nokia-bell-<mailto:olivier.tilmans@nokia-bell->>
>       > labs.com<http://labs.com> <http://labs.com> > > wrote:
>       >
>       >
>       >       Hi Luca,
>       >
>       >       > The protection mechanism assumes that one queue has soft
>       > priority over
>       >       > the other. Strict priority would be stupid, so there must be a
> wfq
>       > weight to
>       >       > schedule the classic queue less frequently. I did not find the
> magic
>       > number
>       >       > that  is set in the DOCSIS specs but whatever number is chosen
> I
>       > wonder
>       >       > which opitimization objective would be the foundation of that.
>       >       > 10%, 20%, 30% or any number would imply that if the priority
>       > queue is used
>       >       > at 100% utilization the other apps would get a small fraction of
> the
>       > link
>       >       > capacity.
>       >       >
>       >       > What number is chosen and based on which calculations?
>       >
>       >       The protection mechanism only affect how competing packets
> within
>       > a RTT gets
>       >       dequeued. In this instance, a WRR protects the classic flows
> from
>       >       unresponsive/misclassified L4S flows, at the expense of the
> other
>       > L4S flows.
>       >
>       >       It does not affect the overall throughput of the flows when
>       > measuring it over a
>       >       timeframe of a few Tupdate interval (e.g., 16ms). Indeed, the
> core of
>       > the
>       >       DualPI2 AQM is a PIE2 controller computing the random
>       > drop/marking probability
>       >       based on the classic queue. This probability is then also used as
> base
>       > to mark L4S
>       >       flows, albeit with a quadratic relationship (i.e., for the same base
>       > probability, the
>       >       L4S flows are marked much more often than the classic ones).
>       >
>       >       Consequently, whenever L4S traffic causes the classic queue to
> build
>       > up (even
>       >       slightly above the PIE2 target), its marking probability drastically
>       > increases as the
>       >       PIE2 controller reacts. As the L4S flows have scalable CCs (i.e.,
>       > understand how to
>       >       leverage multiple CE marks per RTT), they reduce their cwnd
>       > accordingly within
>       >        a RTT (which is much smaller than the classic flows' RTT).
>       >
>       >       In other words, L4S senders end up slowing down themselves
> (to a
>       > point where
>       >       the L4S queue is empty for most of the transmission slots) to
> allow
>       > the classic
>       >       queue to drain, and this reaction is the result of a much faster
>       > control-loop than
>       >       the one used by the classic sender.
>       >
>       >       There are thus no magic number per say for the WRR ratio.
> §4.1.4 of
>       >        draft-ietf-tsvwg-aqm-dualq-coupled explains the guidelines to
> pick
>       > one.
>       >
>       >       Note that the exact throughput ratio between L4S and classic
> flows
>       > can be
>       >       tweaked using a coupling factor, as explained in appendix C. of
>       >        draft-ietf-tsvwg-aqm-dualq-coupled.
>       >
>       >       I hope this was clear enough.
>       >
>       >
>       >       Best,
>       >       Olivier
>       >
>
>