Re: [tsvwg] Fwd: New Version Notification for draft-white-tsvwg-lld-00.txt

[Luca Muscariello <[e-mail]>]

Greg,

You may be right. We may be doing different assumptions.
To me the point where we disagree is that you assume that an application,
to get access to the low latency queue, is not only well behaving
but also sending at a very low rate compared to the link capacity.

The point I'm trying to make is that in general, we do not know that rate.
As long as we have no predictable information about the amount of non
classified
traffic that can be admitted to the low latency queue, the coupling factor,
that
determines how time is shared between the two queues, cannot be set.
Any value would break the stable equilibrium assumed by the system and
therefore
that much traffic has to be kicked out of the queue.

This is not a positive incentive but rather policy enforcement.
This is in itself a subject of discussion.
Another concern I see is more about DoS to the low latency queue and
then to both queues.

It seems a priori definitely possible to create a limit cycle to the
control theoretic
equilibrium by injecting background non classified traffic that generates
little queuing.
Such packet process could then disturb all other sources or worse starve
them enough
to force user to abandon.
In this case even a FIFO queue would be better (providing more predictable
performance)
than two coupled queues relying on
a closed loop equilibrium which assumes only compliant traffic is present.

Now, being aware of how difficult it is to compute stability regions of a
closed loop system
with delays, which is the one we are discussing here, my overall impression
is that
this queueing system is fragile and can provide unpredictable performance.

Assuming that you're pretty confident about the cable use case you have in
mind,
with all the assumptions you are making about today applications' rates,
would you also
be confident about future applications or about other network access
technologies?

Luca

On Tue, Mar 19, 2019 at 10:12 PM Greg White <[e-mail]> wrote:

> I think that is a mischaracterization.
>
>
>
> I don’t believe that (at least on cable broadband links) non-responsive,
> link-saturating traffic is “currently a big chunk of Internet traffic that
> requires low latency”.   Perhaps on very low speed links this may happen,
> but what are the applications that are streaming at (e.g.) 10 Mbps without
> any congestion control?
>
>
>
> The current Internet places expectations on data sources that they
> implement congestion control, and that they respond in a fairly consistent
> way to congestion signals (usually packet drops).  Not all of them do, but
> the majority do, and it has not resulted in a tragedy of the commons.  The
> dual queue mechanism continues that expectation.
>
>
>
> I also don’t agree that the dual queue system lacks incentives.  For an
> application to get the benefit of sub millisecond queuing delay, it needs
> to ensure that it is not the cause of the queuing delay.  It can do that in
> a couple of ways.  One is that it sends at a low and smooth enough rate
> that it is highly unlikely to cause queuing, the other is that it can
> respond to fast congestion signals (new ECN marks).  The applications that
> do this can enjoy sub millisecond queuing delay.  If that isn’t an
> incentive, that’s fine. Some NQB traffic may not care about latency
> performance, and so would be equally happy in the classic queue.
>
>
>
> -Greg
>
>
>
>
>
>
>
>
>
> *From: *Luca Muscariello <[e-mail]>
> *Date: *Tuesday, March 19, 2019 at 2:53 AM
> *To: *Greg White <[e-mail]>
> *Cc: *"Tilmans, Olivier (Nokia - BE/Antwerp)" <
> [e-mail]>, tsvwg IETF list <[e-mail]>
> *Subject: *Re: [tsvwg] Fwd: New Version Notification for
> draft-white-tsvwg-lld-00.txt
>
>
>
> Got it, thanks.
>
>
>
> Consider that one concern that I have is on what you Greg consider a
> pathological case.
>
> That case is currently a big chunk of Internet traffic that requires low
> latency.
>
>
>
> The current scheduler makes substantial assumptions about the behaviour
> and misbehaviour of data sources.
>
> The thing the I find uncomfortable is that this proposal assumes that
> every source conforms to a given profile,
>
> including explicit marking.
>
>
>
> The current queueing system instead of creating incentives to well behave
> creates a mechanism that punishes
>
> sources that do not conform. Even  sources that well behave can be
> punished just because they do not conform.
>
>
>
> My understanding is that this might be difficult to accept for many
> applications and, most important, people.
>
>
>
> Luca
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Tue, Mar 19, 2019 at 4:31 AM Greg White <[e-mail]> wrote:
>
> Hi Luca, see below [GW].
>
>
>
> *From: *Luca Muscariello <[e-mail]>
>
>
>
> On Mon, Mar 18, 2019 at 1:21 PM Tilmans, Olivier (Nokia - BE/Antwerp) <
> [e-mail]> wrote:
>
> Hi Luca,
>
> Just to make sure, did I answer your question?
>
>
>
> maybe. Let me ask a few more clarification questions below.
>
>
>
>
> If not the behavior of putting NQB flows in the L4S queue is dependent on
> the
> AQM used, hence the answer would be to look at
> draft-ietf-tsvwg-aqm-dualq-coupled for a discussion of the effects of
> doing so.
> DOCSIS adds on top of the AQM a protection scheme to sanction flows in the
> L4S that cause queue build up, and uses a WRR to control how packets get
> dequeued.
>
> The final answer would then be: yes, you can put non-ECN controlled flows
> in
> the L4S queue provided those are sparse enough. Otherwise, they would need
> to be wrapped in a scalable CC and/or interpret the received ECN feedback
> (as
>  the 2015 demo showing live video streaming zooming/zoomout did).
>
>
>
> This is where things become unclear or just unspecified in the documents I
> read so far.
>
> Sometimes maths or code is simpler to read that a thousand words.
>
>
>
> My interpretation of what you say is that the time sharing between the two
> queues
>
> depends on a closed loop mechanism that relies on classification first and
> a compliant sender that
>
> is sensitive to ECN according  to the mechanism described in
> draft-ietf-tsvwg-l4s-arch.
>
>
>
> Assuming that the control theoretic equilibrium you get requires a given
> WRR weight,
>
> anytime non conformant traffic is allowed into the low latency queue this
> equilibrium can be
>
> perturbed. Which is fine as long as the perturbation is small enough.
>
>
>
> [GW] When both queues are in use by responsive traffic, the classic queue
> AQM drives the congestion signal for both queues, and gives them both a
> congestion signal that the two sender types interpret in a consistent way,
> leading toward per-flow fairness.  Just as with a single queue AQM, the
> consistent (flow unaware) application of drop probability (and presumed
> consistent interpretation of those congestion signals by senders) leads
> toward per-flow fairness.  The equilibrium is not strongly related to the
> WRR weight.  As long as the weight is high enough, the relationship between
> the congestion signals (drop in the case of the classic queue and ECN mark
> in the case of the low latency queue) drives the equilibrium.  Just as an
> example of a simple case, if all of the flows happened to be long-running
> responsive flows in congestion avoidance, then the congestion signals will
> drive the equilibrium as long as the WRR weight given to the low latency
> queue is greater than or equal to the share of the flows that are occupying
> that queue.  In the DOCSIS spec the default value for the weight is 230/256
> (about 90%, see section C.2.2.7.17.6 in the DOCSIS spec), so as long as
> less than 90% of the simultaneous flows are occupying the low latency queue
> (for example if there are four L4S flows and one CUBIC flow), per-flow
> fairness will be maintained.   If greater than 90% of the simultaneous
> flows are in the low latency queue (e.g. twenty-four L4S flows vs. two
> CUBIC flows), the ~10%  weight given to the classic queue will result in
> the classic flows each getting somewhat more than their share of bandwidth
> (e.g. they will each get 5% of the capacity, while the L4S flows will each
> get 3.75%).  If we could assume that ALL flows are well-behaved responsive
> flows, then strict priority could be used, and per-flow fairness would be
> maintained regardless of the mix.  Since we can’t make that assumption, the
> non-100% weight protects some bandwidth in the Classic queue.
>
>
>
> Non controlled traffic can generate a busy period in the low latency queue
> that can grow
>
> w/o necessarily generating large standing queues. That should not trigger
> the queue protection
>
> mechanism Greg mentioned.
>
> From what I understand WRR would  then make the equilibrium drift to
> another value.
>
> This value should be that the classic queue gets lower service time
> compared to what
>
> the original configuration was meant to provide.
>
>
>
> [GW] Non-responsive traffic in the low latency queue will consume some of
> that queue’s WRR weight (similarly, non-responsive traffic in the classic
> queue will consume some of that queue’s weight), with the result being that
> for the remaining congestion controlled traffic the range of flow mixes
> over which per-flow fairness is achieved is shifted.  As an example, if
> non-responsive traffic in the low latency queue consumed 50% of the total
> capacity, and the rest of the traffic was congestion controlled, the system
> would behave as if (for the remaining 50% of capacity available to the
> congestion controlled flows) the WRR weight was 80% (90% minus 50%, divided
> by 50%).   So per-flow fairness would only be achieved when the low latency
> queue is carrying less than or equal to 80% of the flows, otherwise the
> classic flows will get somewhat more than their share.    As another
> example, if non-responsive traffic in the classic queue were consuming 50%
> of the total capacity, and the rest of the traffic was congestion
> controlled, the system would provide per-flow fairness (with the remaining
> 50% of the capacity) across ALL mixes of L4S/Classic.
>
>
>
> [GW] Yes, there are pathological cases where high-bandwidth non-responsive
> flows are consuming the entire channel capacity.  In those cases, the WRR
> weight will result in the flows in the low latency queue getting more
> aggregate bandwidth than the ones in the classic queue.
>
>
>
>
>
> Do I get this right?
>
>
>
> Thanks
>
> Luca
>
>
>
>
>
> Best,
> Olivier
>
>
> > -----Original Message-----
> > From: Luca Muscariello <[e-mail]>
> > Sent: Monday, March 18, 2019 12:21 PM
> > To: Tilmans, Olivier (Nokia - BE/Antwerp) <olivier.tilmans@nokia-bell-
> > labs.com>
> > Cc: Greg White <[e-mail]>; tsvwg IETF list <[e-mail]>
> > Subject: Re: [tsvwg] Fwd: New Version Notification for
> draft-white-tsvwg-lld-
> > 00.txt
> >
> >
> >
> > On Mon, Mar 18, 2019 at 9:56 AM Tilmans, Olivier (Nokia - BE/Antwerp)
> > <[e-mail] <mailto:olivier.tilmans@nokia-bell-
> > labs.com> > wrote:
> >
> >
> >       > Is there an assumption that the low latency queue is only used by
> > rate
> >       > controlled flows
> >       > that are sensitive to ECN feedback?
> >
> >       The L4S definitions in draft-ietf-tsvwg-l4s-arch state that L4S
> flow
> > have a
> >       scalable CC, in order to perform bandwidth pooling (fairly) between
> > the
> >       L4S flows and the classic ones.
> >       This can however be relaxed a bit, see below.
> >
> >       > I would assume that a low latency queue is open to flows that are
> > non
> >       > necessarily controlled by ECN but just exogenously rate limited
> by
> > say a
> >       > codec rate.
> >
> >       This is what draft draft-white-tsvwg-nqb discusses, namely also
> > adding
> >       non-queue building flows in the L4S queue (e.g., non-capacity
> > seeking
> >       flows such as gaming, DNS traffic, pings, sparse CBR flows).
> >
> >       Note that the presence of such unresponsive flows in the L4S queue
> >       implicitly decreases the throughput of the scalable L4S flows (but
> not
> >       their classic counterpart), as they have to back off on behalf of
> the
> >       non-responsive flows (but can do so without fiddling with a magic
> > WRR
> >       number). This can be accounted for in the coupling factor of the
> > queue.
> >
> >
> >
> > Right, this is what I was trying to understand.
> > I checked draft-white-tsvwg-nqb but the mechanism to share traffic in the
> > two queues
> > is not described. This document refers back to draft-ietf-tsvwg-l4s-arch
> so I
> > cycled back to
> > my starting point as there is no description of how the queueing system
> > works when
> > non marked ECNed traffic is sent  to the low latency queue.
> >
> > I thought that that was described in the DOCSIS spec Greg shared, but did
> > not find an answer there either.
> >
> >
> >
> >
> >
> >
> >
> >       Best,
> >       Olivier
> >
> >
> >       > But now it seems like this is not the case.
> >       >
> >       > Thanks
> >       > Luca
> >       >
> >       > On Sun 17 Mar 2019 at 18:32, Tilmans, Olivier (Nokia -
> BE/Antwerp)
> >       > <[e-mail]
> > <mailto:[e-mail]>  <mailto:
> olivier.tilmans@nokia-
> > bell- <mailto:olivier.tilmans@nokia-bell->
> >       > labs.com <http://labs.com> > > wrote:
> >       >
> >       >
> >       >       Hi Luca,
> >       >
> >       >       > The protection mechanism assumes that one queue has soft
> >       > priority over
> >       >       > the other. Strict priority would be stupid, so there
> must be a
> > wfq
> >       > weight to
> >       >       > schedule the classic queue less frequently. I did not
> find the
> > magic
> >       > number
> >       >       > that  is set in the DOCSIS specs but whatever number is
> chosen
> > I
> >       > wonder
> >       >       > which opitimization objective would be the foundation of
> that.
> >       >       > 10%, 20%, 30% or any number would imply that if the
> priority
> >       > queue is used
> >       >       > at 100% utilization the other apps would get a small
> fraction of
> > the
> >       > link
> >       >       > capacity.
> >       >       >
> >       >       > What number is chosen and based on which calculations?
> >       >
> >       >       The protection mechanism only affect how competing packets
> > within
> >       > a RTT gets
> >       >       dequeued. In this instance, a WRR protects the classic
> flows
> > from
> >       >       unresponsive/misclassified L4S flows, at the expense of the
> > other
> >       > L4S flows.
> >       >
> >       >       It does not affect the overall throughput of the flows when
> >       > measuring it over a
> >       >       timeframe of a few Tupdate interval (e.g., 16ms). Indeed,
> the
> > core of
> >       > the
> >       >       DualPI2 AQM is a PIE2 controller computing the random
> >       > drop/marking probability
> >       >       based on the classic queue. This probability is then also
> used as
> > base
> >       > to mark L4S
> >       >       flows, albeit with a quadratic relationship (i.e., for the
> same base
> >       > probability, the
> >       >       L4S flows are marked much more often than the classic
> ones).
> >       >
> >       >       Consequently, whenever L4S traffic causes the classic
> queue to
> > build
> >       > up (even
> >       >       slightly above the PIE2 target), its marking probability
> drastically
> >       > increases as the
> >       >       PIE2 controller reacts. As the L4S flows have scalable CCs
> (i.e.,
> >       > understand how to
> >       >       leverage multiple CE marks per RTT), they reduce their cwnd
> >       > accordingly within
> >       >        a RTT (which is much smaller than the classic flows' RTT).
> >       >
> >       >       In other words, L4S senders end up slowing down themselves
> > (to a
> >       > point where
> >       >       the L4S queue is empty for most of the transmission slots)
> to
> > allow
> >       > the classic
> >       >       queue to drain, and this reaction is the result of a much
> faster
> >       > control-loop than
> >       >       the one used by the classic sender.
> >       >
> >       >       There are thus no magic number per say for the WRR ratio.
> > §4.1.4 of
> >       >        draft-ietf-tsvwg-aqm-dualq-coupled explains the
> guidelines to
> > pick
> >       > one.
> >       >
> >       >       Note that the exact throughput ratio between L4S and
> classic
> > flows
> >       > can be
> >       >       tweaked using a coupling factor, as explained in appendix
> C. of
> >       >        draft-ietf-tsvwg-aqm-dualq-coupled.
> >       >
> >       >       I hope this was clear enough.
> >       >
> >       >
> >       >       Best,
> >       >       Olivier
> >       >
> >
> >
>
>