Re: [tsvwg] Fwd: New Version Notification for draft-white-tsvwg-lld-00.txt

[Bob Briscoe <[e-mail]>]

Luca,

In reviewing this thread I think you haven't received a full answer to a 
question you raised in your email early in the thread (below).

On 17/03/2019 08:02, Luca Muscariello wrote:
> One point that is not specified anywhere is when a mflow entry is 
> removed front the flow table.
>
> As the table is used to blacklist mflows my intuition was that it has 
> to be based on some sort of timer. Which is another parameter to set 
> in the mechanism.
>
> The table then is tracking flows that are in progress or maybe not. 
> For sure an entry is maintained even for flows w/o packets in the 
> queues.  This has poor scalability properties compared to fq where an 
> entry is maintained for flows with actual data in the queues.
[BB] The queue protection algorithm ages out the queuing score held for 
each microflow (mflow) at a constant rate. In fact, the queuing score is 
normalized to time, so that the only value held in the bucket for each 
flow is the bucket's expiry time. For well-behaved flows, this expiry 
time will typically be exceeded before the next packet of the same flow 
arrives. If it has not expired when the next packet of the mflow 
arrives, the algo adds an additional amount of time to the mflow's 
bucket for the new packet.

Thus, only ill-behaved flows persistently consume flow-state memory, 
while well-behaved flows recycle flow-state memory between themselves as 
their packets interleave each other.

The amount of time that the algo adds to an mflow's bucket is the 
queuing score of each of the mflow's packets, normalized to a duration, 
using the following per-packet calculation:
     qscore = pkt_size * p / V
where:
     p is the current AQM marking probability of the low latency queue 
(dimensionless, in [0,1])
     V is the (constant) congestion-bit-rate deemed acceptable for a 
single mflow (see below for rationale) [b/s]

The division by V normalizes qscore to a duration, because the units are 
[congestion-bit / (congestion-bit / s) = s]. Of course, because V is a 
constant, the division can be implemented as a per-packet integer 
multiply by 1/V. We also choose V to be a power of 2 so a bit-shift can 
be used if more efficient.

The intuition is that the queuing score of an mflow is higher the more 
queuing delay there is when its packets arrive. And the faster the 
mflow's packets arrive, the more likely qscore will accumulate faster 
than the passage of time ages it out. But the formula for qscore is not 
just based on intuition - it comes from shadow pricing theory.

Congestion-bit-rate can be conceptualized as the bit-rate of those 
packets of a flow that are ECN-marked. A scalable congestion control 
averages a constant congestion-bit-rate for any flow-rate (i.e. any 
link-rate and any number of flows). So we set V a few times greater than 
that (currently 4x), to allow some variability in a flow without risking 
sanction. This does not mean that flows have to use a scalable 
congestion control to get through Q Protection into the low latency 
queue, it just sets an upper bound under which all scalable congestion 
controls will be able keep. It also sets the bar for unresponsive flows, 
to prevent them causing significantly more queuing than responsive 
scalable flows.

I hope the rest of your questions have been answered satisfactorily 
already by others.

Cheers


Bob

>
> The protection mechanism assumes that one queue has soft priority over 
>  the other. Strict priority would be stupid, so there must be a wfq 
> weight to schedule the classic queue less frequently. I did not find 
> the magic number that  is set in the DOCSIS specs but whatever number 
> is chosen I wonder which opitimization objective would be the 
> foundation of that.
> 10%, 20%, 30% or any number would imply that if the priority queue is 
> used at 100% utilization the other apps would get a small fraction of 
> the link capacity.
>
> What number is chosen and based on which calculations?
>
> Thanks
> Luca
>
> On Fri 15 Mar 2019 at 00:42, Greg White <[e-mail] 
> <mailto:[e-mail]>> wrote:
>
>     Yes, it is easy for applications to access the low latency queue,
>     and the job of QP is to redirect those that shouldn’t have been
>     there.  We could instead run all traffic through QP and let it be
>     automatically sorted, but if there is some traffic that can be
>     excluded a priori, then doing so reduces the queue delay (recall
>     that the QP algorithm only sanctions packets if the queue delay is
>     above the 1ms threshold), reduces the potential that a
>     well-behaving application is sanctioned (perhaps due to hash
>     collisions), etc.   Also, since sanctioning could result in
>     out-of-order delivery for a microflow, presuming that this is ok
>     for all applications might not be advisable.  We have some ideas
>     in the works for modifications that might be useful to reduce the
>     need for packet marking.
>
>     I don’t believe that this mechanism increases DoSability (is that
>     a word?) compared with single queue or fq systems, but would be
>     interested in some validation of that. Yes it is possible for a
>     malicious sender to mark its packets as NQB, and send at a high
>     rate with various port numbers (to get unique flow ids), and thus
>     have some percentage of packets avoid sanctioning, and in the
>     worst case drive queue delay up.  The default threshold values are
>     being adjusted from what is shown in the spec currently, so when
>     using the new default values, the worst case with carefully
>     crafted attack streams is ~32 ms of queuing delay (which, using
>     brute force, would require the malicious sender to launch ~400
>     Mbps of traffic at the QP function).  Similar traffic patterns in
>     a single queue system (or fq) could, depending on egress rate,
>     result in much more queue delay than this.   Also, if the attacker
>     does not mark their packets NQB, then the system protects the NQB
>     flows from the attack, so in this sense it is also slightly less
>     susceptible than single queue (and maybe fq).
>
>     *From: *Luca Muscariello <[e-mail]
>     <mailto:[e-mail]>>
>     *Date: *Wednesday, March 13, 2019 at 4:07 PM
>     *To: *Greg White <[e-mail]>
>     *Cc: *"[e-mail] <mailto:[e-mail]>" <[e-mail]
>     <mailto:[e-mail]>>
>     *Subject: *Re: [tsvwg] New Version Notification for
>     draft-white-tsvwg-lld-00.txt
>
>     Greg,
>
>     Is there any intention to also grant access to the low latency
>     queue to applications that do not mark?
>
>     Also, if I got this whole mechanism right (big IF), it seems like
>     entering the low latency queue is only based on marking.
>
>     This way it looks like the queue is  easy to access and the
>     mechanism reacts to bad behaviour afterwards.
>
>     Does not this risk to create spurious packet enqueues into the low
>     latency queue? Or worse malicious sources
>
>     to make the low latency queue DoSable?
>
>     Luca
>
>     On Wed, Mar 13, 2019 at 2:03 PM Greg White <[e-mail]
>     <mailto:[e-mail]>> wrote:
>
>         Luca,
>
>         We may work on improving the informative text in that section
>         in order to make the behavior more clear without puzzling
>         through the pseudocode.
>
>         1.A packet enters the QP algorithm if it is classified to the
>         low latency queue.  By default, the intention is to use
>         ((DSCP==NQB) OR (ECN==ECT(1)) OR (ECN==CE)) (see the NQB draft
>         and ECN-L4S-ID draft) to identify likely NQB flows (but this
>         can be changed if needed).  The algorithm of course doesn’t
>         care how the packets get marked, but our view is that the
>         original application (or OS) should be the one that does the
>         marking. For either marking, there is no incentive to lie.
>
>         2.The algorithm sanctions (redirects to the classic queue) on
>         a per-packet basis.  So, a microflow that only briefly causes
>         queuing will have some of its packets sanctioned during that
>         episode only, and even during that episode, all of its packets
>         are scored. Note, in some cases this could result in
>         out-of-order delivery, which might be a minor disincentive for
>         persistently QB flows mismarking themselves as NQB.
>
>         3.See above.
>
>         Greg
>
>         *From: *Luca Muscariello <[e-mail]
>         <mailto:[e-mail]>>
>         *Date: *Wednesday, March 13, 2019 at 2:37 PM
>         *To: *Greg White <[e-mail]>
>         *Cc: *"[e-mail] <mailto:[e-mail]>" <[e-mail]
>         <mailto:[e-mail]>>
>         *Subject: *Re: [tsvwg] New Version Notification for
>         draft-white-tsvwg-lld-00.txt
>
>         Greg,
>
>         Thanks for the reference.
>
>         I have a few questions. I read the document just once so,
>         don't be surprised if I got something wrong.
>
>         1) A packet enters the low latency queue if it is marked. Who
>         is responsible for the marking?
>
>         Is the original application to do that?
>
>         2) Once a packet enters the low latency queue, the flow, or
>         microflow,
>
>         is tracked in a (micro)flow table by reporting a function of
>         the queue occupancy.  How do updates of the table
>
>         take place? Insert is clear unless the flow is banned, update
>         is clear.. Remove is only clear to me when the flow is moved
>
>         out of the low latency queue once it  started to get too much
>         resources. Remove in the general case is not clear to me. Timers?
>
>         3)  It seems like the mechanism trusts the packet marker until
>         the queue protection mechanism takes
>
>         a different decision. How can a marker be redeemed and related
>          packets be readmitted to the low latency queue?
>
>         Thanks
>
>         Luca
>
>         On Mon, Mar 11, 2019 at 7:18 PM Greg White
>         <[e-mail] <mailto:[e-mail]>> wrote:
>
>             Hi Luca,
>
>             You can find the details in Annex P of the DOCSIS
>             MULPIv3.1 spec.
>             https://specification-search.cablelabs.com/CM-SP-MULPIv3.1
>
>             -Greg
>
>
>             From: Luca Muscariello <[e-mail]
>             <mailto:[e-mail]>>
>             Date: Monday, March 11, 2019 at 6:26 PM
>             To: Greg White <[e-mail]>
>             Cc: Dave Taht <[e-mail] <mailto:[e-mail]>>,
>             Jonathan Morton <[e-mail]
>             <mailto:[e-mail]>>, "[e-mail]
>             <mailto:[e-mail]>" <[e-mail]
>             <mailto:[e-mail]>>
>             Subject: Re: [tsvwg] New Version Notification for
>             draft-morton-taht-tsvwg-sce-00.txt
>
>             Hi Greg,
>
>             I'm curious about the queue protection function in the LLD
>             document.
>             It seems to assume that a flow table is maintained to
>             determine if a flow
>             has the right to enter the low latency queue.
>
>             Can you give more details about that component? Or point
>             me to a reference?
>
>             Thanks
>             Luca
>
>
>             On 3/11/19, 5:29 PM, "tsvwg on behalf of Greg White"
>             <[e-mail] <mailto:[e-mail]> on
>             behalf of [e-mail]> wrote:
>
>                 TSVWG,
>
>                 I've posted a new informative draft that gives an
>             overview of the new Low Latency DOCSIS specification (see
>             links below).  This overview may be interesting to TSVWG
>             participants because it includes support for L4S
>             (https://datatracker.ietf.org/doc/html/draft-ietf-tsvwg-aqm-dualq-coupled)
>             and for the NQB PHB
>             (https://datatracker.ietf.org/doc/html/draft-white-tsvwg-nqb).
>
>                 Best Regards,
>                 Greg
>
>
>
>                 On 3/11/19, 5:05 PM, "[e-mail]
>             <mailto:[e-mail]>"
>             <internet-drafts@ietf..org
>             <mailto:[e-mail]>> wrote:
>
>
>                     A new version of I-D, draft-white-tsvwg-lld-00.txt
>                     has been successfully submitted by Greg White and
>             posted to the
>                     IETF repository.
>
>                     Name:  draft-white-tsvwg-lld
>                     Revision:       00
>                     Title:          Low Latency DOCSIS - Technology
>             Overview
>                     Document date:  2019-03-11
>                     Group:          Individual Submission
>                     Pages:          25
>                     URL:
>             https://www.ietf.org/internet-drafts/draft-white-tsvwg-lld-00.txt
>
>                     Status:
>             https://datatracker.ietf.org/doc/draft-white-tsvwg-lld/
>                     Htmlized:
>             https://tools.ietf.org/html/draft-white-tsvwg-lld-00
>                     Htmlized:
>             https://datatracker.ietf.org/doc/html/draft-white-tsvwg-lld
>
>
>                     Abstract:
>                        NOTE: This document is a reformatted version of
>             [LLD-white-paper].
>
>                        The evolution of the bandwidth capabilities -
>             from kilobits per
>                        second to gigabits - across generations of
>             DOCSIS cable broadband
>                        technology has paved the way for the
>             applications that today form our
>                        digital lives.  Along with increased bandwidth,
>             or "speed", the
>                        latency performance of DOCSIS technology has
>             also improved in recent
>                        years.  Although it often gets less attention,
>             latency performance
>                        contributes as much or more to the broadband
>             experience and the
>                        feasibility of future applications as does speed.
>
>                        Low Latency DOCSIS technology (LLD) is a
>             specification developed by
>                        CableLabs in collaboration with DOCSIS vendors
>             and cable operators
>                        that tackles the two main causes of latency in
>             the network: queuing
>                        delay and media acquisition delay.  LLD
>             introduces an approach
>                        wherein data traffic from applications that
>             aren't causing latency
>                        can take a different logical path through the
>             DOCSIS network without
>                        getting hung up behind data from applications
>             that are causing
>                        latency, as is the case in today's Internet
>             architectures. This
>                        mechanism doesn't interfere with the way
>             applications share the total
>                        bandwidth of the connection, and it doesn't
>             reduce one application's
>                        latency at the expense of others.  In addition,
>             LLD improves the
>                        DOCSIS upstream media acquisition delay with a
>             faster request-grant
>                        loop and a new proactive scheduling mechanism. 
>             LLD makes the
>                        internet experience better for latency
>             sensitive applications without
>                        any negative impact on other applications.
>
>                        The latest generation of DOCSIS equipment that
>             has been deployed in
>                        the field - DOCSIS 3.1 - experiences typical
>             latency performance of
>                        around 10 milliseconds (ms) on the Access
>             Network link. However,
>                        under heavy load, the link can experience delay
>             spikes of 100 ms or
>                        more.  LLD systems can deliver a consistent 1
>             ms delay on the DOCSIS
>                        network for traffic that isn't causing latency,
>             imperceptible for
>                        nearly all applications. The experience will be
>             more consistent with
>                        much smaller delay variation.
>
>                        LLD can be deployed by field-upgrading DOCSIS
>             3.1 cable modem and
>                        cable modem termination system devices with new
>             software. The
>                        technology includes tools that enable automatic
>             provisioning of these
>                        new services, and it also introduces new tools
>             to report statistics
>                        of latency performance to the operator.
>
>                        Cable operators, DOCSIS equipment
>             manufacturers, and application
>                        providers will all have to act in order to take
>             advantage of LLD.
>                        This white paper explains the technology and
>             describes the role that
>                        each of these parties plays in making LLD a
>             reality.
>
>
>
>
>                     Please note that it may take a couple of minutes
>             from the time of submission
>                     until the htmlized version and diff are available
>             at tools.ietf.org <http://tools.ietf.org>.
>
>                     The IETF Secretariat
>
>
>

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/