Re: [tsvwg] L4S status: #17 Interaction w/ FQ AQMs

[Greg White <[e-mail]>]

Hi Jonathan,

I believe this is one of the scenarios that was slated for experimentation to observe the behavior.   I don't recall hearing that experiment was conducted.  

But, I'm not sure I fully see what your concern is.  The concern I have in this scenario is that the sluggish response of the RFC3168 AQM (i.e. CoDel) in the fq_codel (or Cake) will cause the L4S sender to experience poor latency and in some cases underutilize the link.   Here is the explanation that I sent Sebastian some time ago.

[Sebastian]     But let's assume I have two flows one L4S one RFC 3168, and my link is already at saturation with each flow getting 50% of the shaper's configured bandwidth (and say 47.5% of the bottleneck's bandwidth) so the next two packets of each flow will be CE marked.

 [Greg] First off, the FQ scheduler will guarantee that both flows get 50% of the shaper capacity as long as neither flow's queue drains completely.   The legacy FQ AQM (i.e. not L4S-aware) will also operate on each queue independently, calculating a marking percentage for each one that aims to keep that queue's latency at the target value.  As long as the AQM does not cause either sender to overreact and cut its congestion window below 50% of BDP, then both will maintain packets in queue.   Since the FQ AQM in this case was designed with RFC3168 senders in mind, its response to queuing delay will be most appropriate for the RFC3168 flow, and that flow will experience packet marks and a latency over time trajectory as expected.  I don't see how the RFC3168 sender would be triggered to reduce its cwnd below 50% of BDP.  The L4S flow, on the other hand will experience a more sluggish increase in marking probability than would be ideal for L4S, and thus queuing delay will grow larger than would be ideal.  Depending on how much buffer space exists, the L4S flow might reach the tail-drop limit (which would trigger a large multiplicative reduction in cwnd, like the traditional Reno response), or the AQM would have ramped up the marking rate sufficiently to bring it under control (albeit more slowly than would be ideal).  It could be possible, due to the sluggish response of the legacy AQM, that the L4S sender cwnd undershoots 50% BDP and thus temporarily yields some capacity to the RFC3168 flow.  It seems as if your concern is that, when the RFC3168 flow cuts its cwnd to drain standing queue, that the L4S sender will shut out the RFC3168 sender on the access network link, and thus force the RFC3168 flow's FQ queue to drain completely. I don't see how this is possible.   Prior to the two CE marks in your example, both senders are (due to ACK clocking) sending at 50% of the shaper rate, plus one segment per RTT (each flow will have its own RTT). When the RFC3168 flow detects the CE mark and cuts its congestion window, it will (assuming no implementation of PRR) pause sending until sufficient outstanding ACKs are received, then it will resume sending at 50% of the shaper rate plus one segment per RTT.  Meanwhile, the L4S flow will not have cut its congestion window, and so will have continued sending at 50% of the shaper rate plus one segment per RTT, due to ACK clocking. There isn't a mechanism for it to steal capacity from the access network link and shut out the RFC3168 flow, unless (as stated previously) the RFC3168 flow's queue drains completely allowing the L4S flow to be dequeued at a higher rate (resulting in a higher ACK clocking rate, and thus a higher sending rate).  Remember that a TCP sender's response to a congestion signal (CE mark) is not to reduce its sending rate, it is to reduce its cwnd.  If the link is not yet at saturation, and a CE mark were to (unexpectedly!) arrive, then this would actually result in a decrease in sending rate.  But once the link is at saturation (the normal case for a CE mark to be generated), ACK clocking limits the sending rate to be just slightly greater than the link rate, and (assuming sufficient buffer exists) a CE mark triggers only a reduction in RTT (cwnd drops to a value that is still greater than BDP), via a pause in sending.  The long term average rate of the flow (assuming queuing delay remains bounded by either AQM or tail-drop) will be exactly equal to the link rate.

-Greg








On 8/11/19, 2:34 PM, "tsvwg on behalf of Jonathan Morton" <[e-mail] on behalf of [e-mail]> wrote:

    > On 11 Aug, 2019, at 5:03 am, Wesley Eddy <[e-mail]> wrote:
    > 
    > I created tickets in the TSVWG "trac" tool in order to help keep track of the individual things that look like they should be addressed in progressing L4S document set:
    > 
    > https://trac.ietf.org/trac/tsvwg/report/1?sort=ticket&asc=1&page=1
    > 
    > I'll try to update these based on the ongoing discussions, updates, etc., but it will make it very easy if you happen to mention the ticket numbers or some key words in threads and messages, when significant.
    
    Thanks for that; most of the issues seem to be well described and understood.  But there is one exception in the form of #17.
    
    The issue I was trying to raise is not that L4S has any particular difficulty with FQ, that it would not have with an equivalent non-FQ qdisc.  Rather I was responding to the proposed point on the slide *about* FQ, and identifying a situation where FQ could be defeated by L4S' poor behaviour in an RFC-3168 environment.
    
    Since this is a relatively subtle point, I'll try to expand upon it here to avoid confusion.  Fundamentally though, this is strongly related to points A, F & G, which is why I raised the four together.
    
    First, let's define a non-issue.  If the first significant bottleneck that an L4S flow encounters is the true one, and that bottleneck is equipped with FQ, then the worst effect that L4S can have on the network is self-inflicted.  The ability of FQ to isolate badly-behaved flows from other traffic traversing the same link is well-known, and there is nothing special about L4S that would defeat that property, on its own.  This is what I meant when I described FQ as being "pretty robust".
    
    Where the problems arise is where the true bottleneck, supporting FQ-AQM, is preceded by a dumb FIFO (or, without very much loss of generality, a single-queue RFC-3168 AQM).  This is what I might refer to as "Sebastian's topology", and reflects the current practice of placing bufferbloat mitigation in the CPE downstream of the last-mile link.  Even with current RFC-compliant traffic, managing overall latency in this topology is challenging; in practice it requires a reservation of some percentage of the last-mile link capacity to ensure the latter's dumb queue drains into the CPE's smarter queue.
    
    For example, the IQrouter automatically measures the last-mile link capacity, using speed tests, and sets its own FQ-AQM shaper to about 95% of that measurement.  But this still means that if a sudden flood of traffic arrives, only a twentieth part of that traffic will immediately collect in the IQrouter's queue, as opposed to the dumb queue at the last-mile link.  The effectiveness of the FQ-AQM thus depends on avoiding large floods of that sort, by quickly and effectively signalling congestion to external senders; only when the dumb FIFO is kept empty do the flow isolation properties of FQ take effect.
    
    So you can see from this why the 4-second ramp up needed for Codel to effectively control an L4S flow with a relatively short RTT is problematic.  Indeed, as the RTT gets shorter, the time needed to gain control using a Codel ramp gets longer, because the signalling frequency needs to be proportionately higher.  Luckily, there is a practical lower limit to this trend in the form of the Codel 'target' parameter, which sets a floor on the effective RTT that needs to be controlled.
    
    Conversely, an RFC-3168 compliant flow responds to a single CE mark after 1 RTT with a Multiplicative Decrease, so its response actually gets *faster* with shorter RTTs.  This is a property that Codel relies on to make its performance less sensitive to the precise choice of 'interval' parameter relative to path RTT.
    
    I can walk you through the maths for the Codel response if you like.  It's reasonably straightforward, but might not be obvious from a cursory reading of the raw RFC.
    
     - Jonathan Morton