Re: [tsvwg] L4S status: #17 Interaction w/ FQ AQMs

Jonathan Morton <> Sun, 11 August 2019 20:34 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 956C7120F3B for <>; Sun, 11 Aug 2019 13:34:39 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.749
X-Spam-Status: No, score=-1.749 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id Ur6LGXjJ7q2M for <>; Sun, 11 Aug 2019 13:33:40 -0700 (PDT)
Received: from ( [IPv6:2a00:1450:4864:20::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 91518120B9C for <>; Sun, 11 Aug 2019 05:05:19 -0700 (PDT)
Received: by with SMTP id t3so7428951ljj.12 for <>; Sun, 11 Aug 2019 05:05:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=xsxBS/I2QPEaP0FJc69JtEJF0MzUw7RhZu6HUBCFSoo=; b=IBZgxofB71Yhx01j8SRrBEkSU9LRL1h2L4oMFqoamnXijM5NeYxX6hh06Yc7yrK/2I 9AuP2wWiyWi7fTFcVkM59nGty6LODJsicTWBqEofqVS7nPtjPk7hQ0aB0bGlyQIAM93Y xBXjm1jjPoXh7n4yrGTABDP2EsLADE4/QL2NulYx26BsmYrcUBr9YSyRT75wvzwHnMpd dWQ+Ajxer8yYeb3cK0V1SxVZeoJkfjn+oIP8nhbMaEX+eV2NW66CyeYF/gqJqVm8JGek D/IAgxkivZzzCHY6j9RMQYKvUb3kKkIWEdrqa0S40CZ9U1XDs0UbnfrRnGthqfMAvn5U YXig==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=xsxBS/I2QPEaP0FJc69JtEJF0MzUw7RhZu6HUBCFSoo=; b=ibBxFEst31q0IbEOtwUCzbOkV6S7WZws00SecezlNnBoPn5MjmVrpOp7DlOSwJR5Ka +j4OI4MsiQRsWALL958bCw3sWZEvYVJWzWYFyxlfdCuOHAmc6WRfnnMeVPC7s2WYpsRn zrincaPcAwJVAqrNQhDtDzbVJ9BtFbGWUrKRivgkgWSJCCdTluAkqOvqiuaPqs4HlcLg pvUBWmhl5zaEZj1NeAEklwH+m7uBKqoYPN3AGHvvJELL9apbSnOBKZJxeqbyKRbXnd6k XlaJ2J8hhVFsr8nWI/CLlEXaab1exSlivY2bJ6nxL0iMfoBanYMYBAN5+17q8QX0zm9k N2Hg==
X-Gm-Message-State: APjAAAU43O4pwpJWx1tuq6r/f8ADFd0Bl65ZiSgVFI766KW31bhDGvuM EZXSh0imXzYcu2E0bZ8t4NtFMVMl
X-Google-Smtp-Source: APXvYqxw5bP6TqZGJ0QnjQSbwSKlE4wgepYe5hC+HaMabgTPFniEf9njkbNBzrc3jmp5d9Q+trv32Q==
X-Received: by 2002:a2e:9ed1:: with SMTP id h17mr3847954ljk.130.1565525117660; Sun, 11 Aug 2019 05:05:17 -0700 (PDT)
Received: from jonathartonsmbp.lan ( []) by with ESMTPSA id b11sm6554597lfi.91.2019. (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 11 Aug 2019 05:05:17 -0700 (PDT)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
From: Jonathan Morton <>
In-Reply-To: <>
Date: Sun, 11 Aug 2019 15:05:15 +0300
Cc: "" <>
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <>
To: Wesley Eddy <>
X-Mailer: Apple Mail (2.3445.9.1)
Archived-At: <>
Subject: Re: [tsvwg] L4S status: #17 Interaction w/ FQ AQMs
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sun, 11 Aug 2019 20:34:39 -0000

> On 11 Aug, 2019, at 5:03 am, Wesley Eddy <> wrote:
> I created tickets in the TSVWG "trac" tool in order to help keep track of the individual things that look like they should be addressed in progressing L4S document set:
> I'll try to update these based on the ongoing discussions, updates, etc., but it will make it very easy if you happen to mention the ticket numbers or some key words in threads and messages, when significant.

Thanks for that; most of the issues seem to be well described and understood.  But there is one exception in the form of #17.

The issue I was trying to raise is not that L4S has any particular difficulty with FQ, that it would not have with an equivalent non-FQ qdisc.  Rather I was responding to the proposed point on the slide *about* FQ, and identifying a situation where FQ could be defeated by L4S' poor behaviour in an RFC-3168 environment.

Since this is a relatively subtle point, I'll try to expand upon it here to avoid confusion.  Fundamentally though, this is strongly related to points A, F & G, which is why I raised the four together.

First, let's define a non-issue.  If the first significant bottleneck that an L4S flow encounters is the true one, and that bottleneck is equipped with FQ, then the worst effect that L4S can have on the network is self-inflicted.  The ability of FQ to isolate badly-behaved flows from other traffic traversing the same link is well-known, and there is nothing special about L4S that would defeat that property, on its own.  This is what I meant when I described FQ as being "pretty robust".

Where the problems arise is where the true bottleneck, supporting FQ-AQM, is preceded by a dumb FIFO (or, without very much loss of generality, a single-queue RFC-3168 AQM).  This is what I might refer to as "Sebastian's topology", and reflects the current practice of placing bufferbloat mitigation in the CPE downstream of the last-mile link.  Even with current RFC-compliant traffic, managing overall latency in this topology is challenging; in practice it requires a reservation of some percentage of the last-mile link capacity to ensure the latter's dumb queue drains into the CPE's smarter queue.

For example, the IQrouter automatically measures the last-mile link capacity, using speed tests, and sets its own FQ-AQM shaper to about 95% of that measurement.  But this still means that if a sudden flood of traffic arrives, only a twentieth part of that traffic will immediately collect in the IQrouter's queue, as opposed to the dumb queue at the last-mile link.  The effectiveness of the FQ-AQM thus depends on avoiding large floods of that sort, by quickly and effectively signalling congestion to external senders; only when the dumb FIFO is kept empty do the flow isolation properties of FQ take effect.

So you can see from this why the 4-second ramp up needed for Codel to effectively control an L4S flow with a relatively short RTT is problematic.  Indeed, as the RTT gets shorter, the time needed to gain control using a Codel ramp gets longer, because the signalling frequency needs to be proportionately higher.  Luckily, there is a practical lower limit to this trend in the form of the Codel 'target' parameter, which sets a floor on the effective RTT that needs to be controlled.

Conversely, an RFC-3168 compliant flow responds to a single CE mark after 1 RTT with a Multiplicative Decrease, so its response actually gets *faster* with shorter RTTs.  This is a property that Codel relies on to make its performance less sensitive to the precise choice of 'interval' parameter relative to path RTT.

I can walk you through the maths for the Codel response if you like.  It's reasonably straightforward, but might not be obvious from a cursory reading of the raw RFC.

 - Jonathan Morton