Re: [tsvwg] feedback and thoughts L4S / SCE

Pete Heist <> Sun, 22 November 2020 19:26 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 2A7133A0B0D for <>; Sun, 22 Nov 2020 11:26:45 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -0.199
X-Spam-Status: No, score=-0.199 tagged_above=-999 required=5 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id qIgfKuQ8myOa for <>; Sun, 22 Nov 2020 11:26:43 -0800 (PST)
Received: from ( [IPv6:2a00:1450:4864:20::32f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 99BFF3A0B0E for <>; Sun, 22 Nov 2020 11:26:43 -0800 (PST)
Received: by with SMTP id s13so15321724wmh.4 for <>; Sun, 22 Nov 2020 11:26:43 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=google; h=message-id:subject:from:to:cc:date:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=mEK0CeeBiOk4IaqWRmLY2Ec22cWUjSHmbV/V0KT4ptQ=; b=hrBO8UsQmC3aUvh515W0YGx7qNZtbkRYokVS+CJeSDVFd5Px/GJPkCG5VJcbFv8mic PsOeIQ6j6kjbmX0OqW/YoseGBkgpYS2osLoFIpE6ULjg+hNEXqWjAH/H+AxNLCvVefu7 jOhGhM7/bKRwgWwhgvy7aOHRI+YMk3jdWi4YoPZT3cZkdAtUpYK0oeT+Ctehg8kKnMNM XuLAk8zDXkA7wCApUh1W0xukcdQoUJP6lgyTMy5HIZ0netESe4CffjWOP8E5ueHbzs+B DQIU+VF7LC9DO5VZy3Yi9JfxkuU0e2sE2ncXBUEToeO0m9kmmRQ5TJ8x4uB4nlub0KNB 3cCg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=mEK0CeeBiOk4IaqWRmLY2Ec22cWUjSHmbV/V0KT4ptQ=; b=R/pb7Pzb6LFxdctLeH4UirOOSLV5LO6j9Bxs6q9os8GpIYW21+uH4lLpcKOtYv4h13 MTIydtvJnB/RkS6fXMsrkz6etARKAKbegdW4q4gbppXk43uQTEv//7aXVcQqxNhaDpGq Z/v63VBigkEqCar/hZCeRiPpGjRTV80qemJn+XAHBPpFsFY86p4+r4gqn6hRHddHHjgj 1pLhHLUQ7S/RdY47pnbWIoylsD9Ipv4ZzsdyYF+yT/1rBucgcftsfkbm3wR9oztSoeZU gAsgazHd/WSI0Ep0MQsdtlcGDpOVChVr0juvgd+ogi8hL1efDLJpwtIvq7yxmcq4LIZo TbVA==
X-Gm-Message-State: AOAM533yElb6zXHw0jt0BUIunquZYvUFds9wwnH+rsJCC29gG0cnpZ/a K7gc4bNXB/dM//gtCkcJg8Kv1AdjeTGoIw==
X-Google-Smtp-Source: ABdhPJwDVGoR3AzRVVYKZCSVvjQrPJ6V3O2RkircGL/ekmeF+rrKEo0jjKX+nFEgntgspVkv9iWz9Q==
X-Received: by 2002:a1c:3907:: with SMTP id g7mr20010447wma.176.1606073201386; Sun, 22 Nov 2020 11:26:41 -0800 (PST)
Received: from [] ( []) by with ESMTPSA id f20sm11738415wmc.26.2020. (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 22 Nov 2020 11:26:40 -0800 (PST)
Message-ID: <>
From: Pete Heist <>
To: Mikael Abrahamsson <>
Date: Sun, 22 Nov 2020 20:26:39 +0100
In-Reply-To: <>
References: <>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.38.1
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Archived-At: <>
Subject: Re: [tsvwg] feedback and thoughts L4S / SCE
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sun, 22 Nov 2020 19:26:45 -0000

Hi Michael, a few replies below...

On Fri, 2020-11-20 at 14:33 +0100, Mikael Abrahamsson wrote:
> Hi,
> I'm trying to catch up on the work has been done, but I thought I'd
> give 
> some thoughts / feedback on what I see:
> "L4S transports experience intra-flow latency spikes at RFC3168 
> bottlenecks, particularly with the widely deployed fq_codel"
> FQ_CODEL isn't widely deployed. Yes, it's enabled in the linux kernel
> but 
> very seldom at the bw chokepoint of the connection. I do not agree
> with 
> the above statement at all. Thus any benchmarking vs FQ_ anything is 
> invalid. What's mostly used today are typically simple FIFOs,
> sometimes 
> diffserv enabled with multiple FIFOs.

I do accept the criticism that the phrase "widely deployed" was only
qualified with a vendor list. So in response, I've contributed some ECN
data under a new thread, and I'll review my part of the writeup later
for any language that I think isn't supported well enough. Just asking
for some patience with that. :)

> This means the tunnel argument against any FQ or non-FQ AQM is
> invalid. I 
> also doubt we'll see widely deployed FQ going forward because of the
> cost 
> of doing this in hw (and most devices are hw accelerated).

However, it's still used, at the edge, in backhauls, and built into
Linux WiFi drivers. The Preseem QoE platform is indirect evidence of
fq_codel use in backhauls, and we know there are ISPs that manage
queues in their backhauls with custom scripts using fq_codel. Instead
of delving into various products, I suspect that trying to gather ECN
stats at relevant places on the Internet might be more fruitful(?)

It's a known issue that any non-FQ RFC3168 bottleneck leads to
domination of non-L4S flows by competing L4S flows, tunnel or not. I'll
post another test scenario with several AQMs showing that once I write
it up.

> "Deployments of fq_codel
> The fq_codel qdisc has been in the Linux kernel since version 3.6
> (late 
> 2012) and is now in widespread use in commercial routers (e.g.
> Ubiquiti 
> EdgeMAX and UniFi products)"
> This is misleading. Yes, EdgeMAX supports FQ_CODEL if you turn off 
> hw_acceleration, bringing down the performance to 10% or less of what
> the 
> hw acceleration yields. I do not agree that it's in "widespread" use.

> I'd gladly take data to prove me wrong, that FQ_CODEL is in
> widespread use 
> at Internet bw chokepoints. I'd be surprised if it was enabled on
> more 
> than 1% of residential connections.

Let's see if we can come up with more data.

>  SCE proponents 
> have to stop arguing that we'll have FQ_ anything near term, or that
> this 
> is in wide use. It's not.

We're working on SCE and know where we're headed at least, but it isn't
on the WG's table now.

> We're spending the last bit in the IP header and we'd better get it
> right. 
> In order to do that, I'd like to see realism and pragmatism in the 
> reasoning.