Re: [tsvwg] path forward on L4S issue #16

Sebastian Moeller <> Wed, 17 June 2020 13:27 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 580113A0486; Wed, 17 Jun 2020 06:27:03 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.649
X-Spam-Status: No, score=-1.649 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id dbcVuT3_2bwA; Wed, 17 Jun 2020 06:27:01 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id E0E8B3A0442; Wed, 17 Jun 2020 06:27:00 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;; s=badeba3b8450; t=1592400414; bh=EJVjUG+147mYVwSZ7BHN5aWb7atgGC4JSdB7WwaiIeQ=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=eqQkhFIRQiXTXgKH9XgVZjo2hHbROuUtiH/wfAGb+12RebynYYvrfmNM8+YreQ8fU iKYfGIRb1GLXK5UzHGT0sGk1a6xLbvLS8oFPGV6X91ez/b+t5m6ZAD8fsqOF/v6Fff FihD55eRiMSor4ywO6a368SVUf3FB76ej5PZn9ew=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from [] ([]) by (mrgmx105 []) with ESMTPSA (Nemesis) id 1MSKy8-1jJ2t408Lo-00SexT; Wed, 17 Jun 2020 15:26:54 +0200
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.14\))
From: Sebastian Moeller <>
In-Reply-To: <>
Date: Wed, 17 Jun 2020 15:26:52 +0200
Cc: "Holland, Jake" <>, Jonathan Morton <>, Ingemar Johansson S <>, "" <>
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <>
To: Ingemar Johansson S <>
X-Mailer: Apple Mail (2.3445.104.14)
X-Provags-ID: V03:K1:8TNnf4guvi6mKB407WbCxfegSGG5m1o754LT0WfYwzLJeVwdyOW NMRE4yUUHsfLVjtFM2XLt2X4G22JY+GzU6is8MhnxnifrEXuIEtMJ2kveAEQmORJ3x6XEF2 CZwy7afbalo+9KEOTIUMoXOHJD6IB3y3FxYi17eu3pdJedeWvcZK0TU5v9KsQ/3oC2PHEdQ gxrtWj7D6bmaU0rfjN5+A==
X-UI-Out-Filterresults: notjunk:1;V03:K0:FinuWK1glgo=:3itwAEJve75Vw2q9GCVawv SPuInR37qpwCfsGxRXaVsRI+oQ59M01HKB7xmF0nSAE++PoC4fsFa8V+H7UUZFLwqFD9UgnBv 7kp1kBKdLxT6QJcS2y/sdv/5bgWK+ClqzeevrN9FhS5GbV5Xu5BcsPF/dJIN43PTDXyuBMjG9 B7UA/E7y6GqqF6rNPaIw3cfSnYBTQItfTL2pLOYsofkMz+6eG3og9kPHNM+iylittnySahf4n M8qFMVdUf1XxHQ06BRWpRujAyEPOkYQtAfoAitFwCFttD8VbKq72VyzRSBMtTU6MuMPN0vZlz 2SQVhiZaZHayT2Z8Dx42kO9hQVZ/he4jMxStM+1e4z4FiALF9wdPKmG3FG9MVem0DI3Sr46SA r2U74iSt3GhsYbL7BdTtZnBJWW932bcy72Na5ZvZN7nF8xxcpjQY3gt6PuOHI1l/9sHHwiHjs NFvtDagYa0csvtI4Tiq/MlgnJLNqxchweXBcnhCDapbZ1NT8k5d9idXBAk1bHG9CPq0zwcqJR ZKLQG+398o7Z0pu7JQ0Vm14yUVMY8mZSgTKpHhMpy1Rr9DpYNcpQQ+GwxASKYlDpKNukenP/8 vjpBOhlDgKSm+ge0YVeTr0Fvkx2lckqegJ2UPbYCOuJxqWWuyzgh5LH3NN8nilJD4H4hIZFSW UhIDH8+rbhiEAFR3Mo1ADH3KRvSbGLTElt+XRa2eDXklAXR9f5yA4gxQOQ8twhMS3fK75wvRT o2fTmBtsz5cEagX6qtAo0PADfrZvPKzbPmXHg+347n02rBQRlnx/PaVdmETvcqUT8a/7qs6cZ X8t+S3N670iL7WNFQQ+a7xtBpsXJfmaiE6b0PAiVYH4m6R8MP+LLIKjtolGmzrXskMSUr9A+U YQvN0sRflA+UnERb1UEbuVJsvFf9wwfKFaXAoW7HwO71cKDPiaB2gmHMsTH8iINcGLWMlgo+w m6bS1IEIVealScu63F/3G521gxSd8uzshkjLfilaWDocoVHMpfvO/cAMv3sUN+UfpcgrAxxvZ 9+WlUxOJuSoQ9oGHrSBSmV6g9DWLwoZibNNcp9H5digk9PVeIbGJLyqIKSkaA9Dt3pt5lTJuW id1FssO3BNVPDwHq3+6aqH2bvkrY7uZ2nMW1SzXRgPZWSa2YHzJd306eFeCrhY+zTPCKLfIxN XXDHxf13YPrAnE/8GsoTzkwZcgOeiCJSvNJ5yRjrkm11tLiGA4AazPu4DRh9GePj/q7jo=
Archived-At: <>
Subject: Re: [tsvwg] path forward on L4S issue #16
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 17 Jun 2020 13:27:04 -0000

Hi Ingemar,

more below in-line.

> On Jun 17, 2020, at 15:20, Ingemar Johansson S <> wrote:
> Hi Jake
> And thanks for the input. 
> Given all the input sofar my impression is that It is difficult to say how serious issue 16 really is. 
> Surely there is a possibility that L4S flows can get an potentially unfair advantage over a congested node that is  RFC3168 capable (and ECN enabled). The question is here when things are deemed as unfair and when starvation is a fact, opinions differ. 
> Another question is to how large extent there will be legacy RFC3168 ECN capable (and ECN enabled ) bottlenecks that will "never" be updated or replaced. This question is currently difficult to answer. 
> Yet another aspect is how successful it will be to implement RFC3168 bottleneck detection e.g. in TCP Prague, I am quite confident that these will not always be successful.

	[SM] +1

> But then, given the current deployment situation and possibility to update nodes in the networks (OpenWrt routers appear to support automatic updates)

	[SM] Please accept my apology for not being clear on that topic in the past. Stock OpenWrt does not offer automatic updates, not even notification of update availability at all. There are some vendors, like Evenroute and Turris, that base their own products on top of OpenWrt that do offer automatic updates and as far as I can tell also notifications. But I have no reliable numbers on the prevalence of either stock OpenWrt or update-enabled versions in the field. I also add that a tiny bit of research should allow to confirm this.

> I am not convinced that these need to be 100% bulletproof either.

	[SM] 100% is not achievable, but what we currently have is an effort just designed to get past the objections int the WG instead of actually trying to tackle the issue for real.

> /Ingemar
>> -----Original Message-----
>> From: Holland, Jake <>
>> Sent: den 15 juni 2020 20:36
>> To: Ingemar Johansson S
>> <>rg>; Jonathan Morton
>> <>
>> Cc: Ingemar Johansson S <>om>;
>> Subject: Re: [tsvwg] path forward on L4S issue #16
>> Hi Ingemar,
>> I posted some summary numbers from some observations I made here:
>> VAZM/
>> I didn't say anything about France or Norway, but the numbers I have seen
>> were roughly in line with the numbers Jonathan gave.
>> The "growing" claim also looks tentatively well-supported, because I've seen
>> 2 new ASNs since then with a high count of CE-marking paths.
>> HTH.
>> Best regards,
>> Jake
>> ´╗┐On 6/10/20, 6:59 AM, "Ingemar Johansson S"
>> <> wrote:
>> Hi
>> Do you have any public sources that confirm the numbers you quote below
>> ?. I tried to look up data on this but it surely is not easy.
>> Which foras are the vendors that manufacture CPEs active in (if any) ?.
>> As regards to endpoints implementing RFC3168, do you refer to servers and
>> clients or something else?. My interpretation is servers and clients and I don't
>> believe that they are central  to this discussion, or do I miss something ?. I
>> understand that traffic shaping on outgoing interfaces can be applied in a
>> Linux host but don't see why they become a problem especially as there are
>> qdiscs that support dualQ.
>> /Ingemar
>> PS. I am not yet trying to convince anybody, I am just trying to find the
>> answers.
>>> -----Original Message-----
>>> From: Jonathan Morton <>
>>> Sent: den 10 juni 2020 14:58
>>> To: Ingemar Johansson S <>
>>> Cc: Sebastian Moeller <>de>;
>>> Subject: Re: [tsvwg] path forward on L4S issue #16
>>>> On 10 Jun, 2020, at 11:35 am, Ingemar Johansson S
>>> <> wrote:
>>>> 1) How widely are  RFC3168 ECN capable AQMs deployed _and_ enabled
>> ?
>>> Enough for the effects to be visible on about 0.1-1.0% of current
>>> Internet traffic - and rising steadily.  That's some combination of
>>> endpoints
>> negotiating
>>> ECN support, and middleboxes implementing an AQM, but not necessarily
>>> both on the same path.  Deployment of ECN-enabled AQM is very high
>>> across a small number of forward-thinking ISPs, including a very large
>>> one
>> in
>>> France and a major one in Norway.
>>> Basically every endpoint out there today implements RFC-3168 ECN, and
>>> every Linux endpoint runs an ECN-enabled AQM by default on each
>>> Ethernet interface.  If Linux and/or Microsoft had decided to switch
>>> it on by
>> default a
>>> decade ago, rather than hiding it behind a non-default configuration
>>> flag,
>> we
>>> would not be having this conversation, because the deployment numbers
>>> would be too big and obvious to ignore.  It's been a long, slow
>> chicken-and-
>>> egg situation which the major consumer ISPs have essentially refused
>>> to
>> help
>>> resolve.
>>> But now there are commercial, off-the-shelf CPE products which anyone
>>> can buy and install, and which implement the sort of bufferbloat
>>> mitigation measures that most ISPs have not.  I've found at least one
>>> of those on the physical retail shelf, here in Finland.  I don't have
>>> sales numbers, but
>> they
>>> must account for most of the deployment on ISPs that haven't deployed
>>> it themselves.
>>>> 2) If they are deployed _and_ enabled, can/will they be updated ?
>>> Some can and will.  But some is baked in hardware so would be
>>> difficult to update, and some is managed by people/vendors who do not
>>> know or care about L4S, so would tend to keep using the old system
>>> which is RFC- compliant today.  When was the last time your CPE
>>> devices got a firmware update?  So L4S has to be able to cope with the
>>> existence of such deployments in at least the medium term.
>>> Frankly, if you believe that tunnels are an intractable problem, then
>>> this
>> is
>>> just as bad.  So please finally drop this argument; you're not
>>> convincing anyone who matters.
>>> - Jonathan Morton