Re: [tsvwg] Comments on L4Sops [was Adoption call for draft-white-tsvwg-l4sops ...]

Pete Heist <> Tue, 16 March 2021 11:06 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 3EEFE3A0AB4 for <>; Tue, 16 Mar 2021 04:06:45 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.099
X-Spam-Status: No, score=-2.099 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id QnSWoDpkgqwh for <>; Tue, 16 Mar 2021 04:06:43 -0700 (PDT)
Received: from ( [IPv6:2a00:1450:4864:20::42f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id D3C983A0A9E for <>; Tue, 16 Mar 2021 04:06:42 -0700 (PDT)
Received: by with SMTP id a18so10198986wrc.13 for <>; Tue, 16 Mar 2021 04:06:42 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=google; h=message-id:subject:from:to:date:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=L0Yz/IhrlRojYvbSzs9AOjHaASG8aB3TcBgH4u5qWPc=; b=g58Cg7xUMP29jmxdwXYy1Fm9e8Pgc7n0jKaNxip2rQ86h8CSCxuhxjQXuFE4gZNXxn 5QpDc8nbEoFN/M2ET9c1KpXxdgWtO35NyaY5SahFgieZrI0BPoXT59Dez/bwSlVU0Dbq /31k+piCeDIsKgGUQluF6+zyVD74oMn9ysrwPs54Hl1YRdcbWP4DjY/l3yCitV5dGu4N uP83BJEDmimxV1PPWTi+QMTQ0qMvFfLciQGAaJfeM67O9ZLON3YbyjIyAJAkttBWxfG/ oUOUz3DI7SZ/E/lOAuzeZh/b82jdTVH5HTBjcm3egwxE6mWmM56Qz7IxWlOMCMeOzpEw vyxw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:message-id:subject:from:to:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=L0Yz/IhrlRojYvbSzs9AOjHaASG8aB3TcBgH4u5qWPc=; b=UWNpu6E0Dae8aX+kJKJtKgby6jRMNpXxZvxC5XzQkBmjjRyEcQgLoXel867nWMNqFf r/0avMG3LwQnxLRaaAkCkTVPPceL1sdl+rdEDuMNZGymf70ncACH69+Mv5mj+olZ6S1H 2p14aPdN9fM9oHXnphyn/RiSJMSzyweOS/3JSWxRATcv8PkloOLjXg80aEv47q478Gqg EZNcUsKO7LNxkQWLiUW+KgsqOquKWNScXt60Ovv3/Heju20zYpouiKcixjCMv+yqcU7Q UhpWnAwOxb28KlCJagNc47dAteu6VVDMHQw5bHs8eEFPRAIPVOqqWnzjnDH9LpTTM5U6 gnzA==
X-Gm-Message-State: AOAM533E6CY6Ek38wXaUk6nQO4GAZ0i/UP0YrryCXbQ+manvrHQzHqcU zikntEtEDiUIdrfKDgmrHKQoNuRV2AF5+Q==
X-Google-Smtp-Source: ABdhPJyYqj42bH3PPmroher8UMSPK0MrbcIS4xKAiPaieoZoMyPd5O7y4KLFVbDzwzXdRXptNP7sfw==
X-Received: by 2002:a5d:56c9:: with SMTP id m9mr4148756wrw.422.1615892799679; Tue, 16 Mar 2021 04:06:39 -0700 (PDT)
Received: from [] ( []) by with ESMTPSA id p12sm21408946wrx.28.2021. for <> (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 16 Mar 2021 04:06:39 -0700 (PDT)
Message-ID: <>
From: Pete Heist <>
To: "" <>
Date: Tue, 16 Mar 2021 12:06:38 +0100
In-Reply-To: <>
References: <>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.38.4
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Archived-At: <>
Subject: Re: [tsvwg] Comments on L4Sops [was Adoption call for draft-white-tsvwg-l4sops ...]
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Tue, 16 Mar 2021 11:06:45 -0000

One comment below... [PH]

On Mon, 2021-03-15 at 23:41 +0000, Greg White wrote:
> Hi Jonathan,
> I think you've misinterpreted the intent of the draft a bit.  Maybe
> there is an opportunity for further clarification in the draft
> itself, but fwiw I'll try to explain it better here.  
> Accurate real-time, in-band detection of RFC3168 by an L4S congestion
> controller is what you are referring to in your first item labeled
> 1.  If such an algorithm were generally agreed to be simple and
> reliable, then it could be argued that there wouldn't need to be an
> L4Sops draft. The detection algorithm that was studied earlier was
> found to suffer false positives at low data rates or high RTTs
> (though the referenced 'fallback' paper does discuss another approach
> for in-band detection that warrants looking into).  While the draft
> intends to include in-band detection as a potential option, it
> recognizes that it isn't the only solution.  I think there is
> confidence that out-of band tests can be reliably performed that
> determine whether RFC3168 bottlenecks exist, and this information can
> be used to make decisions about deployment and use of L4S.
> Further, I don't agree that network operators are always "innocent
> bystanders" in your taxonomy.  Some might be, but others are
> definitely not.  I think it is totally appropriate to provide
> guidance to network operators that want to participate in L4S but
> have RFC3168 deployed in their networks.    

On Tue, 2021-03-16 at 07:17 +0200, Jonathan Morton wrote:
> Since the harm is primarily caused to "innocent
> > bystanders" rather than "involved participants" or "interested
> > observers", the acceptable level of harm and risk is especially
> low,
> > and the mitigations need to be correspondingly robust.

[PH] I think Jon was referring here to end users as opposed to network
operators. End users are "innocent bystanders" because they may be
using a 3168 AQM without knowing it. Examples:

* Their ISP uses fq_codel in its network.
* They bought a WiFi AP that has fq_codel in the driver.
* They're using a public WiFi AP that has fq_codel in the driver.
* They enabled an AQM in their home router.

The draft can't immediately help those users, because it will take
years for the existing AQMs to be updated, reconfigured or undeployed.


> -Greg
> On 3/15/21, 4:35 AM, "tsvwg on behalf of Jonathan Morton" <
> on behalf of> wrote:
>     I do not think this document is ready for adoption in its current
> form.  Let me explain why, and suggest some ways it could be
> improved.
>     L4S has a fundamental incompatibility with conventional AIMD
> traffic in the presence of RFC-3168 ECN AQMs, just like DCTCP upon
> which it was based.  L4S therefore requires mitigations to ensure
> that the harm caused by this incompatibility is minimised to an
> acceptable level.  Since the harm is primarily caused to "innocent
> bystanders" rather than "involved participants" or "interested
> observers", the acceptable level of harm and risk is especially low,
> and the mitigations need to be correspondingly robust.
>     However, robust mitigations are not what l4s-ops currently
> describes.  Most of the measures described fall into three
> categories:
>     1: Reliance on detecting an RFC-3168 AQM and disabling the L4S
> behaviour, using heuristics that have not yet been shown in a
> reliably working state, even under lab conditions.  It is impossible
> to state that such a heuristic can be relied upon until such a
> showing has been made.  A previous attempt at implementing such a
> heuristic was unsuccessful and is now disabled by default in the
> reference implementation.  Hence, the reliability of such a heuristic
> would necessarily be a subject of the experiment, not the primary
> safeguard.
>     2: Requirements placed upon "innocent bystanders" to avoid the
> harm, mostly by reconfiguring, replacing, or disabling their RFC-3168
> AQMs (sometimes in an RFC-ignorant manner).  This is obviously
> unworkable, since by definition "innocent bystanders" are unaware of
> the experiment, and even if made aware, are disinterested in doing
> work to accommodate it.
>     3: Recommendation to deploy L4S hosts on networks that have been
> prepared to receive it.  Which is a step in the right direction.  But
> this is not accompanied by a corresponding requirement to *contain*
> L4S traffic to each prepared network.  Without such a requirement, it
> would be very easy for L4S hosts on different networks, which may
> individually have been prepared, to communicate over the path between
> those networks that has *not* been prepared, and upon which the risk
> of disrupting bystander traffic therefore exists.
>     It is perhaps noteworthy that gaps in the second and third
> classes of mitigation are proposed to be covered by the first class
> of mitigation.  I also note that there is still an assertion in the
> text that RFC-3168 AQMs are "rare", which is refuted by recent data. 
> Finally, in the context of a CDN-ISP pairing for an experimental
> deployment, the ISP subscribers' LANs and WLANs are technically
> separate networks that would be difficult to "prepare" for L4S in
> advance; it would be wise to consider the ramifications of that.
>     I also note in passing that a modification of tunnel
> encapsulation semantics is also proposed.  Given that tunnel
> implementations are more diverse than RFC-3168 AQM implementations, I
> also consider this unlikely to be practical, though I haven't studied
> in detail whether it would be effective if achieved.
>     I am currently aware of four theoretical methods of robustly
> mitigating the risk posed by L4S.  I think that l4s-ops would be
> considerably improved by proposing that at least one of them be
> employed as a prerequisite to the L4S experiment actually taking
> place:
>     1: Develop, implement, demonstrate, and open for scrutiny an RFC-
> 3168 detection heuristic that is reliable and prompt enough to serve
> as a primary safeguard for the experiment.  In my opinion this will
> be difficult and will take significant time, but is not impossible to
> achieve.
>     2: Deprecate RFC-3168, or amend it to remove drop-equivalent
> marking of ECT(1) packets, and require the removal of all unmodified
> ECN AQMs from the Internet.  This is unlikely to get much support
> given the increasing deployment rates of RFC-3168 AQMs at the present
> time.  In any case it would take a very long time to eliminate
> existing RFC-3168 AQM deployments at Internet scale, so I consider
> this impractical.
>     3: Explicitly contain L4S traffic to networks that have been
> prepared or designated for the experiment.  That could be done by
> marking all L4S traffic with a designated DSCP at origin, and
> blocking traffic carrying that DSCP from traversing border gateways
> into unprepared networks.  This has the effect of making users and
> administrators of these networks at least "interested observers" and
> isolating L4S traffic from "innocent bystanders".  Within the
> designated networks, observing the practical interactions between L4S
> and conventional traffic would be part of the experiment.
>     4: Redesign L4S to shift the risk burden away from "innocent
> bystanders".  The most obvious way to do so is to implement
> unambiguous signalling by the network, so that the receiver knows for
> certain whether it is receiving congestion signals from an RFC-3168
> AQM requesting an immediate MD response, or from an AQM of the new
> type requesting a new type of response.  The risk of performance
> trouble is then restricted to network nodes that produce the new
> signals and transport endpoints that understand them - in other
> words, to the relatively small number of "involved participants" who
> have the knowledge and incentive to study the problem and find
> solutions.  The incentives are thus aligned correctly and risks are
> not "externalised".
>     The SCE proposal does exactly that, in a manner that is totally
> transparent to existing RFC-3168 endpoints and middleboxes.  It
> becomes practical, for example, to use a Differentiated Services Code
> Point to differentiate a low-latency service onto a second bearer and
> provide a single-queue SCE AQM there, while providing a single-queue
> RFC-3168 AQM (without SCE) on the primary bearer.  Because of the
> unambiguous signalling, SCE traffic missing the DSCP would still
> compete on equal terms with conventional traffic, instead of
> dominating it or being dominated.
>     I realise that this last method is not strictly in scope for the
> l4s-ops draft (and that mentions of SCE tend to raise hackles among
> L4S proponents), but I include it because it appears to be the most
> robust mitigation method available.  It also has the advantage of
> running code being available to try it out immediately.
>     I am not hugely optimistic that the l4s-ops draft will
> incorporate the above advice before the adoption call ends.  But
> unless and until it does, my position is that it SHOULD NOT be
> adopted.
>      - Jonathan Morton