Re: [tsvwg] FW: New Version Notification for draft-white-tsvwg-l4sops-00.txt

Sebastian Moeller <> Thu, 30 July 2020 21:43 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 6AEDD3A0D27 for <>; Thu, 30 Jul 2020 14:43:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.649
X-Spam-Status: No, score=-1.649 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id u0VfzjKfPGzE for <>; Thu, 30 Jul 2020 14:43:57 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 0B70C3A0D24 for <>; Thu, 30 Jul 2020 14:43:56 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;; s=badeba3b8450; t=1596145433; bh=Jeuo94fiAIi6XmvRejRK0XZnjCwIPAspY0i6FOtCmSo=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=c7K44pNHq5njpLFNOt9N5nK7/R/dA2rt2iiJKRiwJfGBfOqa09f9B7an/AAc8qIJ6 aj7pKf2UPdG6VDbOLGiVft07ESeP9DFfGOsBbzbUJDrf81PrM5NbBj9CCTra8V9CS9 uB2pJxdXwjT1Ww727nfNSjYkhsiAgtDznxQ2Gp6w=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from [] ([]) by (mrgmx104 []) with ESMTPSA (Nemesis) id 1MTzb8-1k9Xpa3iBW-00Qyzy; Thu, 30 Jul 2020 23:43:52 +0200
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.15\))
From: Sebastian Moeller <>
In-Reply-To: <>
Date: Thu, 30 Jul 2020 23:43:52 +0200
Cc: "" <>
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <>
To: Greg White <>
X-Mailer: Apple Mail (2.3445.104.15)
X-Provags-ID: V03:K1:2bH8DAWzXO9nu8/dqQ8PlSiHoaXTH5YSPmly5cI2PXwvvYMhh6U K98qEyU+PU4WyLhfKORhjOOZP8FU39iE0PS2VBz/eh+FSMuCO83opQg+ugQgGrmk+F3eQ9F F/KxfvdMkTuR86K1YhWNnNLXA2X4GJYC+WM7PzHb15+ZcflkDnzFLM5vy1ImYyAk1lQOCR/ qhoMmW3jGBa/iVW3BI3rQ==
X-UI-Out-Filterresults: notjunk:1;V03:K0:0sgKZ+QjfaA=:j96goAlPaqsw6h6qVnuEEf +uQ/xgw70xPqcMJKaTfeM/HF01BBoEy6JL3mp6nWpmYSQdce/xzCkKc0ndoum4ugZt41FGO1+ zfe2yeH0tFnK600kpazD3XbR16F0Tg+OxvX8/lU1uXk98RO6h1IWtDCGp+QxqC3MeDREMqVv4 SVI3kguSy8r//a+Mi6RZRu4FpCA1ftluLkc6V6Hm7RLPhnmCl+bhLni6K6JX/fXmdEUAvQIPt +ijM8hkEkbEYrNT7IgG6JBwZXEy5OY9YuYCTVVx+mXqflXOIhwM67sKAxW+s9yOos/uBhdyeR 1ZhHvbp0i1dK1hnMkxxPAGflUS8pnT662Sm6zMXquUgvYb1ThNDoYYtH7Nc8Wy/UBmCMHao8b FK/dUpbK+IfnJiutTMW8ssZrRKkcB1Sayl/AuJSmD5xC5X9eYs5b/CKcYFKtDontW72FaDogh EcZzZx+fTNyUAoau8qJyFc8YKldX1iWz6jaq4GOiOkhCwKMOj67MaiVpFNhzv2GAytSqxCtpd 1lDMmOSyj+lQnFUw77TZfSCsPSVAA9Zg6FevUE1Z5UzVxMGDU7ZrJf0b8NjOdHT2VyeCGxSqB loB7LNbMLCqzQxuZ4wuY9lkuJ90HqiNAZ4SrsMmjt8PtHdbGE/wPW50MIzNkYzL/wZIEp136Q YC4b0hDEkWN2nqjIbj38lBEYCmvMN+8oFQ/mUngVJdSLo2EB/ciEcYX+gTdN3kauPxX9Yaeso JtYNwPFQrzspel3qYBW0YK6Q/qhkTNQkArMs+oCvNyI/4z/XXo1LwSYL1lsjFR+vVirNzGT1h /UZuwFMFWNCQqC6xjEhyvA79TVvJI+6I3H3NYTEW2zCBr/9590qHoIpuaQgaUjTHuRzKeSYnn v1bRRA2/YO8KvmvU3ey10+Hj/xN95PRkEkVCj3J7/9YppsCzvIsg2ZQtaIwsOK6cUdxogSgR3 7D/UqiP41+b8z3ghrHiJUhDii5rVEtLgE7kfaUb7oaaukQjxK8UW6uG7qn2XhPPZbR7nHlidq 9tkJdFcxvduLUSb7OACBqDVcXlJyrMbQAttrG4+oMVPd6Lp7ICodCMxeFRedTxnSDKuWjuPI8 On0bQ/sxOS6tg8WIeWQs5de0ue63kJm5V2o3x5y8ZzZXXKnvhSzB05AJRtyRYnIlYNNrufbd9 sHYXAsOKtljyU5iIAyeVJF6RedIdK39ypHo7MnSzgY176HSfpEF/WXxfhE0k2Oh9WZUYXZfaE W11bq+FEm1fOfXKDM
Archived-At: <>
Subject: Re: [tsvwg] FW: New Version Notification for draft-white-tsvwg-l4sops-00.txt
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 30 Jul 2020 21:44:00 -0000

Hi Greg,

my comments for sections 1 & 2 below (more will follow):

"1.  Introduction

   In the majority of network paths, including paths where the
   bottleneck link utilizes packet drops (either due to buffer overrun
   or active queue management) in response to congestion, as well as
   paths that implement a 'flow-queuing' scheduler such as fq_codel or
   Cobalt, and those that implement dual-Q-coupled AQM, L4S traffic
   coexists well with classic congestion controlled traffic."

[SM] This description ignores the birthday paradox issue that Pete Heist demonstrated. A misbehaving L4S will cause less havok there but it is not without side-effects on stochastic flow queueing systems, we might as well acknowledge that, no?

"   On network paths where the bottleneck link implements a shared-queue
   (FIFO) with an Active Queue Management algorithm that provides
   Explicit Congestion Notification signaling according to RFC3168, it
   has been demonstrated that when a set of long-running flows
   comprising both "Classic" congestion controlled flows and L4S-
   compliant congestion controlled flows compete for bandwidth, the
   classic congestion controlled flows may achieve lower throughput when
   compared to the L4S congestion controlled flows.  This 'unfairness'
   between the two classes appears to be more pronounced on longer RTT
   paths (e.g. 50ms and above) and/or at higher link rates (e.g. 50 Mbps
   and above)."

This is rather cautious I would use more drastic terms, the observed unfairness approaches starvation of the non-L4S flows and this text makes it sound like it is a minor almost theoretical concern.

"  The root cause of this unfairness is that RFC3168 does not
   differentiate between packets marked ECT0 (used by classic senders)
   and those marked ECT1 (used by L4S senders), and provides an
   identical congestion signal (CE marks) to both classes, whereas the
   two classes respond differently to that congestion signal."

How about keeping causality intact and frame this as a consequence of L4S redefining what CE means. There are reasons for doing that, but this text leaves it unclear who caused this problem (or rather the text implicates rfc3168, which IMHO violates temporal causality).

"The result is that the
   classic senders respond to the CE marks provided by the bottleneck by
   yielding capacity to the L4S flows.  While this has not been
   demonstrated to cause starvation of the classic flows, the resulting
   rate imbalance can be a cause of concern."

pretty much demonstrated starvation and that is with TCP-Prague with rfc3168 detection. 

"2.  Per-Flow Fairness

   There are a number of factors that influence the relative rates
   achieved by a set of congestion controlled flows sharing a queue in a
   bottleneck link.

   TODO: discuss startup & convergence times, short flows, RTT-
   unfairness, differences in deployed CC algorithms, etc.

   TODO: also mention that flow sharding is commonplace, so per-flow
   fairness does not imply per-application fairness"

This obviously needs flashing out before one can meaningfully comment, but regarding the last section, you realize that application not the relevant classification for intermediate nodes? I would assume that for an ISP a (potentially weighted) per-end-host fairness would be the exact tool to avoid sharding as work-around... This is obviously not a new idea or comment, so if you opt for elaborating on fairness, please also include known remedies. Or better avoid that discussion here altogether.

Best Regards

> On Jul 30, 2020, at 13:05, Greg White <> wrote:
> TSVWG members-
> I've posted a rough draft of Operational Guidance for L4S deployment.  This is not much more than an outline at this point, and is almost certainly incomplete even at that, so please read it with that in mind. 
> -Greg
> From: "" <>
> Date: Thursday, July 30, 2020 at 4:53 AM
> To: Greg White <>
> Subject: New Version Notification for draft-white-tsvwg-l4sops-00.txt
> A new version of I-D, draft-white-tsvwg-l4sops-00.txt
> has been successfully submitted by Greg White and posted to the
> IETF repository.
> Name:		draft-white-tsvwg-l4sops
> Revision:	00
> Title:		Operational Guidance for Deployment of L4S in the Internet
> Document date:	2020-07-30
> Group:		Individual Submission
> Pages:		7
> URL:  
> Status:
> Htmlized:
> Htmlized:
> Abstract:
>   This is an early, work-in-progress draft - a start at getting some of
>   the ideas from the mailing list and email exchanges on paper.
>   This draft is intended to provide guidance to operators of end-
>   systems, operators of networks, and researchers in order to ensure
>   reasonable fairness between L4S and Classic flows sharing a single-
>   queue RFC3168 bottleneck link.  This draft identifies opportunites to
>   prevent and/or detect and resolve fairness problems in such networks.
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at
> The IETF Secretariat