Re: [tsvwg] [Ecn-sane] Comments on L4S drafts

Sebastian Moeller <> Sun, 21 July 2019 20:49 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 8582812014A for <>; Sun, 21 Jul 2019 13:49:50 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.349
X-Spam-Status: No, score=-2.349 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id EMnpAQBLXmb6 for <>; Sun, 21 Jul 2019 13:49:48 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 85F53120147 for <>; Sun, 21 Jul 2019 13:49:47 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;; s=badeba3b8450; t=1563742137; bh=mWhORcl79dCKZnQkAeX6Tjlge3u0yOhXMgpCBLE6BVg=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=SZ1hXjszzEb6Wf9QA1FkIUuxtVJxpSsFuFLfW4SJuaFDnPXGVbYXaFV3iGcc2o1Z6 L3CPGnC8BISZJgOqSOosSkIYyoHptfSpfVDb/6qXRqqPBYUPXJcUEBkay3U3b3xMNk C3znCcLrTGlmUBoZhFCHCv/j5LeRwNmSXD6WlMLU=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from hms-beagle2.lan ([]) by (mrgmx104 []) with ESMTPSA (Nemesis) id 1MWAOQ-1hwQJ12Rju-00XczD; Sun, 21 Jul 2019 22:48:57 +0200
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
From: Sebastian Moeller <>
In-Reply-To: <>
Date: Sun, 21 Jul 2019 22:48:54 +0200
Cc: "" <>, "Black, David" <>, "" <>, Dave Taht <>, "De Schepper, Koen (Nokia - BE/Antwerp)" <>
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <>
To: Bob Briscoe <>
X-Mailer: Apple Mail (2.3445.104.11)
X-Provags-ID: V03:K1:l6gecLrS/leNX1R3ObNoAz/msc7W8y0kLSQnQZx05X1hqeR3OZf daHBNF5iUXbVQAlTy5YpmMg0zOzXTAzKQ/Xe180sl5dH/o0alyo1DcCv63sIyKJI/NmZFcy aPOXaZVu/CYkRLbRA+WwKEUK8ozZrS2cO8T/6rw8ZoKSyNhXNzGQHkbTO1GmjinDSaLRUWp 6e+2yODQ6AQyjkJxIZxmQ==
X-UI-Out-Filterresults: notjunk:1;V03:K0:ykzRnyRK+VI=:JkdUBtv99qv9KHLmFv4RUT Eqw9jQoq/qYtZZUSh3VyvbcUQ9kSdlkIOJ02OeRQc+e3JjCoP5UHGPp3zzbCGX2BOrlqRzWV+ hvFizUUU3mnUsifzlSkDyO4fJNIngnVC6c4P6BKLBkqJDDD2XtfM8KsAc5TcWTe5wz3zUNwD+ aU+ojE2lNkef7H1ZM3eBoqG/pgFLt115SyaOBTg8B35gqhs1cZM7F0JWBEs/7IE0+9XOU7DxS M/uXxUsdCJlsBg+KeAf5m/PyCUmVhIV5zaPJZ5Y1jyR93E2LWDJV0aN7ZxjZlWOTnVVPqVIbX gZEKS9dm3z+SB/lTGIxCQDT5d1A6wTQUzcEKzMrNfrebJWfVczcwnxawAWf0LbXTHWpGJAWQo xNWMDEVSF2L10Qv/02baFNWNXS+gibvmNBscABExQ1zaatjwutwN4BPDpEztDqiKfhmyYUsyJ iOSDDbEEdOyOQSMkUK+1m3Z0fqHRhRRasOPWeFVZlO+Nsee10k2Lr7rSDB2CDn6THg6SQtbQq M7ysX0dUAZX373DzZA4UPU5l1GZIrDxiGQz306xwwIw8Vlpo1yIE2ZSHfLGJBrHnwS2YHNmvZ TOKFKJsq/4sOl0/WkEYWCQydKW7FtMggX4gM8AMwyJFqPmsWjuHyAEDAe91Vvzef7obNo5qzA 5WL+7AghrBFanC+jB5thsKq1mBNTZEnOs7ktzQOBi1zj2P1xpoHr23P3HwvzFucIokIxgH6Xa hF/K17GOpfuS9c8e8ROQCDMUyeyfavgzE5M/4X+PikIPh1wW7ZpK5XUYhq+edCt614ijGfnso cDM9y6m4tc/GNhXjpwSGQm0MzwR2UeIpt0TH28w/4ydjWMcfqEoDreId+sZk1valrOiRCOde4 CaeqiUIyOqKX04VEEbF8HeRKWVlV7b1Sqb2Lfov9CcseoXkeD5i0bmOFM6zKlg5sxwniaWcxq pb7OtcVJmh96CX5ZamxenDGSGUkKoMtPn2gUm2KZZKafxJfFq/8rXBInMfR10MT4+rlVfqWIG QqXKdO5c5wlLN+D7EVicFaFCEMaxvyEqLEMd/7lL/sTL2wxM3VVxDWsUmWH2xfGb4hOzX3II/ d3mP48cZWoW9Fw=
Archived-At: <>
Subject: Re: [tsvwg] [Ecn-sane] Comments on L4S drafts
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sun, 21 Jul 2019 20:49:51 -0000

Dear Bob, 

> On Jul 21, 2019, at 21:14, Bob Briscoe <> wrote:
> Sebastien,
> On 21/07/2019 17:08, Sebastian Moeller wrote:
>> Hi Bob,
>>> On Jul 21, 2019, at 14:30, Bob Briscoe <>
>>>  wrote:
>>> David,
>>> On 19/07/2019 21:06, Black, David wrote:
>>>> Two comments as an individual, not as a WG chair:
>>>>> Mostly, they're things that an end-host algorithm needs
>>>>> to do in order to behave nicely, that might be good things anyways
>>>>> without regard to L4S in the network (coexist w/ Reno, avoid RTT bias,
>>>>> work well w/ small RTT, be robust to reordering).  I am curious which
>>>>> ones you think are too rigid ... maybe they can be loosened?
>>>> [1] I have profoundly objected to L4S's RACK-like requirement (use time to detect loss, and in particular do not use 3DupACK) in public on multiple occasions, because in reliable transport space, that forces use of TCP Prague, a protocol with which we have little to no deployment or operational experience.  Moreover, that requirement raises the bar for other protocols in a fashion that impacts endpoint firmware, and possibly hardware in some important (IMHO) environments where investing in those changes delivers little to no benefit.  The environments that I have in mind include a lot of data centers.  Process wise, I'm ok with addressing this objection via some sort of "controlled environment" escape clause text that makes this RACK-like requirement inapplicable in a "controlled environment" that does not need that behavior (e.g., where 3DupACK does not cause problems and is not expected to cause problems).
>>>> For clarity, I understand the multi-lane link design rationale behind the RACK-like requirement and would agree with that requirement in a perfect world ... BUT ... this world is not perfect ... e.g., 3DupACK will not vanish from "running code" anytime soon.
>>> As you know, we have been at pains to address every concern about L4S that has come up over the years, and I thought we had addressed this one to your satisfaction.
>>> The reliable transports you are are concerned about require ordered delivery by the underlying fabric, so they can only ever exist in a controlled environment. In such a controlled environment, your ECT1+DSCP idea (below) could be used to isolate the L4S experiment from these transports and their firmware/hardware constraints.
>>> On the public Internet, the DSCP commonly gets wiped at the first hop. So requiring a DSCP as well as ECT1 to separate off L4S would serve no useful purpose: it would still lead to ECT1 packets without the DSCP sent from a scalable congestion controls (which is behind Jonathan's concern in response to you).
>> 	And this is why IPv4's protocol fiel/ IPv6's next header field are the classifier you actually need... You are changing a significant portion of TCP's observable behavior, so it can be argued that TCP-Prague is TCP by name only; this "classifier" still lives in the IP header, so no deeper layer's need to be accessed, this is non-leaky in that the classifier is unambiguously present independent of the value of the ECN bits; and it is also compatible with an SCE style ECN signaling. Since I believe the most/only likely roll-out of L4S is going to be at the ISPs access nodes (BRAS/BNG/CMTS/whatever)  middleboxes shpould not be an unsurmountable problem, as ISPs controll their own middleboxes and often even the CPEs, so protocoll ossification is not going to be a showstopper for this part of the roll-out.
>> Best Regards
>> 	Sebastian
> I think you've understood this from reading abbreviated description of the requirement on the list, rather than the spec. The spec. solely says:
> 	A scalable congestion control MUST detect loss by counting in time-based units
> That's all. No more, no less. 
> People call this the "RACK requirement", purely because the idea came from RACK. There is no requirement to do RACK, and the requirement applies to all transports, not just TCP.

	Fair enough, but my argument was not really about RACK at all, it more-so applies to the linear response to CE-marks that ECT(1) promises in the L4S approach. You are making changes to TCP's congestion controller that make it cease to be "TCP-friendly" (for arguably good reasons). So why insist on pretending that this is still TCP? So give it a new protocol ID already and all your classification needs are solved. As a bonus you do not need to use the same signal (CE) to elicit two different responses, but you could use the re-gained ECT(1) code point similarly to SCE to put the new fine-grained congestion signal into... while using CE in the RFC3168 compliant sense.

> It then means that a packet with ECT1 in the IP field can be forwarded without resequencing (no requirement - it just it /can/ be).

	Packets always "can" be forwarded without resequencing, the question is whether the end-points are going to like that... 
And IMHO even RACK with its at maximum one RTT reordering windows gives intermediate hops not much to work with, without knowing the full RTT a cautious hop might allow itself one retransmission slot (so its own contribution to the RTT), but as far as I can tell they do that already. And tracking the RTT will require to keep per flow statistics, this also seems like it can get computationally expensive quickly... (I probably misunderstand how RACK works, but I fail to see how it will really allow more re-ordering, but that is also orthogonal to the L4S issues I try to raise).

> This is a network layer 'unordered delivery' property, so it's appropriate to flag at the IP layer. 

	But at that point you are multiplexing multiple things into the poor ECT(1) codepoint, the promise of a certain "linear" back-off behavior on encountered congestion AND a "allow relaxed ordering" ( "detect loss by counting in time-based units" does not seem to be fully equivalent with a generic tolerance to 'unordered delivery' as far as I understand). That seems asking to much of a simple number...

Best Regards

> Bob
> -- 
> ________________________________________________________________
> Bob Briscoe