Re: [tsvwg] Follow-up to your DSCP and ECN codepoint comments at tsvwg interim

Sebastian Moeller <> Wed, 25 March 2020 08:50 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id CEDA43A0848 for <>; Wed, 25 Mar 2020 01:50:22 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.648
X-Spam-Status: No, score=-1.648 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id emU4abL7wj6s for <>; Wed, 25 Mar 2020 01:50:20 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 42FFA3A0818 for <>; Wed, 25 Mar 2020 01:50:20 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;; s=badeba3b8450; t=1585126133; bh=b4AfxJPhBMm68D5jw5p9ywRtJv/h9OlA0YF9HiyZGqc=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=L3R012hjQhV9UVQJ7m9YigFPhBb2qVSx1J4USieqG8ZjVPEWl0Ca100CW/rLLfsI7 lgm6fFankNpsCk4g7pJvVOXcb3DZNRUR6uYSrtJEMv6mqlOjBOTdNSx2yKbyRlasT1 9svABY1DjOCp/jVxy7hSG0C3bdNtkL9dwI23rF9w=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from [] ([]) by (mrgmx004 []) with ESMTPSA (Nemesis) id 1MEFzr-1j95kn3As1-00AC4a; Wed, 25 Mar 2020 09:48:52 +0100
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
From: Sebastian Moeller <>
In-Reply-To: <>
Date: Wed, 25 Mar 2020 09:48:50 +0100
Cc: tsvwg IETF list <>, Steven Blake <>
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <> <> <> <> <> <>
To: Bob Briscoe <>
X-Mailer: Apple Mail (2.3445.104.11)
X-Provags-ID: V03:K1:gcoTQr8v7JnjiQblOiESaahQ0wElh4JrLFZGKqhsuJAUThcKixM zcArvsmuoLV6AE2F1c/gboClbocSBzNkUP6dK74uQpeka2ketjhDsaSCGK7z9DfENGRrISf MMFAwxTEIboqiXV5LVdKbzsRp+/SNuam2hjx0V+MkvtWaQ70qzd4dDGjBK7ci6DY5XuI+/N 8ttsdxr70Duga9vj98TAQ==
X-UI-Out-Filterresults: notjunk:1;V03:K0:U4T7LVLnQmg=:2Yn9lO5aNQBou8qhXee7/A QKUbbyqDcDVnZSw2s8+0FSf8iFho2E7/kY+VU7jKJi7MB4Hots/s68IuBZjUpGzeIrWOsURyP 2c4jy5RaYdVVY7eOAQu2ZXtpqNYCyUUNJbTnqPXqgwQlTjSHuy87TH8lkk5gYnRaD6Bl8XmGA 2tnUgWD0MCNwaL7oUl9x2zGGnTc9rdhkYLVqzZUBC5hN+WgUbXYKSNDAJHwXPlvRymW94NsMY /ZV3GSpgHwqdejLB/YH1Sjr9BrB12TbsbiRBM7/FjGKJt2HkPY6yAFO+EmhLG9xNAPcK6wa5A JFjcSmtCttkFLnkW6VjkcbIHhYrrm7IgIUwg0Rp47NcsQRpP+lMOBr+R0lZE8bK/7cc1ZEYkZ 3ehJ69Ck3cmEcLS6/sIAKayte8IWhto4m3eLIDauVq0/eLMTWAcSz5QMVfwe7U782VOnHxfLq gb7vv04BlwKubRzRgVzDmglWrVztNPabhFiNTcLRCW1Nzmcpe6jx6FfcsvMVWK5HnQh8riurH LFt5rqp+CLBCMKoe+RJ3g0fYMXKOJwznom0fFysLW1TCHV1FOMwoX+Cw+P4YPAynMdukr5n0K oGqAKh9UAKI6Nqo6fCt30u0ufvkFVxEayOQukC768l+sxtLmp8wJJfpqDfKFd0bVb4ZpBYUDW TXnegdzHb+aP6yIENRb3m8rur4hAKUq0uw3nSPX6rmKBUVWiofqWeIKKiSbhobDqUAnw+GPuM kAYXAthwKjRJLnKGYxLhtWUmbJovatakHs/xAbp7X3LfUbPBGvtaEzVb9XCAkrWFl9jo+Ykjs 6kdRMezOhlFXOwc0Y1py5FJBKXIFarXkmj3r/oaPL4UkIKOJbMBEb5ua48OyIHTDd3bVtPObm 1vN4BUqCV69o2iy9vgQAWQPot44louJm8jTf8SVKicrb+RPd1DlPOIJ5RsQ56GlSFSzEKu3yI 9Yvx2kD4x24n5wT2bF+amv7tiyz5J+N6V76jw40NKsZx2ylSCxCF34KTch86Ld2be+Ov6onKw HB5AUdVvzhlwINAWo5Nc+xvyroPnPAY6jQvE+fnGhY7Js5UGsEbtw0oqN5OhWxRCXA4U/CbnO GpZOQVjNBsGhKk7oSZ4UQD8G/6oY64tKNNRhqLOi4IZfs0mY+yF5xlGbwfwRfkh/0+SbIX2Y4 mlkPCCUPovX3G86vTXPgEKqhb01CAzGfcplOwY923ki5K+DqTiLn3UqS0kXgZCaEFpG7uvOn9 UQp8wlf71E5vudnXZ
Archived-At: <>
Subject: Re: [tsvwg] Follow-up to your DSCP and ECN codepoint comments at tsvwg interim
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 25 Mar 2020 08:50:23 -0000

Hi Bob,

Summary: more detailed discussion about semantics and disconnects, safe to ignore.

> On Mar 16, 2020, at 11:32, Bob Briscoe <> wrote:
> Sebastian,
> There are two a very important misconceptions in this email that other people might be labouring under, so sorry for dredging up this week-old thread... inline...
> On 08/03/2020 19:52, Sebastian Moeller wrote:
>> Hi Bob,
>> more below in-line.
>>> On Mar 8, 2020, at 19:27, Bob Briscoe <> wrote:
>>> Seabstian,
>>> On 08/03/2020 15:34, Sebastian Moeller wrote:
>>>> Hi Bob,
>>>> More below.
>>>> On March 8, 2020 3:23:39 PM GMT+01:00, Bob Briscoe <> wrote:
>>>>> Sebastian,
>>>>> On 07/03/2020 19:16, Sebastian Moeller wrote:
>>> especially not for applications that might not be possible any other way (both low latency and capacity-seeking high bandwidth has traditionally been considered impossible).
>> 	[SM] The bigger point is IMHO, is greedy and low-latency actually a desirable combination of features?
> [BB] Not to have grocked this basic requirement is a serious disconnect.

	[SM] That is a tad impolite (given your own sensibilities about tone), no? Not accepting your premises is not the same as not understanding them.

> Short answer:
> Application-limited does not preclude needing to be capacity-seeking.

	[SM] "capacity-seeking high bandwidth" I was mostly referring to the high bandwidth part, but I did not explicitly mention that, sorry if that caused confusion.

> "Application-limited" is decided when the application is written, whereas "capacity-seeking" (a.k.a. "greedy") copes with potential conditions encountered at run-time. So when a flow needs to be capacity-seeking, you don't want the onset of queuing to ruin the low latency.

	[SM] You are basically trying to sell me on the idea of  1/p-type congestion signaling, you can stop, that part I agree with (and both components, lower amplitude signaling and higher temporal fidelityas compared to rfc3168). Where we disagree is whether L4S is a sensible design to deploy such a congestion signaling into the wider internet.

> Longer answer:
> Presumably you're not disagreeing that being able to have high throughput at the same time as very low latency would enable many new applications (natural video-based interaction, video-based remote control, cloud-based AR/VR, online gaming, remote desktop, etc).

	[SM] I am realist enough to understand that no low latency application will work robustly and reliably over the wider internet. But I also realize that you are arguing here for a "build it and they will come" approach, as you only have vage examples of use-cases that require low-latency-high-bandwidth, "would enable many new applications" is rather unspecific...

> Even if the throughput of a video flow is usually application-limited, the higher the throughput, the more likely it will sometimes encounter scenarios where it is limited by available capacity. For instance,
> a) when a path with a lower capacity bottleneck is used (e.g. I communicate with someone who has a lower capacity access link, or I move to a lower capacity access), or
> b) because enough other traffic has arrived at the existing bottleneck.
> Then you want the congestion control to become capacity-seeking (not unresponsive), without losing the low latency.

	[SM] Well adaptive streaming is an already deployed technique to deal with variable bandwidth paths, that works reasonably well, especially for pre-canned video where the real-time tolerance of users is relatively high... You really need an application where the end-user's feedback ends up in a tight control loop for the generation of the video before ultra-low queueing delay becomes essential. The roll-out of google stadia and nvidia gforce now with existing internet technology indicates that even without L4S the currently deployed tech allows sufficiently tight "real-time" control loops for interactive "video streaming". I am not saying that things can not be improved and that some of L4S components might not be great solutions, but I question the premise that L4S will allow anything qualitatively novel. Which IMHO is a big point when assessing L4S's side-effects.

> Every time L4S is presented or written about, the same high level requirement of high throughput and low latency is repeated.

	[SM] Well, repetition of a statement does not change its truth value, if at all it tells us about the persistence of the person repeating it.

> What did you think the whole L4S activity was about?

	[SM] This is a direct question I am not really sure you are interested in my answer, but here you go. As far as I see it, you are building a high priority high bandwidth lane onto customer access links that then can be monetized by ISPs by selling this high priority access to interested parties (on the content generating or consuming side). IMHO that is not a bad thing in itself, under the condition that it is fully opt-in and the end-user can decide whether they want that or not. A lot of users might be willing to spend money to improve their gaming quality (but that assumes that L4S will actually work well enough over arbitrarily long path between end-users and game-servers).

> [I've snipped the next part of the conversation, because it was opinion-based and going nowhere.]

	[SM] ??? Most of our discussion is opinion-based including most of your arguments and I have repeated asked for hard data only to be told data is either irrelevant or that I should go measure myself if I want to see data.

>> 	[SM] But here is the rub, the L4S architecture makes in unduly hard to separate low-latency queueing from 1/p-type congestion control, as I fully agree these are ORTHOGONAL to each other, so artificially coupling them, e.g. by making ECT(1) both denote a 1/--typre response to CE AND requestimg admission to the LL-queue is exactly the wrong thing to do. Glad that you agree.
> [BB] Another rudimentary disconnect here...
> Queuing delay is not caused by the queue, it's caused by the congestion-controllers that induce the queue.

	[SM] Semantics, semantics semantics. Not terribly interested in that discussion. IMHO it is the queue that can, by say dropping immediately, shed the excess packets that cause the delay, not the end-points that can not really affect data once in flight. Sure it is the traffic (initiated by endpoints) that shows up in intermediary queues, but in the context of an AQM misbehaving (like dualQ is) pointing to the end-points is IMHO not helpful.

> 1/p congestion controllers are what keeps the sawteeth small so that the ECN-marking threshold in the queue can be shallow without causing underutilization.
> The CoDel config of 'target' (min queue depth) was recommended to be set quite low despite leading to underutilization for higher RTTs.

	[SM] Could you please give a citation where you got that idea from? The "underutilization for higher RTTs" is what we called RTT-dependence in another thread and where we agreed that it also affects TCP Prague, aka 1/p type AQMs/signaling.

> However, if the congestion controllers are not scalable (1/p), even if they keep the sawteeth fairly small today, they will grow as link rate scales. Then we'll be back in the same hole in a few years time.

	[SM] IMHO the reason why L4S is (still) under consideration for standardization is because everybody agrees that 1/p signaling with high temporal fidelity is really desirable.

> Unless we sort this out properly, the problem will always stay that we have to remain friendly to original Reno flows still likely to be running over the Internet. The L4S queue is a transition mechanism that
> a) creates a clean-slate isolated from that slow-death problem.

	[SM] That green-field assumption is IMHO wish-ful thinking, there is no clean slate you need to coexist peacefully with the eExxisting internet; but that mindset effortlessly explains why you accept all the side-effects of the L4S design.

> b) gives immediate benefit today, which encourages adoption (it patently already has)

	[SM] Only that the amount of benefit is arguably not commensurate to its side-effects.

> Again, every time L4S is presented or written about, this explanation is given up-front.
> [Again I've snipped some opinion-based conversation, going nowhere...]
>> 	[SM] That seems all irrelevant to L4S though, and especially why gauarding L4S use of ECT(1) with a bespoke DSCP would hinder EXPERIMENTAL roll-out of L4S in any meaningful way. As far as I can tell, you seem to think about how to deploy L4S as a standards RFC, but how about first making sure there is a viable path to it becoming a standard first?
> [BB] If an experiment cannot transition to wider usage, potential adopters will notice and not even start.

	[SM] That is not wrong, except that dropping a DSCP requirement when transitioning from experimental to standards track status seems like a very minor modification, and one that most ISPs are already comfortable with. So no I do not buy the argument that requiring a concomitant DSCP with ECT(1) to be the straw that breaks the camel's back in regard to experimental roll-out. And the fact that all you offer are bland general principles makes me believe that you grudgingly agree. 

> The limits to the experiment are best put in place by the experimenters, not by adding inherent limits to the technology.

	[SM] That indicates, that you consider the actual experimentation to be somebody else's problem though; pretty much what I feared releasing a half-backed design into the wider internet without appropriate safety mechanisms and no plan what the experiment is supposed to achieve. I guess I am tainted by what experiment means in my line of research.


> Bob
>> Best Regards
>> 	Sebastian
>>> Bob
>>> {Note 1}: E.g. initially some might only classify ECT1 into the L queue for business customers, or as a product for a higher tier of customers, or solely for the operator's own services (if allowed in their jurisdiction). Other ISPs, like you say, will want to use it as a differentiator for their whole regular service (see draft-ietf-l4s-arch).
>>> {Note 2}: Of course, certain ISPs might pervert the ECN signal, but I think that's less likely, 'cos they can only access to traffic in their own network, which inherently hits the e2e congestion control of their own customers. If we think that's a possibility, L4S senders could cover the least significant bit of the ECN field with integrity protection to raise the bar against network interference, by tying it to the integrity failure of each whole packet.
>>>> And especially for the scope of the experimental deployment? The experiment is required to make sure that L4S can deployed in a safe and backward compatible fashion and that it can deliver it's promises under real-world conditions.
>>>> The experiment is NOT about how to deploy something in a fashion that offers the least part of resistance/the highest adoption rate. As a rule of thumb, I would assume the IETF be interested in the technical aspects and not the marketing side....
>>>> Regards
>>>>         Sebastian
>>>>> Bob
>>> -- 
>>> ________________________________________________________________
>>> Bob Briscoe                     
> -- 
> ________________________________________________________________
> Bob Briscoe