Re: [tsvwg] Follow-up to your DSCP and ECN codepoint comments at tsvwg interim

Jonathan Morton <> Thu, 12 March 2020 11:45 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 846DB3A0DD1 for <>; Thu, 12 Mar 2020 04:45:24 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.848
X-Spam-Status: No, score=-1.848 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id comvOgFeq-wq for <>; Thu, 12 Mar 2020 04:45:23 -0700 (PDT)
Received: from ( [IPv6:2a00:1450:4864:20::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id A8D2F3A0DDC for <>; Thu, 12 Mar 2020 04:45:19 -0700 (PDT)
Received: by with SMTP id f10so6024575ljn.6 for <>; Thu, 12 Mar 2020 04:45:19 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=ruwemoI5nvjknqxqw7lOBmK3sTuWabAYmEQCYfT6lNo=; b=THtQZIz0LIEurcMC+ASXw1oFflMRVNrEmjzVzC/4LhS234qlOujyHwfLHJulIGXOT+ S8dYxi4aQx1vdJf+Wyy/xzBUP5u950J4ELmZvo7jaWz2dk/niUAwHzFS5Zmrzj218tSl zS6tLz78AP3QudoBGX3xCUBlgX4H7OAjm+ucJvZ2AyeajEnM7ohKSRRzNlyiO70+Z3oj Z0OADVPhADkzW+QvwTaXEkh+pz3c0L6AgRKTUeyPxdN4wsa0VTbgIguJYSL6lX5fE+/2 GHJMxr3x+/y0VAea+e2mz4f/6Nz4mEJMffv3EEkyagZcizYX+250fb3Mena5xE3GV9g4 p54g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=ruwemoI5nvjknqxqw7lOBmK3sTuWabAYmEQCYfT6lNo=; b=hQVTeY9+aGODiiKm5TTmPxqyZT/HhwUARFLWU9nk+viLOt693AJZI/juHnNldy00d2 Ro3IUA/Lo/j+WlrVgf6jiNuCXgQ2m9IOVKBwTsxzN/BvgivPjJszqQdClqlhEf6DtfPM nPvsj7HaPKt8SvcoXK5oeRdEkX+mN6g/V93Ud5uAtluPptATXcgj6QiYpFAEr21ZbwJD 4DN2iPb5FNO9ykc7rKD/0IiZ31AIk0LEKYlAHN+HsoK6kkkXskzd9ZvjmuwcDYDRYAbq qo+oot6afU1fWCD2R3agac7B0LaULY18gR+UkD2Eg5DnfUiwTNUwJTJwTM7tj5GkZFkP PlyA==
X-Gm-Message-State: ANhLgQ0IlOq/koICa7F1UfgRrxu/39HmugxffJQgUFCoyMRN+M7vbVuz cbVqFnvm3HYawpx1ZK4vGtU0BLop
X-Google-Smtp-Source: =?utf-8?q?ADFU+vviElM/i3jCj7n+UZ5vnIi9CC7KnyFUzOuIje+o?= =?utf-8?q?G4o8X8/W+hfXc6sbm/q4ypvzayHSOHmboA=3D=3D?=
X-Received: by 2002:a2e:88d9:: with SMTP id a25mr4050392ljk.267.1584013517598; Thu, 12 Mar 2020 04:45:17 -0700 (PDT)
Received: from jonathartonsmbp.lan ( []) by with ESMTPSA id x13sm14510988lfq.97.2020. (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 12 Mar 2020 04:45:16 -0700 (PDT)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\))
From: Jonathan Morton <>
In-Reply-To: <>
Date: Thu, 12 Mar 2020 13:45:14 +0200
Cc: Wesley Eddy <>,
Content-Transfer-Encoding: quoted-printable
Message-Id: <>
References: <> <> <> <> <> <>
To: Lars Eggert <>
X-Mailer: Apple Mail (2.3445.9.1)
Archived-At: <>
Subject: Re: [tsvwg] Follow-up to your DSCP and ECN codepoint comments at tsvwg interim
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Thu, 12 Mar 2020 11:45:25 -0000

> On 12 Mar, 2020, at 10:18 am, Lars Eggert <> wrote:
> I'd much rather see some short answers on some basic questions, such as:

I think the question posed by David Black's slides (as presented and briefly discussed at the interim meeting) is intended to elicit relevant answers to these sorts of questions.  But since you asked, I'll have a go at answering them standalone, for the SCE proposal.

> Is the solution incrementally deployable?

Yes.  Each component of SCE is inherently safe to deploy into an existing network, and will have no detrimental effect on conventional traffic.  SCE flows traversing an existing network will behave conventionally and in an RFC-compliant manner.  This is not merely aspirational, but baked fundamentally into the design.

> Is it simple?

Yes.  I could summarise the technicalities on two slides - or one, if I really had to.  Implementing SCE also does not require very intrusive or extensive changes to existing network stacks; a year ago, we got a basic proof of concept running in about a man-week.

> Does it offer better performance, to participating and ideally non-participating flows?

Yes - adding SCE to an existing ECN network usually improves throughput, peak latency and jitter for SCE flows, and improves total link goodput when at least one SCE flow is present.  Adding SCE to a dumb-FIFO network also brings significant benefits to conventional traffic, as any good AQM deployment would.

> Is it at worst a no-op, or can it degrade performance for non-participating flows?

Non-SCE flows see only an RFC-3168 compliant AQM on the path, and ignore the SCE signals.  SCE flows at worst have a tendency to defer to existing traffic - the opposite to what you fear - but with a well-designed bottleneck qdisc, this effect is limited or eliminated.  The SCE proposal presents several such qdiscs as examples.

> Can a partially deployed solution be rolled back?
> Can it stay deployed but become a no-op if a different mechanism wins out in the end?

This is probably the most complex question to answer.  Un-deploying an experiment is always more difficult than deploying it in the first place, since it's harder to prove a negative.

SCE consumes the spare ECN codepoint and one spare TCP header bit, without explicitly negotiating for them (beyond what is normal for RFC-3168 ECN), and adds a permitted ECN codepoint transition in middleboxes: ECT(0) -> ECT(1).  As far as conventional ECT flows are concerned, ECT(1) is semantically identical to ECT(0), and the spare header bits are ignored if received, so conventional traffic would not be affected.  The question is really about how to free up these consumed header resources, should that be necessary.

Reversing an SCE deployment thus means removing all sources of this new usage, which means both middleboxes and endpoints.  The reference implementation leaves SCE functionality switched off by default, but it is easy to automate switching it on and potentially then forget about it.

The following probes can be used to seek out latent deployments, and ask their operators to disable them.  This does not require a new kernel to be compiled or installed, nor any downtime, only that the SCE functionality is configured off at runtime, eg. through a sysctl or the iproute2 tc utility.

SCE enabled middleboxes are easy to detect, by passing ECT(0) traffic through it and seeing if ECT(1) marks appear alongside (or before) CE marks.  If CE marks or an AQM-like (or tail-drop-like) pattern of drops appears but ECT(1) does not, it's not an SCE enabled middlebox.  This is probably sufficient to reclaim the spare ECN codepoint.

SCE enabled TCP receivers are also easy to detect, by probing them with ECT(1) traffic and seeing if the corresponding TCP header bit is used to echo them.  Some other transports may have ECT(1) feedback designed into them, which is not affected by SCE per se, and these can be ignored.  This is probably sufficient to reclaim the TCP header bit.

SCE enabled senders are a little more difficult to detect unambiguously.  An indication can be obtained by probing them with the TCP header bit artificially set in all acks, and seeing if the resulting throughput is markedly less than if that is not done.  SCE senders behave essentially identically to normal ECN senders when the corresponding TCP header bit remains cleared in acks, so their continued presence is mostly benign, unless the header bit starts being used in some other experiment in a way that confuses the SCE sender.

If considered necessary by the WG, it would be possible to associate SCE with an experimental-series DSCP, and require that DSCP be present for SCE functionality to be enabled.  This would however limit experiments to networks that do not bleach or otherwise disrupt that DSCP, which may inhibit exploration of behaviour on longer Internet paths (which we think SCE is much better suited for than L4S).  The upside would be easier reclamation of the header resources upon termination of the experiment.

Our proposed use of DSCP is instead an optional addition, using SCE as part of providing a low-latency Diffserv PHB, but not requiring any particular DSCP to support SCE.

 - Jonathan Morton