Re: [tsvwg] Neal Cardwell's rationale for supporting ECT(1) as an input/L4S signal

[Roland Bless <[e-mail]>]

Hi Neal and all,

see inline.

On 08.05.20 at 17:19 Neal Cardwell wrote:
> As part of the discussion around the consensus call on ECT(1), I
> wanted to post my personal rationale for supporting ECT(1) as an
> input/classification signal, along the lines of the L4S proposals.
>
> This is my personal opinion; I'm not speaking for any company or
> institution.
>
> Many of these points will echo the excellent points that were made at
> the TSVWG interim on April 27, by Greg White, Bob Briscoe, Jana
> Iyengar, Stuart Cheshire, and Andrew McGregor; I agree with all of the
> points they made. But for the sake of discussion and debate I wanted
> to amplify what they said and enumerate the particular considerations
> that are most compelling to me.

I had the feeling that this was more a discussion around L4S vs. SCE
proposals and not about the
ECT(1) codepoint in question. Several arguments were along the lines of
"L4S is more advanced
and needs to be finished as it already took too long". I don't think
that this is a good argument as
the L4S proposal could also use ECT(1) as output and use a DSCP as
classifier. See below.

>
> Here they are:
>
> - SCE is basically an extension of RFC3168 ECN: senders mark their
> packets ECT(0), and respond to CE like RFC 3168 says they should, by
> halving cwnd. This would not work well for large sites with a
> substantial installed base of shallow-threshold (DCTCP-style) ECN: if
> these sites marked their traffic with ECT(0) to try to use SCE, their
> local switches configured for shallow-threshold ECN would make CE
> marks at low queue occupancies, which would cause large RFC3681-style
> reductions in cwnd, which would cause underutilization. To the best of
> my knowledge this applies to several large Internet services.

Let's leave the classifier issue aside for a moment:
using ECT(1) as mark for "shallow-threshold exceeded (DCTCP-style)"
would make it
an unambiguous congestion signal. CE would still mean: use RFC3681-style
reduction.
These could even be mixed along a path (CE after ECT(1), but CE will
stay CE).
Sure: shallow-threshold switches need to use ECT(1) instead of CE then;
I have no clue how difficult this would be to change in the existing
deployed gear.
So the situation you are describing is caused by the semantic overload
for CE used for both types of marking (RFC3681-style and DCTCP-style).
The current solution is to distinguish the semantics by context (e.g.,
controlled environment/pure in-DC use), which doesn't allow DCTCP-style
marking and RFC3681-style marking for an
end-to-end connection at the same time, but using ECT(1) as output would
allow this.

> - Much of the installed base of switch hardware around the world can
> already be configured to signal shallow-threshold congestion with
> DCTCP/L4S-style CE marks. Many have CE-marking mechanisms baked into
> hardware, and would need to be replaced in order to deploy SCE.
The question is: could they easily support using ECT(1) instead of CE
for the shallow-threshold marking?
Supporting L4S's dual queue and classification by ECT(1) would IMHO
require even more extensive hardware changes.
> - Encapsulation/decapsulation is a widely prevalent and important
> technology today in production networks. With the installed base of
> encap/decap mechanisms, it is likely that for many implementations any
> SCE marking applied to packets would just get stripped off with the
> outer header when decapsulated.
>
I'm not sure about this one. L4S could be affected equally or do you
think that CE is handled differently? I wasn't able to find a quick
answer by looking at
https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-encap-guidelines-13
> - To get low latency in the public Internet, we need multiple queues.
> With SCE AFAICT there's not yet a compelling story about how to
> classify traffic to maintain a low-delay queue and  a classic queue,
> since there is no ECN code point to identify the low-delay traffic.
> AFAICT with SCE you likely must either (a) have a single high-delay
> queue with a jumble of Reno, CUBIC, classic RFC 3168, and
> SCE-respecting RFC 3168 senders, (b)  use fair queuing (tricky in
> hardware), (c) use DSCP to classify (academically pure but unrealistic
> across multiple ASes in the public Internet), (d) use a dual-queue
> system that heuristically classifies all flows into the low-delay
> queue by default but kicks them out using a queue-protection algorithm.

I agree here and queue separation would also be essential for SCE. But
using a DSCP for classification would
be the more extensible solution as ECT(1) wouldn't always mean to use
the L4S queue semantics. Instead both
solutions could use the ECT(1) mark with nearly the same congestion
signal semantics. Moreover, both solutions have
problems with "misbehaving" flows in the wrong queue.

> - SCE seems to involve an ecosystem with a more complex and more
> experimental CC (with two different kinds of ECN signal) and little
> real-world/production experience yet..  L4S seems to involve an
> ecosystem that provides a queue that  is basically a single-threshold,
> shallow-threshold, DCTCP-style, ECN ecosystem, which is simpler and
> for which the world has a lot of accumulated academic research and
> real-world/production experience over the last decade.
>
> - L4S flows potentially causing unfairness in RFC3168 ECN bottlenecks
> has been mentioned as a potential concern. However, a robust RFC3168
> ECN bottleneck should already have a mechanism to avoid unfairness
> caused by flows that are marked as ECT(0|1) and yet not performing
> RFC3168 responses. In particular, many of the large sources of known
> deployments of  RFC3168 --  Linux fq_codel and cake -- are already
> deployed with fair queueing. In such bottlenecks L4S traffic should
> not cause harm to other non-L4S flows. Furthermore, if there really
> are ISPs with deployments of RFC3168 bottlenecks that have neither FQ
> nor any other protection from non-RFC3168-ECT(1) flows, then they can
> bleach incoming ECT(1) code points to Not-ECT and treat L4S as Not-ECT
> (ISPs typically already transform the DSCP byte at their ingress
> anyway). So I do not see harm to RFC3168 ECN bottlenecks as a
> prohibitive concern.
>
> - More generally, if there is any problem discovered with the L4S
> experiment, either the algorithm or particular implementations,
> bottlenecks can easily identify L4S traffic and bleach it into
> Not-ECT, and treat it like Reno/CUBIC traffic.
>
> Best regards,
> Neal
>
Regards,
 Roland