Re: [tsvwg] L4S DSCP (was: L4S drafts: Next Steps)

[Bob Briscoe <[e-mail]>]

Jonathan,

In May 2020, a very intense period of focus on the question of the L4S 
identifier came to a close. The possibility of a FIFO RFC3168 ECN AQM 
that you are focusing on (to the exclusion of all else) was one of many 
issues that made the choice of identifier difficult. In this thread, I 
have been digging into this DSCP proposal from you and David to show 
that, when another candidate identifier is investigated more deeply, 
different concerns of feasibility, safety, etc. emerge.

We can now summarize the impact of this proposal to use a DSCP to 
confine the L4S experiment to one network:
* You've agreed that the DSCP scheme doesn't itself avoid any FIFO 
RFC3168 ECN AQMs in the network participating in the experiment, or in 
its customer networks upstream or downstream. It still requires a 
participating L4S network to follow the l4sops draft to bypass these 
AQMs, as if they were using the original L4S ECN scheme.
* You've also agreed that the DSCP scheme needs changes to all standards 
track transport protocols, so that they somehow feed back the DSCP as 
well as the ECN field, even though bits are scarce, and even though the 
only motivation for these changes would be to make this DSCP idea work 
for L4S.
* In addition, the DSCP will not necessarily even serve its confinement 
function if it is within a pipe-mode tunnel.

In summary, this DSCP proposal adds very little to safety and it 
subtracts greatly from viability of the experiment.

To you and one or two others, destroying the viability of an L4S 
deployment experiment doesn't seem to matter. However, the point of that 
session in 2020 was for people to choose /on balance/ which identifier 
they wanted the WG to focus its work on, taking account of the 
compromises that each choice would entail.

On a question of balance between competing requirements, we were never 
going to get unanimity. Nonetheless, a very large majority of the people 
polled last May indicated that they wanted to move forward with the L4S 
identifier as it stood. This reaffirmed the same decision in Nov 2015. 
Also, at that meeting in May 2020, a very  large number of companies had 
confirmed they had plans to deploy L4S (all as a company, not just 
individuals in those companies). In contrast, of the three companies 
that you volunteered, two publicly admonished you for implying they 
supported you, and it turned out the other one had not actually 
expressed support as a company.

As a result, the chairs then set the agenda such that the WG would focus 
only on getting L4S through the WG stage; with L4S ECT(1) as the 
identifier. All other identifier proposals are currently off the table. 
You have no standing to demand that L4S uses a different identifier, let 
alone that L4S-specific variants of standards track transport protocols 
have to be invented to make your ideas work.

I only dug in depth into this DSCP scheme to prove how little it adds 
and how much it subtracts. This was primarily for those who joined David 
Black's discussion on the list such as Steven Blake, Jake, Kyle, and 
Ruediger, who otherwise might have continued to think it might be a 
feasible approach. I suspect there was some confusion that the DSCP was 
being used as a classifier in the network, whereas you are proposing it 
for a per-connection cross-layer path traversal feedback loop, without 
its classifier function at all.

Like this DSCP proposal, the SCE proposal also subtracted greatly from 
viability of the experiment. But at least it had the potential to 
improve safety if FIFO 3168 AQMs turned up. (However, as you admitted, 
SCE traffic gets squashed by legacy traffic in the same queue). So this 
is the second of your proposals that has not stood up to scrutiny.

See inline for detailed responses, tagged [BB]...


On 29/03/2021 04:23, Jonathan Morton wrote:
>> On 29 Mar, 2021, at 3:12 am, Bob Briscoe <[e-mail]> wrote:
>> There is no L4S design 'baked in' to existing standardization work. L4S is on the experimental track, and designed to build /on/ ECN feedback in existing standards track IETF transport protocols. Specifically QUIC, RTCP and TCP.
> Well, in that case it should still be feasible to improve L4S' design and implementation before deployment.  Of course, the longer you spend resisting any and all efforts to suggest such improvements, the narrower that window of opportunity gets before the whole WG just gives up in disgust and refuses to publish.

[BB] See above. The WG decision has been made (twice). This entitles me 
and others to resist further debate about the L4S identifier. It is 
those like you and David who are holding back the WG - by still 
proposing different identifiers. The question of the relative importance of:
     a) addressing RFC3168 AQMs
versus
     b) viability of the chosen identifier
was well understood by all those who made a choice last May.

>> An L4S congestion control acts at the sender in one direction. The other direction might use a completely different congestion control. This is outlined in l4s-arch, but it's no different from the architecture of any CC. For instance, if Cubic is acting in one direction, you can have Vegas or BBR or whatever in the other.
> That is indeed a case that would be problematic for the two-DSCP scheme as I presented it.  I think you could salvage it by adding explicit feedback of the relevant information to AccECN.

[BB] AccECN is TCP only - you are ignoring QUIC and RTCP. And there is 
extremely limited space in TCP anyway.

We all understand that you are quite prepared to make L4S depend on 
ridiculous additions to standards track transport protocols, because you 
care if L4S fails to deploy.

> Or, for the purposes of the experiment, require that L4S signalling be used in both directions or not at all.  Or - and this is my preferred option - switch to a non-ambiguous signalling scheme where these complications are far less necessary.  Your choice.

[BB] You need to understand that others are entitled to weigh the 
importance of different issues differently to you. They have made their 
choices already (twice), and you are in the rough.


>
> Using a single DSCP would be adequate for preventing L4S traffic originating within the participating network from escaping it, provided the appropriate filtering is actually implemented at the network border.

[BB] Correct. The "appropriate filtering" condition is one of the major 
problems (see later).

> It would prevent, for example, L4S endpoints in two *different* participating networks from accidentally making an L4S connection over the non-participating path between those networks.

[BB] Correct. This is the /only/ case where the idea adds anything.

> But it would not prevent L4S endpoints from being set up outside the participating network entirely and communicating with each other over unprepared networks.

[BB] Correct.

> It does have the virtue of being very easy to set up.

[BB] Incorrect. Adding scarce bits to every standard transport protocol 
when they have no purpose other than to make an experimental idea work 
is not easy, let alone very easy.

>> You are proposing that, in a connection between host X and host Y, if X sends using a scalable CC, it has to set both ECT(1) and a defined L4S DSCP. Then when peer Y sends, it has to set the same DSCP on the packets it sends to X. There are a number of problems with this:
>> a) There is no reason or need for peer Y to know anything about L4S.
> If a host is participating in an L4S-TCP transport connection then *of course* it needs to know about L4S, if nothing else to be able to provide the correct feedback.  Maybe that is different for QUIC et al.

[BB] No. The receiver doesn't know the congestion control being used by 
the sender.

The IETF builds protocols as independent components wherever possible. 
An AccECN receiver rightly knows whether the sender supports AccECN, but 
not what CC it is using. AccECN is intended to update RFC3168, and can 
be used for any congestion control: Reno, Cubic, BBR, DCTCP, Prague, 
whatever.

QUIC, RTCP and DCCP already provide general-purpose ECN feedback. AccECN 
makes TCP's ECN feedback similarly generic, replacing the feedback in 
RFC3168, which was specific to only one class of congestion controls. 
SACK did a similar job for loss feedback.

Dumb reflection from the receiver is a lower layer facility than flow 
regulation, which can evolve on top of it. it can be thought of as the 
second-lowest sub-layer of Layer 4, as follows:

| Application Layer         |
+---------------------------+
| Other Transport functions |2-ended
| Flow Regulation           |1-ended (sender)
| Feedback                  |2-ended
| Endpoint Identification   |2-ended
+---------------------------+
| Internetwork Layer        |

This influential paper about transport sub-layers separated out flow 
regulation as a sub-layer: 
http://conferences.sigcomm.org/hotnets/2008/papers/15.pdf
Taking sub-layering further, feedback (of delivery sequence and timing 
as well as ECN) is actually a sub-layer below flow regulation, because 
other higher transport functions depend on it, such as loss recovery, 
setting of timeouts, and so forth.

>
>> So how does it know to set this DSCP? A host doesn't normally even check what the incoming DSCP is.
> Not reflecting the DSCP as required by the experimental protocol would indicate that it's not participating in the experiment.  That should be a strong indicator to the sender that it must switch out of L4S mode immediately.

[BB] As above, making the L4S experiment depend on L4S deployment at the 
receiver is only of interest to those who don't care if the L4S 
experiment fails to deploy.

>
>> b) This currently would only be useful to make the L4S DSCP scheme work. So it would seem unlikely to be adopted in any of the busy WGs working on these transport protocols. Particularly given it's unorthodox architecture (overloading a L3 protocol with an additional L4 meaning)
> Protecting an "alternate ECN" scheme with a DSCP is explicitly called out as a possibility in published RFCs.  The fact that is hasn't actually been done before should not be a barrier in itself.

[BB] The DSCP has been used for confinement - of pre-congestion 
notification (PCN):
     https://tools.ietf.org/wg/pcn/
Indeed Ruediger, Steven Blake and myself were involved in that.

PCN was one of a number of proposals for two ECN severity levels that 
motivated the publication of RFC4774  (all cited in the 2nd para of its 
introduction). PCN went on to be standardized in the IETF but only 
implemented by three router developers (AFAIK).

RFC4774 uses a DSCP in its traditional role as a classifier, as an 
example of a confinement mechanism. RFC4774 was not thinking of using a 
DSCP in the way you are proposing, as part of a per-connection 
cross-layer path traversal feedback loop, without its classifier 
function at all.

>
>> c) You seem to be assuming (or perhaps requiring) that the peer at one end of a connection can only use a scalable CC if the other peer also uses a scalable CC.
>> d) This also means that a scalable CC cannot be tested with a Classic CC in the reverse direction.
>> e) Requiring a 2-ended deployment greatly reduces the chances that any flow will use L4S.
> That last one is not an argument.  We explicitly *want* to limit use of L4S signalling to connections where both endpoints know what's going on.

[BB] Inventing an unnecessary dependency to turn a 2-party deployment 
into a 3-party deployment is only of interest to someone who doesn't 
care about the viability of L4S.

>
>>>> LoF (2) There are no FIFO RFC3168 AQMs a) upstream of the participating network, b) downstream of it, or c) within it (see details below).
>>> Incorrect.  Remember that when using a DSCP as a containment mechanism, it is assumed (as I spelled out previously) that the borders of the network at least bleach that DSCP, and may even block traffic carrying it.  That's how containment works.  So if the L4S traffic crosses a border of the participating network, it automatically switches to Classic mode and there is no more problem with RFC-3168 AQMs on any part of the path.
>> The a/b/c examples I gave (still below) solely involved /customers/ attaching to a single ISP participating in the L4S experiment. If an L4S network bleaches at the border where its own customers attach, how can any L4S traffic ever enter or leave the network? I'm not understanding you.
> Obviously.
>
> In your scenario, the customers are clearly within the participating network as defined by the border at which filtering of the DSCPs occurs.  Which means that any RFC-3168 equipment those customers happen to use, and the consequent effects, are automatically part of the experiment.

[BB] Incorrect. I'm afraid it is just plain wrong to imagine that 
customers are inside the DSCP 'filtering' border of their ISP. An ISP 
re-marks DSCPs ('filters' in your language) at the IP hop that is 
closest to the attached customer but still under its own control. In the 
upstream of an access network, this is the Broadband Network Gateway 
(BNG), Cable Modem Termination System (CMTS), or equivalent. In the 
jurisdictions I know of, competition law allows a customer to source 
their own home gateway. So no ISP relies on DSCP bleaching at a home 
gateway.

>
> I hope you have plans to inform and educate those customers of what ill effects are possible, how to opt out of the experiment to avoid any possible ill effects, and how to inform relevant people about ill effects they might observe in practice during the experiment.
>
>> Specifically:
>> * at the first IP hop controlled by the L4S network, if it bleaches the L4S DSCP from attached L4S sending customers, how does any L4S traffic ever enter the network?
>> * at the last IP hop controlled by the L4S network, if it bleaches the L4S DSCP before forwarding to attached customers, how does any host ever receive L4S traffic?
>>
>> Please don't answer this here. Instead, please address the specific examples below (a/b/c), to keep the discussion concrete.
> Please don't deliberately misunderstand me for the sake of argumentation.  Instead please re-read points 1 and 2 below from my earlier reply, in light of the brief clarification above.

[BB] No-one is /deliberately/ misunderstanding you. Everyone is going to 
misunderstand someone whose definition of an ISP's border is so 
"unorthodox".

>
>> [BB] Now, please address the a/b/c points below.
>>

[BB] Now that you've been corrected on where the ISP's borders are, 
these points are valid critique of your DSCP proposal. So you do need to 
address them. I notice you had snipped them, so I've reinstated.

>
>>>>
>>>> [BB] Now let's see how realistic LoF (2) is by putting this DSCP 
>>>> idea to the test - by hiding these needles in each type of haystack:
>>>>
>>>> a). Upstream:
>>>> Imagine a FIFO RFC3168 AQM in the queue of a home router in the 
>>>> upstream direction. An L4S client sends L4S packets through this 
>>>> AQM, which marks some as CE. Then the network participating in the 
>>>> L4S experiment delivers them, with the L4S-ID DSCP intact, to an 
>>>> on-net server. But the CE markings were RFC3168 not L4S.
>>>>
>>>> b) Downstream:
>>>> Consider an RFC3168 AQM in the network of a corporate site or 
>>>> campus network receiving traffic from a connected L4S-participating 
>>>> network. Or perhaps on the access into a data centre connected to 
>>>> the participating L4S network. Same problem; the DSCP says its L4S, 
>>>> but the CE was RFC3168.

[BB] The point in case (b) is that a provider network rarely (if ever) 
bleaches DSCPs at egress. And a receiving customer network will only 
bleach DSCPs at its ingress if it's using Diffserv itself. Many do not. 
But your DSCP proposal relies on bleaching at every border.


>>>>
>>>> c) Within:
>>>> Consider an RFC3168 AQM within the network participating in the L4S 
>>>> experiment. Same problem. The participating network delivers the 
>>>> packet with the L4S-ID DSCP intact, but the CE marking is RFC3168, 
>>>> not L4S.
>>>>
>>>> In this last case, the L4S-paticipating network has to use 
>>>> techniques like those in l4sops to make sure any FIFO RFC3168 AQMs 
>>>> don't mark ECT(1). That's no different to if ECT(1) alone was the 
>>>> identifier. 
>>> Which leaves open two scenarios where an RFC-3168 could be encountered without crossing a border:
>>>
>>> 1: Within the participating network.  But all RFC-3168 AQMs are supposed to have been removed from such a network before L4S was deployed on it.  So either the onus is on the network operator (who is at least an "interested observer") to deal with the problem, *or* the RFC-3168 AQM is present specifically for the purpose of experimentation, eg. to find out how the combination with L4S traffic really works in the real world.
>>> 2: The path runs entirely outside the participating network.  This implies that two communicating L4S endpoints were deployed outside the experimental network, which would violate the spirit of the experimental deployment and should not be allowed.  The two-DSCP scheme I outlined in my other post specifically attempts to detect and protect against this eventuality.
>>>> In this last case, the L4S-paticipating network has to use techniques like those in l4sops to make sure any FIFO RFC3168 AQMs don't mark ECT(1). That's no different to if ECT(1) alone was the identifier.
>>> Thank you for confirming that you understand that using ECT(1) alone as the identifier suffers from all of these problems.  That was a big sticking point over the past two years, and I'm glad we can now get past that.
>> We identified these potential problems before we brought L4S to the IETF, and at the very first ad hoc meeting about L4S (Prague Jul 2015), we proposed that this should be on the list of requirements that would need to be solved.
> Okay, so why have you not solved it in the intervening time?

[BB] We're trying to find someone interested in continuing this work. 
You already mentioned at the mic in IETF-110 that you'd read my new 
ideas on how to solve this in-band. But unfortunately, I can't find 
anyone who wants to devote their time to evaluating solutions to a 
probably hypothetical problem. As soon as a set of results is produced, 
everyone knows that you will contrive cases where it doesn't work and 
try to convince everyone that these are critical cases for the safety of 
the Internet. So I'm hardly surprised that no-one wants to run around on 
your playing field. But I'll keep on trying to find someone.

>
>> Here, we are putting your idea for using a DSCP to the test. So please address the point about how an L4S-participating network still has to use the techniques in l4sops, whether it adds a DSCP or not.

[BB] I can confirm that you've accepted this point.

Regards


Bob

> My point is that l4s-ops as currently written doesn't solve the problem at hand.  I'm trying to suggest things that should go into either l4s-ops or the core design of L4S, in order to solve the problem.  I'm a problem solver, you see.
>   - Jonathan Morton

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/