Re: [tsvwg] David Black (individual) on safety of L4S for the Internet

["Black, David" <[e-mail]>]

Bob,
[still posting as an individual, not as WG chair]

> Firstly an appeal to everyone to stop talking about wiping the ECN field
> as if it is acceptable. Bleaching ECN is not a safety mechanism, except
> as a last resort. Treating ECT(1) /as if/ it is Not-ECT is far
> preferable to wiping it so it /is/ Not-ECT.

I concur with that appeal ... but ...

... We've been through an analogous adventure with DSCP bleaching, as RFC 2474 recommends that behavior for DSCPs (treat unknown DSCPs as best effort without changing the DSCP) but the network operators went ahead and bleached DSCPs at network boundaries, as described in Section 1 of RFC 8100 (https://www.rfc-editor.org/rfc/rfc8100.html#section-1).  I strongly suggest reviewing that history towards understanding why things turned out that way.

> Consider an RFC3168 AQM in network A, and an L4S or RFC3168 ECN AQM in
> network B downstream. If network A wipes the ECN field, 

To be specific, my understanding of the wiping under discussion is to rewrite ECT(1) to Not-ECT, not zero out all ECN field values (e.g., rewriting CE to Not-ECT is very wrong and dangerous).  In particular, ECT(0) would not be wiped.

> but the
> bottleneck is in network B, then the wiping destroyed the ECN service in
> network B, with no safety benefit in network A.

But there may be a safety benefit in network B.  Suppose a flow uses DCTCP and the RFC 3168 bottleneck in network B drops a Not-ECT packet that it would have been marked CE if network A had not wiped its ECT(1) mark to Not-ECT.  That drop triggers DCTCP's drop reaction, which is safe for an RFC3168 bottleneck, in contrast to DCTCP's CE-mark reaction, which is not.  Please refer to the prior discussion with Neal Cardwell on DCTCP's differing reactions to drops vs. CE-marks.

Now for the good news (perhaps) ... on this concern:

> Some non-participating networks will be willing to actively take such
> measures. Others won't. Some might be unaware of the L4S experiment.
> Clearly a safety case is weaker if it requires action by
> non-participants. But that doesn't rule it out.

I generally agree.  Over in the IETF Security area, there's a long track record of dealing with this sort of situation, where a counter-measure may be vitally necessary in some cases, but not in others, via the use of "MUST implement, MAY use" requirements.  Such requirements ensure that the counter-measure will be available if a network operator discovers that it is needed.  Among the reasons that this approach makes sense here is that the measures under discussion may be needed by a network operator to counter denial-of-service attacks that take advantage of the L4S low-latency service.

Returning to Alex's original question:

> So I was wondering what your position is, regarding third party actions being necessary  as part of a safety case -
> what limitations should there be, on  third party actions that a safety case may necessitate?

At a minimum, the third parties MUST have the tools readily available to take any safety actions that may become necessary.   Analogy - a fire inspector is not interested in whether the building owner could buy a fire extinguisher at a nearby store - the fire extinguisher has to be installed and maintained so that it's readily available if needed.

Beyond that, I welcome discussion on how to reduce the occurrence of situations in which such actions become necessary.

Thanks, --David (as an individual)

> -----Original Message-----
> From: Bob Briscoe <[e-mail]>
> Sent: Thursday, May 14, 2020 4:54 PM
> To: [e-mail]; [e-mail]; Black, David
> Subject: Re: [tsvwg] David Black (individual) on safety of L4S for the Internet
> 
> 
> [EXTERNAL EMAIL]
> 
> Alex, David, all,
> 
> Firstly an appeal to everyone to stop talking about wiping the ECN field
> as if it is acceptable. Bleaching ECN is not a safety mechanism, except
> as a last resort. Treating ECT(1) /as if/ it is Not-ECT is far
> preferable to wiping it so it /is/ Not-ECT.
> 
> Consider an RFC3168 AQM in network A, and an L4S or RFC3168 ECN AQM in
> network B downstream. If network A wipes the ECN field, but the
> bottleneck is in network B, then the wiping destroyed the ECN service in
> network B, with no safety benefit in network A.
> 
> Now to your point, Alex, pls see inline...
> 
> 
> On 14/05/2020 18:03, [e-mail] wrote:
> > Hi David,
> >   you write "In contrast to reliance on TCP Prague, that combination of FQ and
> bleaching looks like it could provide a robust safety case."
> >
> > Some of those who have responded to the consensus call seem to
> assume  that a safety case should not necessitate any action on the part of third
> parties, eg operators who are running RFC3168 AQMs. That you reject
> bleaching not on that ground, but because it would nobble the use of ECT(1),
> makes me wonder if your position is more nuanced.
> >
> > Some responders are evidently more  relaxed about third parties needing to
> act in order to maintain effective  congestion control on the internet, as
> evidenced by their willingness to contemplate ECT(1) bleaching.
> >
> > If action on the part of third parties is allowed to be necessary as part of a
> safety case, as long as that action is sufficiently easy, then it seems possible
> that an action could be found which does make L4S safe without making ECT(1)
> bleaching likely, or make it  limited in scope. For example, maybe there is a
> simple way for operators to signal to endpoints that RFC3168 AQMs are in use.
> So I was wondering what your position is, regarding third party actions being
> necessary  as part of a safety case - what limitations should there be, on  third
> party actions that a safety case may necessitate?
> 
> [BB] Regarding safety actions by nodes or networks not taking part in
> the ECT(1) experiment; I don't think the real question is whether a
> safety case is 'allowed' to include these. As long as a proposal clearly
> states which measures would need non-participant action, then we can all
> decide how much weight to give their proposal.
> 
> Some non-participating networks will be willing to actively take such
> measures. Others won't. Some might be unaware of the L4S experiment.
> Clearly a safety case is weaker if it requires action by
> non-participants. But that doesn't rule it out.
> 
> This whole ECT(1) question does not have a formula that gives a correct
> answer. Different people weigh different factors more strongly than
> others, and the current quest is to find out which factors weigh most
> heavily for most people.
> 
> For instance, if there is a simple way to detect any problems and
> configure RFC3168 AQMs to be safer, some people will find that to be an
> acceptable compromise if they believe that single-queue RFC3168 AQMs are
> likely to be rare, even though they would not accept such a compromise
> if these AQMs were widespread. Others see safety as an absolute matter.
> For them, an inherently safe signalling system is paramount,
> particularly if they are unsure whether single-queue RFC3168 AQMs are
> rare. Others don't see this as a safety issue anyway, because the
> unfairness disappears at low link rates.
> 
> 
> Bob
> >
> > Alex
> >
> >
> >
> > On Friday, May 8, 2020, 9:10:18 PM GMT+1, Black, David
> <[e-mail]> wrote:
> >
> >
> >
> >
> >
> >
> >
> >
> > This the longer explanation of why I find myself in group 2 on the ECT(1)
> question because “If you believe L4S will not be safe for the internet without
> significant architectural changes, you are in this group.”
> >
> >
> >
> > Current practice is not to mix DCTCP-like traffic (1/p-class congestion control)
> with TCP-like traffic (1/sqrt(p)-class congestion control, e.g., NewReno) in the
> same queue because the DCTCP-like traffic outcompetes the TCP-like traffic to
> the point of starvation of the latter.  By using ECT(1) as the only classifier for
> low-latency traffic, L4S causes mixing of those two classes of traffic at network
> nodes that have not been modified to use ECT(1) as a classifier.  Hence the L4S
> experiment proponents are responsible for the safety of that traffic mixing at
> unmodified nodes.  I understand the L4S approach to achieving safety at
> unmodified nodes to be use of TCP Prague, specifically its heuristics that detect
> RFC 3168 bottlenecks.  As previously discussed, this traffic mixing does not
> cause problems at otherwise unmodified nodes that use FQ.
> >
> >
> >
> > For the proposed Internet-wide L4S experiment, the technology to be tested is
> necessarily experimental, but the safety of the experiment needs to be
> fundamental to avoid serious damage if things go wrong.  I do not currently see
> a strong safety case for L4S+TCP Prague – an example of such a strong safety
> case can be found in QUIC congestion control, where the fact that it is based on
> NewReno congestion control provides strong assurance that QUIC congestion
> control is safe for the Internet.   In contrast, TCP Prague is still research (it’s
> great research and I applaud the people who have invested their time and
> effort in it for what has been achieved), but a safety case ought not to be based
> on not-yet-stable research, and I don’t think another 6 months of testing and
> bug fixing will leave TCP Prague as anything other than research (ruling out
> group 3 for me).  Further, the L4S low-latency service will likely be used by
> protocols other than TCP Prague, which may not receive the level of scrutiny
> and testing that has been applied to TCP Prague.  The safety case needs to
> come from elsewhere.
> >
> >
> >
> > If ECT(1)-marked traffic starts causing problems, network operators are not
> going to wait for TCP Prague or other protocol implementations to be patched
> – they’re going to take action, e.g., as Neal Cardwell describes (and I’ve heard
> about this sort of potential consequence in other conversations, so please don’t
> blame Neal for this):
> >
> >
> >
> >> - L4S flows potentially causing unfairness in RFC3168 ECN bottlenecks has
> been mentioned as a potential concern. However, a robust RFC3168 ECN
> bottleneck
> >> should already have a mechanism to avoid unfairness caused by flows that
> are marked as ECT(0|1) and yet not performing RFC3168 responses. In
> particular,
> >> many of the large sources of known deployments of  RFC3168 --  Linux
> fq_codel and cake -- are already deployed with fair queueing. In such
> bottlenecks L4S
> >> traffic should not cause harm to other non-L4S flows. Furthermore, if there
> really are ISPs with deployments of RFC3168 bottlenecks that have neither FQ
> nor
> >> any other protection from non-RFC3168-ECT(1) flows, then they can bleach
> incoming ECT(1) code points to Not-ECT and treat L4S as Not-ECT (ISPs typically
> >> already transform the DSCP byte at their ingress anyway). So I do not see
> harm to RFC3168 ECN bottlenecks as a prohibitive concern.
> >
> >
> > In contrast to reliance on TCP Prague, that combination of FQ and bleaching
> looks like it could provide a robust safety case.  It would be most unfortunate if
> things wind up depending on this, especially bleaching, as the fact that ECT(1) is
> not bleached at network boundaries was a major reason for selection of ECT(1)
> as the identifier for low-latency L4S traffic.  Among the implications is that if
> the L4S experiment fails, that bleaching could be around much longer, making
> it infeasible to use ECT(1) for anything else Internet-wide.  In my view, it is the
> responsibility of the L4S proponents to design the L4S  technology and structure
> the experiment in a fashion that makes ECT(1) bleaching unlikely and/or likely
> to be limited in scope if it happens.
> >
> >
> >
> > Thanks, --David
> >
> > ----------------------------------------------------------------
> >
> > David L. Black, Sr. Distinguished Engineer, Dell EMC
> >
> > Dell Technologies, Infrastructure Systems Group
> >
> > 176 South St., Hopkinton, MA  01748
> >
> > [phone]           Mobile: [phone]
> >
> > [e-mail]
> >
> > ----------------------------------------------------------------
> >
> >
> >
> >
> >
> >
> 
> --
> ________________________________________________________________
> Bob Briscoe                               http://bobbriscoe.net/
>