Re: [tsvwg] L4S issue #22: CE Ambiguity and Reordering

["Black, David" <[e-mail]>]

Hi Bob,

While this email wasn’t addressed to me, I’ll offer what I hope is a useful response, based on something that I heard in the recorded discussion of this topic after the formal conclusion of the interim meeting.  This note is posted as an individual, e.g., it should not be treated as instructions from a TSVWG chair.

I’ll start by addressing the implied question about what’s wrong with the current Appendix B.1 text in the ecn-l4s-id draft.  That Appendix B.1 is primarily an argument that reordering of CE-marked packets is sufficiently unlikely that it can be safely ignored (itemized points 1-5 are the core of this argument, and are most of the content of Appendix B.1).  While that ordering may be very unlikely, that does not lead to the conclusion that it can be safely ignored – this is because applying the law of large numbers to the overall size/scale/scope of the internet suggests that such reordering is almost certain to happen, somewhere.   Moreover, if that reordering has security implications, e.g., could be used as part of mounting a denial of service attack, then it has to be considered as likely to happen … because the adversary/attacker gets to choose her shots.

The good news starts with something that was mentioned in the discussion after the end of the official part of the interim meeting: Because the reordered packets are CE-marked, a sender congestion response is coming independent of whether the packets are reordered or not.   That point needs to be added to Appendix B.1, as it may be the core of an argument to the effect of: if this unlikely reordering occurs, its effects are limited to a few spurious retransmits and the difference in congestion response to CE vs. loss/drop (in the CE case, ABE backs off somewhat less than the backoff for drop, with credit to Gorry for the reminder about that), which could be argued to be acceptable (I will want to see the detailed text before agreeing).

Thanks, --David

From: tsvwg <[e-mail]> On Behalf Of Bob Briscoe
Sent: Friday, February 21, 2020 3:52 AM
To: Sebastian Moeller; tsvwg IETF list; Holland, Jake
Subject: [tsvwg] L4S issue #22: CE Ambiguity and Reordering


[EXTERNAL EMAIL]
Jake, Sebastian,

In yesterday's tsvwg L4S interim, issue #22 was discussed.
RTT dependence has been moved to issue #28.
Classic ECN FIFO AQM concerns are in issue #16.
That leaves issue #22 solely for the remaining issue about CE ambiguity mentioned by Jake under #22: reordering.

There was a stand-off in the discussion yesterday as to who is now more obliged to produce data about this: the complainant about L4S or the defendant of L4S.

As a defendant, let me explain why the WG needs the complainant to give more data or more argumentation. In summary, this judgement call was made when the WG considered it back in the 2017 timeframe. If you want to resurrect the question, we need new data or new argumentation.


I have pointed to the justification for this not being an issue in the ecn-l4s-id draft. It is pasted below, and has been broadly unchanged since the very first draft (during the process of choosing an identifier, we obviously had this concern ourselves - until we took advice from the tcpm WG). Jake's posting in the issue tracker says this argument relies on RACK deployment. You will see that the argument does not rely on RACK deployment, it is merely even less likely to be an issue as RACK is deployed.

No-one can produce data for the lack of prevalence of a rare event, at least not without enormous cost. So, before we can proceed, if you think it might not be rare, we need to know:

  *   which of that list of points you dispute, with some evidence or argumentation for why it is wrong.
  *   Then the WG can discuss whether that threatens the whole argument or whether you're just clutching at straws (i.e. does the risk of an event with minor impact become worrying when the probability of it happening is the combination of 2 middling probabilities and 3 small probabilities instead of 5 small ones?).
  *   Only then do we need to discuss whether more data is needed to resolve a specific dispute and who needs to produce that data.

/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

   "Risk of reordering Classic CE packets" in

   Appendix B.1<https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-09#appendix-B.1> discusses the resulting ambiguity if packets originally

   marked ECT(0) are marked CE by an upstream AQM before they arrive at

   a node that classifies CE as L4S.  It argues that the risk of re-

   ordering is vanishingly small and the consequence of such a low level

   of re-ordering is minimal.
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

If you follow the ref to the appendix, you find the explanation below.

/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

   Risk of reordering Classic CE packets:  Classifying all CE packets

      into the L4S queue risks any CE packets that were originally

      ECT(0) being incorrectly classified as L4S.  If there were delay

      in the Classic queue, these incorrectly classified CE packets

      would arrive early, which is a form of reordering.  Reordering can

      cause TCP senders (and senders of similar transports) to

      retransmit spuriously.  However, the risk of spurious

      retransmissions would be extremely low for the following reasons:



      1.  It is quite unusual to experience queuing at more than one

          bottleneck on the same path (the available capacities have to

          be identical).



      2.  In only a subset of these unusual cases would the first

          bottleneck support Classic ECN marking while the second

          supported L4S ECN marking, which would be the only scenario

          where some ECT(0) packets could be CE marked by an AQM

          supporting Classic ECN then the remainder experienced further

          delay through the Classic side of a subsequent L4S DualQ AQM.



      3.  Even then, when a few packets are delivered early, it takes

          very unusual conditions to cause a spurious retransmission, in

          contrast to when some packets are delivered late.  The first

          bottleneck has to apply CE-marks to at least N contiguous

          packets and the second bottleneck has to inject an

          uninterrupted sequence of at least N of these packets between

          two packets earlier in the stream (where N is the reordering

          window that the transport protocol allows before it considers

          a packet is lost).



             For example consider N=3, and consider the sequence of

             packets 100, 101, 102, 103,... and imagine that packets

             150,151,152 from later in the flow are injected as follows:

             100, 150, 151, 101, 152, 102, 103...  If this were late

             reordering, even one packet arriving 50 out of sequence

             would trigger a spurious retransmission, but there is no

             spurious retransmission here, with early reordering,

             because packet 101 moves the cumulative ACK counter forward

             before 3 packets have arrived out of order.  Later, when

             packets 148, 149, 153... arrive, even though there is a

             3-packet hole, there will be no problem, because the

             packets to fill the hole are already in the receive buffer.



      4.  Even with the current TCP recommendation of N=3 [RFC5681<https://tools.ietf.org/html/rfc5681>]

          spurious retransmissions will be unlikely for all the above

          reasons.  As RACK [I-D.ietf-tcpm-rack<https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-09#ref-I-D.ietf-tcpm-rack>] is becoming widely

          deployed, it tends to adapt its reordering window to a larger

          value of N, which will make the chance of a contiguous

          sequence of N early arrivals vanishingly small.



      5.  Even a run of 2 CE marks within a Classic ECN flow is

          unlikely, given FQ-CoDel is the only known widely deployed AQM

          that supports Classic ECN marking and it takes great care to

          separate out flows and to space any markings evenly along each

          flow.



      It is extremely unlikely that the above set of 5 eventualities

      that are each unusual in themselves would all happen

      simultaneously.  But, even if they did, the consequences would

      hardly be dire: the odd spurious fast retransmission.  Admittedly

      TCP (and similar transports) reduce their congestion window when

      they deem there has been a loss, but even this can be recovered

      once the sender detects that the retransmission was spurious.
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

Before you ask about bullet 1 (multibottleneck path), I described it as "quite unusual" because the measurement approach used in the only study I could find to support this assertion was rather suspect. The whole argument only hangs on the likelihood of experiencing multiple bottlenecks on your path being "quite unusual", which is supported by:

  *   most paths are from data centres to clients where only the client access link is the bottleneck,
  *   but still many paths do have 2 access bottlenecks, ...
  *   there would be a chance of having the same available bandwidth in both access links in two cases:

     *   the traffic is alone in both links and the capacities of both are the same
     *   there is other traffic in either link that temporarily makes the available bandwidths the same
I think it's fair to say this latter case will be transient, so I'll only proceed with the question of whether the link capacities might be the same...

  *   taking residential or mobile access technologies first, the bandwidths of some tend to be sold in round numbers, however the bandwidth of most access technologies is asymmetric, so the upstream round number of one would have to match the downstream round number of the other
  *   large multi-user access technologies (e.g. DC network access, campus network access, corporate network access) will nearly always be carrying other traffic, so they fall into the transient case
Taking all this, I think it is reasonable to say multiple bottlenecks will happen, but they will be 'quite unusual'. Even if this is not the case, the other 4 bullets lead to a very small probability once all is considered. And the impact if the event does happen is itself minor anyway.

Note also that bullet 2 makes this solely a transition phenomenon. It only becomes a permanent phenomenon if the L4S experiment half succeeds so that Classic ECN and L4S ECN permanently co-exist.

Regards


Bob




--

________________________________________________________________

Bob Briscoe                               http://bobbriscoe.net/