[tsvwg] L4S issue #22: CE Ambiguity and Reordering

[Bob Briscoe <[e-mail]>]

Jake, Sebastian,

In yesterday's tsvwg L4S interim, issue #22 was discussed.
RTT dependence has been moved to issue #28.
Classic ECN FIFO AQM concerns are in issue #16.
That leaves issue #22 solely for the remaining issue about CE ambiguity 
mentioned by Jake under #22: reordering.

There was a stand-off in the discussion yesterday as to who is now more 
obliged to produce data about this: the complainant about L4S or the 
defendant of L4S.

As a defendant, let me explain why the WG needs the complainant to give 
more data or more argumentation. In summary, this judgement call was 
made when the WG considered it back in the 2017 timeframe. If you want 
to resurrect the question, we need new data or new argumentation.


I have pointed to the justification for this not being an issue in the 
ecn-l4s-id draft. It is pasted below, and has been broadly unchanged 
since the very first draft (during the process of choosing an 
identifier, we obviously had this concern ourselves - until we took 
advice from the tcpm WG). Jake's posting in the issue tracker says this 
argument relies on RACK deployment. You will see that the argument does 
not rely on RACK deployment, it is merely even less likely to be an 
issue as RACK is deployed.

No-one can produce data for the lack of prevalence of a rare event, at 
least not without enormous cost. So, before we can proceed, if you think 
it might not be rare, we need to know:

  * which of that list of points you dispute, with some evidence or
    argumentation for why it is wrong.
  * Then the WG can discuss whether that threatens the whole argument or
    whether you're just clutching at straws (i.e. does the risk of an
    event with minor impact become worrying when the probability of it
    happening is the combination of 2 middling probabilities and 3 small
    probabilities instead of 5 small ones?).
  * Only then do we need to discuss whether more data is needed to
    resolve a specific dispute and who needs to produce that data.


/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ 


    "Risk of reordering Classic CE packets" in
    Appendix B.1  <https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-09#appendix-B.1>  discusses the resulting ambiguity if packets originally
    marked ECT(0) are marked CE by an upstream AQM before they arrive at
    a node that classifies CE as L4S.  It argues that the risk of re-
    ordering is vanishingly small and the consequence of such a low level
    of re-ordering is minimal.

/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ 


If you follow the ref to the appendix, you find the explanation below.

/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ 


    Risk of reordering Classic CE packets:  Classifying all CE packets
       into the L4S queue risks any CE packets that were originally
       ECT(0) being incorrectly classified as L4S.  If there were delay
       in the Classic queue, these incorrectly classified CE packets
       would arrive early, which is a form of reordering.  Reordering can
       cause TCP senders (and senders of similar transports) to
       retransmit spuriously.  However, the risk of spurious
       retransmissions would be extremely low for the following reasons:

       1.  It is quite unusual to experience queuing at more than one
           bottleneck on the same path (the available capacities have to
           be identical).

       2.  In only a subset of these unusual cases would the first
           bottleneck support Classic ECN marking while the second
           supported L4S ECN marking, which would be the only scenario
           where some ECT(0) packets could be CE marked by an AQM
           supporting Classic ECN then the remainder experienced further
           delay through the Classic side of a subsequent L4S DualQ AQM.

       3.  Even then, when a few packets are delivered early, it takes
           very unusual conditions to cause a spurious retransmission, in
           contrast to when some packets are delivered late.  The first
           bottleneck has to apply CE-marks to at least N contiguous
           packets and the second bottleneck has to inject an
           uninterrupted sequence of at least N of these packets between
           two packets earlier in the stream (where N is the reordering
           window that the transport protocol allows before it considers
           a packet is lost).

              For example consider N=3, and consider the sequence of
              packets 100, 101, 102, 103,... and imagine that packets
              150,151,152 from later in the flow are injected as follows:
              100, 150, 151, 101, 152, 102, 103...  If this were late
              reordering, even one packet arriving 50 out of sequence
              would trigger a spurious retransmission, but there is no
              spurious retransmission here, with early reordering,
              because packet 101 moves the cumulative ACK counter forward
              before 3 packets have arrived out of order.  Later, when
              packets 148, 149, 153... arrive, even though there is a
              3-packet hole, there will be no problem, because the
              packets to fill the hole are already in the receive buffer.

       4.  Even with the current TCP recommendation of N=3 [RFC5681  <https://tools.ietf.org/html/rfc5681>]
           spurious retransmissions will be unlikely for all the above
           reasons.  As RACK [I-D.ietf-tcpm-rack  <https://tools.ietf.org/html/draft-ietf-tsvwg-ecn-l4s-id-09#ref-I-D.ietf-tcpm-rack>] is becoming widely
           deployed, it tends to adapt its reordering window to a larger
           value of N, which will make the chance of a contiguous
           sequence of N early arrivals vanishingly small.

       5.  Even a run of 2 CE marks within a Classic ECN flow is
           unlikely, given FQ-CoDel is the only known widely deployed AQM
           that supports Classic ECN marking and it takes great care to
           separate out flows and to space any markings evenly along each
           flow.

       It is extremely unlikely that the above set of 5 eventualities
       that are each unusual in themselves would all happen
       simultaneously.  But, even if they did, the consequences would
       hardly be dire: the odd spurious fast retransmission.  Admittedly
       TCP (and similar transports) reduce their congestion window when
       they deem there has been a loss, but even this can be recovered
       once the sender detects that the retransmission was spurious.

/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ 


Before you ask about bullet 1 (multibottleneck path), I described it as 
"quite unusual" because the measurement approach used in the only study 
I could find to support this assertion was rather suspect. The whole 
argument only hangs on the likelihood of experiencing multiple 
bottlenecks on your path being "quite unusual", which is supported by:

  * most paths are from data centres to clients where only the client
    access link is the bottleneck,
  * but still many paths do have 2 access bottlenecks, ...
  * there would be a chance of having the same available bandwidth in
    both access links in two cases:
      o the traffic is alone in both links and the capacities of both
        are the same
      o there is other traffic in either link that temporarily makes the
        available bandwidths the same
        I think it's fair to say this latter case will be transient, so
        I'll only proceed with the question of whether the link
        capacities might be the same...
  * taking residential or mobile access technologies first, the
    bandwidths of some tend to be sold in round numbers, however the
    bandwidth of most access technologies is asymmetric, so the upstream
    round number of one would have to match the downstream round number
    of the other
  * large multi-user access technologies (e.g. DC network access, campus
    network access, corporate network access) will nearly always be
    carrying other traffic, so they fall into the transient case

Taking all this, I think it is reasonable to say multiple bottlenecks 
will happen, but they will be 'quite unusual'. Even if this is not the 
case, the other 4 bullets lead to a very small probability once all is 
considered. And the impact if the event does happen is itself minor anyway.

Note also that bullet 2 makes this solely a transition phenomenon. It 
only becomes a permanent phenomenon if the L4S experiment half succeeds 
so that Classic ECN and L4S ECN permanently co-exist.

Regards


Bob


-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/