Re: [tsvwg] Update to Position Statement on ECT(1)

[Bob Briscoe <[e-mail]>]

David,

On 19/05/2020 14:19, Black, David wrote:
>
> [still posting as an individual, not WG chair]
>
>
> Bob,
>
> > [BB] I don't know how you get to one codepoint in ACE being enough for 
> CE. The whole idea of AccECN is to be robust to ACK loss and thinning.
>
> > If it was as simple as you make out, we could have sorted out this 
> problem snip-snap and all be home in time for tea.
>
> I think you may have missed the point.
>

[BB] I'll get to who is missing the point later...

First,
a) Suspending disbelief regarding ECN marking and tunnels for one moment,;
b) suspending any judgement as to whether the "ECT(1) as an output" boat 
has just sailed;
c) and setting aside terminology problems (a 1/p decrease is just as 
multiplicative as a 1/2 decrease),

> The topic under discussion is a 2-level marking system that retains 
> the semantics of CE as calling for an RFC3168-style Multiplicative 
> Decrease (MD), and uses ECT(0) [remarked from ECT(1)] as the queue 
> occupancy mark for 1/p congestion control.  With two signals (CE and 
> ECT(0)) arriving at the receiver, there need to be ways to reflect 
> both to the sender.
>
> The MD signal (CE) overrides the queue occupancy signal, as it calls 
> for a much larger backoff.
>

[BB]
The idea of 1/p is for the extent of congestion signalling to give a 
/variable/ reduction between 0 and 0.5. Not just a /small/ reduction.

RFC3168 CE mandates a once-per-RTT reduction that has been fixed 
differently in different CCs: 0.5 in Reno, 0.3 in Cubic, 0.15-0.3 in ABE 
Cubic or ABE Reno. None of these numbers is "larger" than the 0.5 that 
1/p will give when there's an "Oh sh*t" moment.

So please be correct and precise and say that a CE response is needed 
for the possibility of encountering a single-queue RFC3168 AQM /during 
transition/.

Beyond transition, a fixed 'back-stop'  response in addition to 1/p 
signalling would be completely redundant.

Protocols should be designed with their deployment progression in mind. 
This is what Jana said - the future is a lot longer than the past or the 
present. Other people prioritize this way of thinking too. When there is 
no right answer, a consensus call is designed to find out how most 
people think.


> ACK thinning should be less of a concern for MD because any arrival of 
> CE (or detection of a drop) at the receiver is supposed to cause MD at 
> the sender, and that sender MD reaction is limited to once per RTT.
>

[BB]
OK, the proposal is one-for-one feedback of CE marks, i.e. completely 
unreliable feedback delivery. That's asking the tcpm WG to overturn the 
requirement that CE feedback must be reliable [RFC7560], and to undo the 
reliable CE feedback that was in RFC3168. Doesn't sound like progress.

Remember the idea of AccECN is a single improved ECN feedback protocol 
for TCP, not a fork with different wire protocols depending on the 
congestion control. So this will be used by RFC3168 senders that only 
recognize CE, or where only CE is induced on the path.

This proposal seems to be based on the totally unproven idea that 
usually/sometimes/maybe at least one CE mark per RTT might get through 
ACK thinning or ACK loss. Why? If there's a congestion situation on the 
back channel, reverse queues will be large, with a potentially large 
fraction of ACKs lost. And thinning is typically more aggressive the 
more ACKs are queued up. It's probable/possible that the window in the 
forward direction will be reducing. Then there would be less ACKs sent 
per RTT in the first place. For RFC3168-style marking, often only a 
small proportion of the forward packets is CE-marked, even during fairly 
congested episodes on the forward path (e.g. 1%, or maybe 5-10% 
worst-case if there's also bad congestion on the forward path).

Surviving_CE_marks/RTT = window * forward_CE_marking_prob * (1 - 
ACK_loss_prob) * (1 - proportion_of_ACKs_thinned)

What makes anyone believe that feedback of at least one CE mark will get 
through per RTT in this scenario? Everything tends towards the opposite 
conclusion. Congestion feedback has to be carefully designed to survive 
congestion (the clue is in the name).


And... if that argument isn't enough (it is, actually), what evidence is 
there that all congestion controls used by real-time applications 
nowadays respond to no more than one congestion signal per RTT? The 
IETF's equation-based CC TFRC does, but most real-time congestion 
controllers respond to the level of loss or ECN probability in 
proprietary ways.

> The underlying reasons why a single codepoint may suffice to signal MD 
> support via the AccECN field are two-fold:
>
>   * The MD signal (CE) overrides the queue occupancy signal (ECT(0)),
>     as it calls for much more sender backoff, so there’s no point in
>     AccECN for queue occupancy if a CE has been received.
>

[BB] See above about being clear on why a second signal is needed - 
backstop is not the reason.

>  *
>
>
>   * The MD signal does not require the gradations of AccECN – it’s an
>     undifferentiated “Oh sh*t!” signal calling for MD backoff (and
>     reversion to RFC3168 sender behavior).
>

[BB] Again, see above - the second signal is purely for transition, so 
no need to try to convince yourself that it's also a useful backstop in 
its own right. Extent-based congestion signalling inherently contains 
its own backstop.


> That would still leave the rest of the AccECN codepoints to reflect 
> the queue occupancy signal in the absence of CE arriving or a drop 
> being detected.
>

[BB] Next, did no-one hear me say that the AccECN ACE field was 
finalized after lengthy consultations with the folks who design and 
build segmentation offload hardware? A more complex encoding of the 
3-bit ACE field than a simple counter is very unlikely to fly.

There's a large system of dependencies here. Every detail can be a 
show-stopper. You are talking to someone who has been trying out all the 
different ways of putting together this jig-saw for years. I am getting 
thoroughly ticked off with being told that I'm the one missing the point.


_________________
And of course, we can't suspend disbelief regarding tunnelling...

The way ECN interacts with protocol layering at the IP layer and below 
has to be laid down over many years, long before you can start to rely 
on it at higher layers.

Many IP-in-IP tunnels and particularly IPSec implementations comply with 
RFC6040.
You (David) helped with those liaisons we did with 3GPP and the IEEE, 
getting them to take note of RFC6040.
They, particularly the 3GPP, put a lot of work into ECN.
Then there was all the evangelizing of RFC6040 I did in loads of intarea 
WGs.
That resulted in all sorts of people going off and fixing their 
implementations.

The guess is that 60-70% of Internet paths involve some form of 
layering, tunnelling, overlay, etc. At the moment, L4S just works with 
any of the whole history of ECN tunnels: RFC3168, RFC4301 and RFC6040.

This ECT1->0 proposal would make L4S not work at all with /any/ existing 
tunnels or layerings. Instead, it proposes to obsolete and reverse 
RFC6040 (standards track) {1} and force L4S to /depend/ on deployment of 
this new tunnel behaviour before L4S will work at all (even tho it 
doesn't currently depend on RFC6040).

There are a hundred or so companies that have said they are planning to 
deploy L4S, many with products imminent, but hanging on this IETF 
decision. Under this proposal, they will then be expected to wait a few 
years (probably more like a decade) until these tunnels are designed, 
specified, implemented and deployed. Until then, L4S just doesn't work 
at all over 60-70% of the Internet.

This isn't going to happen. This is fantasy. Can you not see how this 
will look? Actually, how it does look? It looks like you are trying to 
give the IETF a reputation for shooting itself in both feet, then laying 
land mines and blowing off both legs.

We need to focus on other ways of mitigating any problems with existing 
single-queue RFC 3168 AQMs.


Bob

{1} Not to mention reversing other standards track RFCs, like RFC6660.


> Thanks, --David
>
> *From:*tsvwg <[e-mail]> *On Behalf Of *Bob Briscoe
> *Sent:* Tuesday, May 19, 2020 8:59 AM
> *To:* Martin Duke; Holland, Jake
> *Cc:* TSVWG
> *Subject:* Re: [tsvwg] Update to Position Statement on ECT(1)
>
> [EXTERNAL EMAIL]
>
> Martin,
>
> On 18/05/2020 23:11, Martin Duke wrote:
>
>     Jake,
>
>     I'm intrigued by this discussion of the ECT(1)->ECT(0) proposal,
>     as something that could definitively solve the safety concerns.
>     I'll make two unrelated points:
>
>     1) If the current L4S proposal is in need of an MD signal, there's
>     always dropping the packet.  Although packet loss is bad, maybe
>     some drops at the end of slow start is the tradeoff we have to
>     make to get low latency. Implementations really concerned about
>     loss can be less aggressive during slow start.
>
>     2) Clearly, there is no AccECN signaling problem for
>     ECT(1)->ECT(0) for QUIC, and for TCP paths where the option gets
>     through. It this is an issue of the three ACe bits, I think one
>     codepoint in ACE would be sufficient to indicate that a CE mark
>     was received, which IMO would trump whatever other feedback is in
>     that header.. Unless there's some sort of performance cliff in not
>     being able to encode 7 ECT(0) marks, this seems like a non-problem
>
>
> [BB] I don't know how you get to one codepoint in ACE being enough for 
> CE. The whole idea of AccECN is to be robust to ACK loss and thinning. 
> If it was as simple as you make out, we could have sorted out this 
> problem snip-snap and all be home in time for tea.
>
> As we all try to remove more and more latency, we are all going to 
> find that the problems get harder and harder - little details matter a 
> lot. Without really solid feedback precision, you lose the consistent 
> latency of L4S.
>
> Richard Scheffenegger and I designed a 4-3-1 codepoint scheme for ACE 
> at one point (4 for CE, 3 for ECT(1) and one for Not-ECT). See 
> https://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-03#section-3.2.1 
> Table 3. However, it was removed because it was "jack of all trades, 
> master of none." By trying to cover all the codepoints, it didn't give 
> sufficient robustness to any.
>
> Even using all 8 values of the 3-bit counter for feeding back just one 
> codepoint is on the edge of what is needed today. I sometime liken the 
> ACE part of AccECN to a "stalking horse" - i.e. a vehicle that works 
> for today, while also acting as a platform to get the larger AccECN 
> Option deployed for the longer term - when the ACE field might have 
> become too small for the possible future level of ACK thinning on some 
> paths.
>
> In fact, over the years {1} that we've been trying to get AccECN 
> standardized, you will find all sorts of weird and wonderful 
> compressed encodings, but in the end, as well as inadequate 
> robustness, arguments for simplicity took precedence. Not just because 
> complexity breeds bugs (which is not an insignificant consideration 
> for TCP), but also because the semantics of ACE also have to be 
> supported by segmentation offload hardware.
>
> When, I highlighted the problems with tunnelling that the ECT1->0 
> scheme has, I only hinted at these problems with TCP, because I 
> thought incompatibility with tunnelling should be a sufficient argument.
>
> In summary, if you think deploying a change to IP is easy, you've 
> probably not absorbed the full breadth of the problem.
>
>
> Regards
>
>
>
> Bob
>
> {1} Since Oct 2005: 
> https://tools.ietf.org/html/draft-briscoe-tsvwg-re-ecn-tcp-00#section-4.1
>
>
>     Martin (as an individual)
>
>     On Sun, May 10, 2020 at 5:09 PM Holland, Jake
>     <jholland=[e-mail]
>     <mailto:[e-mail]>> wrote:
>
>         Hi Mike,
>
>         From: "C. M. Heard" <[e-mail] <mailto:[e-mail]>>
>         > Yes, combinations marked (**) below would have to changed
>         from RFC 6040:
>         ....
>         > Similar changes would be needed for
>         draft-ietf-tsvwg-rfc6040update-shim and
>         draft-ietf-tsvwg-ecn-encap-guidelines.
>         >
>         > Clearly, the need to get such changes deployed would be a
>         barrier to barrier to adoption.
>
>         Yes.  I think in a recent thread I heard it confirmed that
>         current tunnel
>         handling of these is kind of spotty today, specifically:
>
>           > as an endpoint we'll be dealing with weird inconsistencies
>         that basically
>           > never fully understood anything beyond "don't lose CE
>         marks" and maybe
>           > "loss is acceptable in confusing cases", if we're lucky.
>
>           [BB] See reply to Dave Taht just now, which pretty much
>         confirms what
>           you've just said.
>
>         that's from here:
>         https://mailarchive.ietf.org/arch/msg/tsvwg/QudqLu1RTQZCVnS4HNf8jnZ_kIY/
>
>         (I think the Dave Taht reply he was referencing was this one:
>         https://mailarchive.ietf.org/arch/msg/tsvwg/2ElPK72IiFg2gHJZ_rLMUKTDfCI/
>         )
>
>         I'm beginning to think the reason I've come down differently
>         than the
>         L4S team on the judgement call for this approach being better,
>         in light
>         of the state of tunneling encapsulation deployments, might
>         boil down to
>         a disagreement over the answer to a question like:
>           Which is better:
>           - losing the safety response of MD from a loaded classic
>         queue, or
>           - losing some of the reliability on the low-latency response
>         when there
>             is a dualq on-path?
>
>         I'm beginning to think we might be stuck with one of those
>         options for
>         tunneled paths until tunnel decap implementations can be
>         widely upgraded
>         in deployment, however this lands.
>
>         > - the existing accecn spec would often lose non-CE signals
>         >
>         > Actually, I would go farther and say that something rather
>         different from the existing AccECN draft would be needed.
>         AccECN provides accurate feedback of the number of CE marks
>         observed. Under the proposed scheme L4S would need to getting
>         accurate feedback of the number of ECT(0)  (pre-congestion /
>         some congestion) marks. AccECN would need to be re-worked to
>         provide both that and, in addition, either the existing
>         ECE/CWR handshake or something else that performs the
>         equivalent function. The most obvious solution would be to
>         repurpose NS and one or more currently reserved flag bits (or
>         use other ideas from RFC 7560 Sec 5.2) and leave ECE/CWR
>         unchaged. I note in passing the SCE proposal would have to do
>         something along the same lines (though AFAICT that has not yet
>         been fully fleshed out).
>
>         Agreed, I think a different feedback than AccECN would be
>         smarter if the
>         ECT(1)->ECT(0) approach goes forward, and I like the NS
>         reflection approach
>         that SCE's implementation started with.  (Although it might
>         lose fidelity
>         from some ack aggregation responses, I'd expect usually that
>         the marking
>         rate maintains proportionality on the low-congestion signal,
>         and where that
>         fails, the standard ECE response is at least reliable, so it
>         covers the
>         safety considerations the same way as classic marking.)
>
>         However, I also think for anyone who disagrees, other viable
>         approaches
>         for the feedback might be possible.  But in that direction, I
>         do think
>         it probably would need to differ from AccECN.  This would of
>         course need to
>         be nailed down in the end, though I didn't get into it in the
>         first email.
>         But the potential complexity here is one of the reasons I rate the
>         suggestion as perhaps a major architectural change for L4S.
>
>         I tend to think that the per-ECT(0) reflection in NS is the
>         best way,
>         but I don't think it would change the rest of the argument if
>         that position
>         turned out wrong.
>
>         > - For paths with multiple AQMs, the classifier partially
>         loses integrity in
>         >   later AQMs when earlier AQMs are loaded.  (Note also the
>         worse downside
>         >   that increasing deployment of new AQMs potentially reduces
>         the fidelity
>         >   further.)
>         >
>         > If I understand what is being said, this is because ECT(0)
>         would become ambiguous, as it can appear either on an L4S
>         packet with a pre-congestion marking, or a non-L4S packet. 
>         Doesn't the same issue exist with the current L4S proposal for
>         CE-marked packets?
>
>         Yes, but the L4S specs go over this and walk through the
>         reasoning for
>         why they landed on classifying CE into the LL queue, and the
>         net result
>         in that case is  that the ECT(1)->CE marking strategy that L4S
>         currently
>         follows will keep the 1/p packet signals in the LL queue for
>         the later
>         hops.
>
>         (The potential problems were mostly limited to mis-classified
>         marked
>         classic traffic, which will tend to be fewer in number and
>         also less
>         severe given that the flow is slated to back off anyway, plus
>         a review of
>         the main implementations suggested they wouldn't be doing
>         double-backoffs
>         if the CE packet was out of order, IIRC.)
>
>         It might be better to phrase this not as "loses integrity",
>         but rather as
>         "might systematically increase the latency experienced" for
>         the 1/p signal,
>         since when there are multiple dualqs in line, those packets
>         (but not others
>         from L4S flows) will land in the classic queue on the later
>         dualqs.  This
>         is arguably a worse downside than the classification failure
>         from putting
>         CE-marked classic traffic in the LL queue.
>
>         > Actually, it seems to me that this approach would yield
>         exactly the same congestion signaling capability as using
>         ECT(1) as a  pre-congestion / some congestion mark. All that
>         has been done is to reverse the role of ECT(1) and ECT(0)
>         compared to what the SCE draft and RFC 6040 envisioned. In
>         other words:
>         >
>         >      +-----+-----+
>         >      | ECN FIELD |
>         >      +-----+-----+
>         >         0     0        Not-ECT
>         >         0     1        ECT(1) - L4S/SCE Capable AND No
>         Congestion
>         >         1     0        ECT(0) - Some Congestion OR RFC 3168
>         ECN Capable
>         >         1     1        CE
>
>         Yes, that's my understanding.  I think the whole proposal can
>         reasonably be
>         summarized as "SCE with the bits flipped".
>
>         > Jake, you said that the three issues discussed above --
>         tunnels, AccECN, and multiple AQMs in the path are "a few of
>         the known tradeoffs." What are the others?
>
>         These are all the ones I know of yet, but I think Bob and Koen
>         might have
>         some others they already know.  I'm not sure I got the whole
>         story yet.
>
>         Thanks for your comments.
>
>         Best regards,
>         Jake
>
>
>
> -- 
> ________________________________________________________________
> Bob Briscoehttp://bobbriscoe.net/

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/