Re: [tcpm] [EXTERNAL] Re: Seeking WG opinions on ACKing ACKs with good cause

[Bob Briscoe <[e-mail]>]

Neal,

On 11/07/2021 19:28, Neal Cardwell wrote:
>
>
> On Sun, Jul 11, 2021 at 11:55 AM Christian Huitema 
> <[e-mail] <mailto:[e-mail]>> wrote:
>
>
>
>     > On Jul 11, 2021, at 4:42 AM, Bob Briscoe <[e-mail]
>     <mailto:[e-mail]>> wrote:
>
> ...
>
>     > The implementation will have its own ACK ratio - that's out of
>     scope of AccECN, except to set this max of 6 CEs, which is to
>     mitigate wrap of the 3-bit counter of CE-marks (which in the worst
>     case of 100% marking in the data direction could then induce 1 ACK
>     per 6 packets). This shouldn't limit forward performance, because
>     it only increases the reverse ACK rate if there is heavy
>     congestion in the forward path, when the Data Sender should be
>     reducing the forward rate anyway.
>
>     1 ACK per 6 packets would cause performance issues on high speed
>     links. Common setup is "4 to 8 ACK per RTT", which means intervals
>     much larger than 6 packets.
>
>
> I share this concern about the AccECN requirement of one ACK per 6 
> CE-marked data segments potentially causing performance issues with 
> high-speed links. Today, with most high-speed last-hop link 
> technologies (wifi, cellular, Ethernet) TCP receivers often receive 
> (from lower layers of the hardware and software networking stack) 
> large aggregates  (often up to 64KBytes or around 44 packets), and 
> generate a single ACK for that aggregate. AFAICT changing this 
> scenario to require 1 packet per 6 CE-marked data segments means that 
> during CE-marked scenarios receiving such an aggregate would require 
> ceil(44/6) = 8 ACKs, increasing the number of ACKs by up to 8x.
>
> I have two main concerns in that scenario:
>
> (1) CPU load: This seems like it would impose much higher CPU load on 
> the data receiver, at just the moment when the receiver may already be 
> under stress due to receiving data near the maximum rate of the link.
>
> (2) Congestion and ACK loss: This requirement for generating up to 8 
> ACKs per aggregate seems like it is likely to produce a tight burst of 
> 8 ACKs, which is going to increase congestion and increase the odds of 
> losing at least one ACK, which is going to cause accuracy problems for 
> "Accurate" ECN. :-)

[BB] I hadn't intended this wording to apply in response to a large 
burst. I'd want to alter the wording to allow just one ACK in this case.

>
> I understand that this proposed "ACK per 6 CE-marked data segments" 
> rule is necessary to avoid issues with the 3-bit ACE field wrapping. 
> So IMHO this is one of the good arguments against including the ACE 
> field in the AccECN design.

[BB] I think the subtext here is a preference for the DCTCP style of 
feedback vs ACE (when there is no AccECN TCP Option). I don't think this 
particular issue is any different between the two.

With DCTCP feedback, when a large data burst like this arrives, if there 
are transitions to and from CE marking within the burst, does Linux 
DCTCP generate an ACK for each transition, and for every n repetitions 
of CE not ECT, like the RFC says it should? Or does the implementation 
just pop out one ACK at the end?

Ilpo & I have been doing experiments with high levels of ACK coalescing 
causing the ACE field seen by the Data Sender to wrap multiple times 
under high congestion. For example, 1 ACK per ~4ms is one common scheme, 
which would result in about 1 ACK for 34 data packets at just 100Mb/s.

>
> The other main concerns I have with the ACE field are:
>
> o Complexity

[BB] I understand there could be a tradeoff between complexity and 
accuracy. But I think the complexity of ACE and DCTCP feedback are 
pretty much the same but, with even low levels of ACK coalescing, the 
accuracy of ACE is superior. Nonetheless, I understand that deploying 
something different to DCTCP when you've already got DCTCP could involve 
deployment complexity.

>
> o Redundancy with respect to the AccECN counter options

[BB] The redundancy is intended. For cases where the TCP Option doesn't 
traverse the path, or where TCP option space is limited.

>
> o Potential problems with middleboxes

[BB] Well, we've tested the 3 header bits over millions of paths without 
problems. But yes, there are billions more to test.

>
> o Known problems with NICs and drivers (based on Ilpo's nice talk, 
> "Accurate ECN Linux Implementation: Experiences and Challenges", at 
> the April 2020 TCPM interim meeting)

[BB] I understood that talk as concluding there weren't really problems. 
What specific problem do you have in mind?

> I realize that if AccECN does not have the ACE field feature, then 
> AccECN and TCP L4S will not be usable on paths with middleboxes that 
> strip the AccECN counter options. But IMHO living without the ACE 
> field is preferable. IMHO it's acceptable to say that L4S can only be 
> used with (a) QUIC, or (b) TCP connections where no middleboxes are 
> stripping AccECN options.

[BB] When you say 'living without the ACE field', alongside the AccECN 
TCP Option, would you leave classic ECN feedback? Or put DCTCP feedback 
in its place?

I don't think there's any case for using classic ECN feedback within 
AccECN. I see the competition as between ACE and DCTCP feedback, each 
optionally with the AccECN TCP Option.

If DCTCP feedback were a stop-gap until we could get good traversal of 
the TCP Option, it might have some merit, but I think DCTCP isn't going 
to work well with the level of ACK coalescing in the Internet. However, 
this is very difficult to judge unless we do large scale A-B experiments 
on the real Internet (and potentially within DCs). I'll think further 
about enabling some way to do that, possibly within AccECN's negotiation 
framework. But today, I have to prioritize for the draft deadline.

Cheers



Bob


>
> cheers,
> neal
>
>
>
>

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/