Re: [tcpm] Comments on draft-ietf-tcpm-accurate-ecn

["Scheffenegger, Richard" <[e-mail]>]

Yuchung, Bob,

I really like this discussion!

Two points:


Am 14.07.2018 um 02:52 schrieb Bob Briscoe:
> Yuchung,
> 
> On 13/07/18 21:45, Yuchung Cheng wrote:
>> On Fri, Jul 13, 2018 at 12:42 PM, Mirja Kühlewind
>> <[e-mail]>  wrote:
>>> Hi Yuchung,
>>>
>>>> Am 13.07.2018 um 15:30 schrieb Yuchung Cheng<[e-mail]>:
>>>>
>>>> On Fri, Jul 13, 2018 at 11:53 AM, Mirja Kühlewind
>>>> <[e-mail]>  wrote:
>>>>> Hi Yucheng,
>>>>>
>>>>> please see below.
>>>>>
>>>>>> Am 13.07.2018 um 14:38 schrieb Yuchung Cheng<[e-mail]>:
>>>>>>
>>>>>> hi --
>>>>>>
>>>>>> I agree:
>>>>>> 1. delayed/streched ACK aren't going away (in fact will be more common)
>>>>>> 2. GRO isn't and should not be a show-stopper
>>>>>> 3. SYN option is running tight
>>>>>> 4. HW opt comes after SW
>>>>>>
>>>>>> I worry:
>>>>>> 1. GRO is a unavoidable issue in deployment (let's not produce an
>>>>>> undeployable RFC). the ACE counter won't work as GRO can pack up to
>>>>>> 64KB/MTU =~ 45 pkts under heavy congestion.
> [BB] I'm really not expert on offload, but I thought GRO collects (or 
> can be made to collect) a set of fields from the headers it strips off?
> 
> If I'm wrong (quite likely), bear in mind the following:
> * you only need the ACE counter (main header) when the AccECN Option is 
> being stripped by a middlebox
> * on those connections that need ACE (cos of middlebox meddling) 
> couldn't GRO be limited to 8 packets, at least while experimenting with 
> AccECN to see if it is useful and gather data?

[RS @ Group] To be pedantic, GRO can coalesce packets, until the ACE 
counter has increased by 7 counts (one less than the previously 
delivered "superpacket"). Under heavy CE marking, this may be as few as 
7 or 8 packets, correct. However, this is when the network expiriences 
congestion. Thus "slowing things down" by reducing the processing speed 
on the client is not an acute problem IMHO - as a slightly delayed 
ACKing would implicitily also reduce the sending speed somewhat.

With many flows, higher RTTs etc, I would expect GRO to be able to 
coalesce many more than just 7 packets.


[...]



 >>>> 3. Leave ACE-count and ACE option optional (i.e. MAY)
 >>>
 >>> I don’t understand this. If both is optional, you don’t have any 
feedback. Or what do you mean by „leave ACE-count optional“?
 >> use-case: We can negotiate DCTCP-style ECN for the internet.
 >>
 >> Then interested parties can progressively experiment on more accurate
 >> "options" (!= TCP-option)


>>>>>> 3. Leave ACE-count and ACE option optional (i.e. MAY)
>>>>> I don’t understand this. If both is optional, you don’t have any feedback. Or what do you mean by „leave ACE-count optional“?
>>>> use-case: We can negotiate DCTCP-style ECN for the internet.
>>>>
>>>> Then interested parties can progressively experiment on more accurate
>>>> "options" (!= TCP-option)
 >>>
 >>> As Appendix A of RFC7560 says I don’t think it is a safe option for 
 >>> the Internet where packet loss more likely then in a full
 >>> ECN-enabled data center.
 >>>
 >>> Again I disagree RFC7560 Appendix A is a big problem based on my
 >>> experience with at times loss-heavy ECN-enabled data-center (Google
 >>> data-center runs very hot and uses a DCTCP-variant).
 >>>
 >>> We can quabble forever w/o data. That's why I asked for some
 >>> (non-simulation) data.

[RS @ Yuchung] I'm afraid I'm with Mirja's comment (other fork of this 
discussing added here), I don't quite follow what you proposing. An 
AccECN-style negotiation, which is used in conjunction with ECN++ (and 
possibly L4S style ECT-1 marking of all packets instead of ECT-0?). But 
how do you envision the feedback scheme? Like the DCTCP-like state 
machine to send change-triggered ACKs, and stretches of ECE and non-ECE 
marked TCP header bits? (Which does not cater well for ACK thinning, ACK 
loss)


However, I think I see where you are coming from. Let me try to 
summarize what I assume here:

With DCTCP-style feedback in a datacenter, and some level of ACK losses, 
the sender-side estimation of the CE levels with with equal probability 
over- and under-estimated the marking levels during one specific RTT. If 
the sender underestimated congestion, the network will provide a higher 
marking level during the next RTT, and thereby correct the false 
estimate. On overestimation, the flow would only have reduced its 
sending rate more than necessary, and regular CA across all flows of 
that bottleneck link will readily make use of that free bandwidth. All 
of that happening in datacenter-typical RTTs of a few dozend nanoseconds 
to a few microseconds... So any deviation will only persist for a very 
short period of time, while most flows have very similar RTTs.

Now, the problem space AccECN tries to address includes the public 
internet, with RTTs over a bottleneck, that vary many orders of 
magnitude potentially. Thus the goal should be to avoid the over- and 
more so, the under-estimation of congestion as much as possible already 
within the first RTT.

Or summarized differently, in a datacenter, small errors in congestion 
estimation have a predictable, very short life-time until they will be 
corrected. On the public internet, it is a better approach to try and 
avoid an estimator error on congestion as good as possible, as these 
errors would result in more unfairness among flows.

I hope that is a fair summary, but please expand on how/what your 
feedback scheme would encompass.

Best regards,
   Richard