Re: [tcpm] I-D Action: draft-ietf-tcpm-accurate-ecn-13.txt - ACK Thining

[Bob Briscoe <[e-mail]>]

Gorry,

Thanks as always for checking all this... inline tagged [BB]

On 06/11/2020 14:01, Gorry Fairhurst wrote:
> I see this rev introduced a section on ACK Filtering, which I have 
> specific comments on. We need to consider the cost of sending ACKs - 
> especially anything that results in them being large - however, this 
> needs to be done by choosing a default at the receiver - not as an 
> adaptation, the adaption if necessary needs to be introduce more ACKs 
> when this is beneficial, as in some cases for DAAS, not in trying to 
> figure something from delayed or "missing" ACKs.
>
> 1) So let me say first that I think the suggestion for updating the 
> receiver to thin, and every 2-6 packets seems reasonable.
>
>     “MUST immediately send an ACK once 'n' CE marks have arrived since
>       the previous ACK, where 'n' SHOULD be 2 and MUST be no greater
>       than 6.”
>
> I do agree that endpoints can do much more to reduce the return path 
> capacity that they consume - and that the endpoints have the 
> opportunity to encode the feedback information more efficiently than 
> can be done by a device (or queue) in the network. From our 
> experiments, n=10 would also often be likely OK.

NB: n is the no. of *CE marks*, which is only the ACK thinning ratio in 
the worst case if there's 100% marking.
IOW, with n=6 the receiver could ACK less than 1 in 10 data packets 
whenever marking probability < 60%.

Altho the receiver's ACK ratio will have to fall to 6 to 1 for the 
duration of a 100% marking sequence, whenever some data packets are not 
marked, the receiver aACK ratio can rise. This *could* allow any 
build-up of ACKs queued on the reverse path to relax, which could 
improve the chances of not overflowing into ACK loss.

It's hard to predict when 100% marking in the forward direction will 
coincide with reverse path congestion, and I'm not saying it won't.
Just pointing out that n does not equal the ACK thinning ratio, it's the 
floor of it.

n=6 is a consequence of only having a 3-bit field to count CE marks. We 
could push the envelope and make the max n=7. But not n=8, otherwise 
wrap after 8 marks would be indistinguishable from zero marks.


>
> 2. Althouigh in some cases a receiver can predict a good "n", I do not 
> believe a receiver can typically find out how to choose “n” by 
> observing the ACKs on a connection.
> 2. I do not believe a receiver can find out how top to choose “n” by 
> observing the ACKs received on a connection.

[BB] I think you intended to delete one of these sentences. Whatever, 
both of them seem to contain a typo - otherwise they are circular? 
Surely the receiver is sending the ACKs, so either sentence as it stands 
ends with "...I do not believe a receiver can typically decide how often 
to send an ACK by observing how often it is sending ACKs itself." ???

>
> Sure, if the path is modelled as a simple bit-congestive path that has 
> a queue, then you could monitoring timing/loss of packets and 
> determine a rate that tries not to fill a queue. I would not know how 
> common such a bottleneck link is.
>
> I do know that many bottleneck links are more complex and needs to 
> consider the scheduling transmission of small ACK packets on the 
> return path performance. In many radio technologies, the overhead from 
> transmission of ACKs is proportionally much higher than for data 
> segments -  These layer 2s are constrained in terms of packets (e.g. 
> where lower layer framing and transmission scheduling is an important 
> consideration to the efficiency/cost of using the service). This has 
> often been “fixed” using ACK Thinning or PEPs. Protocols that send 
> fewer ACKs/data packet are not impacted.

[BB] I agree that the receiver alone cannot predict a good 'n'.
AccECN is a not the complete solution, but we've grasped the opportunity 
to take a step in the right direction (within the chartered scope of 
AccECN). AccECN at least allows the data sender to measure the extent of 
congestion marking on the arriving ACK stream (if there's ECN support at 
the bottleneck). Whereas before AccECN, a random ECN mark on one pure 
ACK in a round couldn't be distinguished from 100% of ACKs marked.

>
> Scheduling (at the IP layer,can also often result in variations in 
> timing - something that is hard to predict and I would suggests make 
> automated selection of “n” very hard, and the need to thin is often 
> not directly visible to a user of the path. Traffic multiplexed with 
> protocols that use different ACK policies (perhaps QUIC?) also make 
> this sort of detection less reliable.

[BB] Being hard shouldn't prevent us trying.
The remaining piece would be to advance a TCP ACK congestion control 
protocol on the experimental track (RFC5690 
<https://tools.ietf.org/html/rfc5690> is informational).


>
> 3) A queue might be competing for scheduling opportunities against 
> other IP flows (e.g., WFQ), or competing for transmission 
> opportunities with a shared resource (common in radio links).
>
> ACKs can and do take transmission opportunities away from the resource 
> pools that would be used by other services that share the same 
> resource pool. This can be reduced by some form of per-flow scheduling 
> with deeper per-flow queues - but that in itself can induce unwanted 
> coupling between flows - limiting the peak rate of one flow, to 
> achieve a higher ACK ratio for another flow.
>
> Consider a QUIC flow sharing a return path with a TCP flow, the higher 
> rate of larger QUIC ACKs reduces the transmission opportunities for 
> TCP ACKs, which in turn suppresses the TCP flow, limiting the overall 
> system performance. If the ACKs use the same transmission resource, 
> they could take directly away from transmission opportunities in the 
> forward direction, but even if they do not - the rate of ACKs retuned 
> impacts the sending rate end to end.

[BB] This says two things to me:
* if TCP is going to introduce control over the ACK ratio, QUIC and 
other transports ought to as well {Note 1}.
* and all ACK CC algorithms ought to be designed to ensure some degree 
of equitable capacity sharing {Note 2}, including between the ACK stream 
of flow A and the data stream of flow B transferring in the opposite 
direction.

{Note 1}: I don't think this needs any formal "all or nothing" rule 
where the transport ADs co-ordinate multiple WGs. Enough of the people 
involved are the same for the right thing to happen (I think). But some 
AD oversight wouldn't do any harm.
{Note 2}: There are numerous challenges to face, but I think it's 
paramount to first establish which node is considered responsible for 
solving the problem (the bottleneck or one of the transport endpoints). 
This doesn't imply that the IETF will ever ensure that other nodes don't 
'interfere'. For instance, if the sender is deemed responsible for 
determining the level of ACK filtering, it will still have to make 
allowance for bottlenecks attempting to ACK thin as well.



>
> 4) The new section adds some cross-layer requirements on sub-IP 
> layers, which seem to imply more complexity. I’m not comfortable with 
> such a proposal.
>
> It seems that the text suggests not implement thinning and that these 
> systems implement logic to detect the presence of the option by 
> observing SYNs and then configure the link to do something special. 

This was the proposal that I was most interested to hear your views on. 
We can remove it if the WG doesn't want it, but first I want to make 
sure the proposal is fully understood:
* The aim is for the RFC to tell bottleneck implementers (via this text) 
that *only in cases where AccECN has been negotiated* the preferred way 
to improve flow performance is for the link to signal ECN to the 
transport, not to drop the ACKs itself.
* If the link doesn't support ECN, it still does the ACK filtering itself
* Links aren't going to delegate their control unless transports 
actually act on the ECN signals that links emit - and improve 
performance better than links have been. So, it's important that 
transports take on these responsibilities - the data sender takes 
responsibility for determining the ACK rate, asking its correspondent 
receiver does actually thin its own ACKs.

Here's who signals what, and who does what

                link-.
                      \
                       '-[ECN CE]-.
                                   \
                                    '-> receiver
                                     .-
                                    /
             .-[(Acc)ECN feedback]-'
            /
  sender <-'
         -.
           \
            '---[ACK CC ratio]---.
                                  \
                                   '-> receiver(treats as polite 
request, not mandatory)


> I don’t think think this is the correct approach - the result will be 
> throttling of the connection speed. Is it necessary that the sub-IP 
> layer does not implement thinning for connections that use AccECN 
> feedback?

Why do you say the connection speed would be throttled?
Did the text not explain the proposal well?
Or did you understand the proposal but disagree with it?


>
> When the text says: “SHOULD preserve the timing of each ACK”
> - does that imply that nodes need to discard rather than delay packets 
> when link scheduling or transmission scheduling is used. This seems a 
> very odd requirement, it might be better too simply strip-out the 
> AccECN option or discard all packets that set this.

Sorry, that may be clumsy wording. It just means "the link SHOULD NOT 
filter ACKs" (if it supports ECN marking).

>
> I can’t see how queuing the ACKs, or deleting ACKs rather than queuing 
> them would improve the performance of AccECN TCP connections.

May be the piece where the link's preferred role is to signal to the 
transport to slow down the ACK rate using ECN wasn't clear.


Bob

>
> Gorry
>

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/