Re: [tcpm] AccECN and DSACK

[Markku Kojo <[e-mail]>]

Hi Richard, all

On Fri, 17 Nov 2023, [e-mail] wrote:

> Hi Markku,
>
>
> Am 17.11.2023 um 18:26 schrieb Markku Kojo:
>> Bob, Richard, Yoshi, all,
>> 
>> On Wed, 15 Nov 2023, Bob Briscoe wrote:
>> 
>>> Richard,
>>> 
>>> Thx for the clarification. So the draft will say nothing about
>>> timestamps.
>>> 
>>> To confirm, in the editor's copy of ECN++, I'll add the Δ(ACE)≥3
>>> requirement to the additional DupACK check (In
>>> deployments where SACK blocks might sometimes be removed, and ideally
>>> only if no SACK blocks have been seen at all, despite SACK having been
>>> negotiated).
>> 
>> I'm afraid that the suggested rule (ACE counter is 3 or more) is
>> extreamily unreliable. It should be quite easy to see that there are a
>> multitude of scenarios where the data pkt that triggers a genuine DupACK
>> can itself be CE-marked and preceded by two (or more) CE-marked pure
>> ACKs, in which case the ACE counter in the DupACK is three (or higher),
>> resulting in genuine DupACK being ignored. Sounds like a very bad idea
>> to me.
>
> Sorry. You completely lost me.
> 
> How is an ACK of new data a dupack?

[MK] And you lost me totally, sorry. ;)
I don't know what made you think that the data pkt could carry new, 
in-order data? I said "data pkt that triggers a genuine DupACK". How can 
this data pkt be other than out-of-order or duplicate data pkt if it 
triggers a DupACK? It is definitely not a new data pkt arriving in order 
and triggering a new ACK. And, yes, in case of out-of-order data pkt it 
carries new data but that data won't get acknowledged but triggers a 
DupACK.
I thought this should not require a specific scenario to understand it, 
sorry. I should have been more accurate and say that the data pkt 
is either duplicate or out-of order pkt as well as the additional details 
I now add below. Whichever specific scenario you select where a pkt 
receiver receives the following sequence of pkts that I described, will 
result in ACE delta being 3. Or maybe I am missing something?

Assume no unacknowledged data at the pkt receiver and all earlier CE marks 
have been acknowledged:
1. A pure ACK marked CE arrives (CE pkt counter incremented by 1, does not 
trigger an ACK).
2. A pure ACK marked CE arrives (CE pkt counter incremented by 1, does 
not trigger an ACK).
3. A duplicate or out-of-order data pkt arrives (CE pkt counter 
incremented by 1, triggers genuine DupACK with ACE delta = 3).

Please find also a specific scenario that breaks TLP logic (see below).

> Or, since I have to imagine what scenario you are thinking about (please
> - provide a good depiction of the segment sequences exchanged), how
> would a discontinous data packet, with the same ACK seq number (to be a
> dup ack) NOT carry a SACK block?

[MK] I think I have said already earlier that with F-RTO or RACK-TLP we 
are not considering arrival of out-of-order data pkts that trigger three 
DupACKs or three AoAs that may be incorrectly interpreted as three 
DupACKs.
When a duplicate data segment for the highest seg number already 
received by the data receiver arrives it triggers DupACK without a SACK 
option as per RFC 2018. Or, if SACK blocks were removed like Bob 
mentioned above, then it may be an out-of-order data pkt.

> Furthermore, a singular ACK for a new data packet is commonly not
> refered to as DupACK, but a regular in-sequence ACK.
>
> Or are we suddenly bringing in subsequent data packets with or without
> loss/reordering?
>
> In any case, as described it is *extremely* easy to differentiate from a
> genuine ACE-ACK (new ACK number, likely more recent timestamp, while not
> containing SACK).
>
>
>> Maybe the best way is to require the other end to implement and apply
>> DSACK (MUST), and use DupACKs arriving without SACK and ACE<3 to turn
>> off ECN-capable pure ACKs for the rest of the connection.
>
> Again, I can not follow. How would DSACK help in the above situation?
>
> If the new CE-marked data segment is beyond the right edge, it either
> advances ACK, or carries a SACK (but the scenario didn't contain any
> losses...). If it is a TLP packet overlapping with the first
> transmission, by virtue of (still) being at the very right edge, this
> may be the 1st DupACK (not triggering any DupACK based mechanism), but
> even if we now conjecture 3 TLPs for some arcane reason, and no DSACK
> (despite all RACK implementations I know do support DSACK by virtue to
> improving the RACK mechansims), since these would still be at the
> rightmost edge, no erraneous (additional) retransmission (with CC
> response) would be triggered. (Please explain first, how three
> consecutive TLP transmissions would occur, since these are only
> happening once).

[MK] Sorry, I don't follow. We are discussing RACK-TLP (TLP part for 
it) and F-RTO that have been designed to distinguish whether arriving 
ACKs are due to real loss or duplicate data segment(s). A single 
extra DupACK (or ignored DupACK) may break the logic.

>
>> Without the data sender (AoA receiver) knowing that the other end
>> employs SACK, I cannot find a reliable enough way to distinguish genuine
>> DupACK and AoA.
>
> Which is why the guidance is already to only allow the use of
> ECT-enabled ACKs, when the session does have SACK enabled.

[MK] Yes, and I have been trying to explain that enabling SACK is not 
enough if the other end (data receiver) does not implement DSACK.

>> Currently there is no RFC I am aware of that says
>> negotiating SACK would negotiate DSACK as well or that if one implements
>> SACK one SHOULD/MUST implement DSACK as well. As long as this is not
>> said in any RFC, IMO it would be inappropriate to rely on DSACK being
>> employed as specified in RFC 2883. That is one important reason (not the
>> only even though I earlier only mentioned this) why RFC 5682 and RFC
>> 8985 do not rely on DSACK alone but treat the non-SACK DupACKS the same
>> as DSACK DupACKs to ensure correct operation in the current Internet
>> (works also with F-RTO even if SACk-blocks are removed).
>
> Please provide a fully worked example showing how excluding an ACK which
> has delta-ACE >= 3, no SACK blocks, which is not advancing snd_una from
> subsequent DupAck processing breaks existing mechanisms. Ideally in a
> situation where DSACK would have provided a different outcome.

[MK] Please see the scenarios (Fig 5 a) that I have added and are 
available at:

  https://www.cs.helsinki.fi/u/kojo/IETF/AccECN2023.pdf

For your convenience, the first two added scenarios (Fig 4 a) and b)) are 
without any AoA interference. That is, they are for the two normal cases 
(duplicate data segment due to spurious TLP and real loss) that TLP logic 
has been designed to distinguish from each other even when DSACK is not 
implemented by the data receiver. See RFC 8985, Sec 7.4.2 that I have 
pointed to already earlier (maybe more than once). Pls, see also the 
description in the RFC (case 2). In both cases for Fig 4 the ACKs are 
delayed. The only difference is that the end of flight data segment (data 
200) is not lost in Fig 4 a) while in Fig 4 b) it is lost.

In the Figures 5 a) and 5 b) I have modified the data traffic from B to A 
such that an appropriate amount of ECN-capable ACKs may get triggered from 
A to B and (some of them) marked CE. In Fig 5 a) there may be any number 
data pkts (>=2) from B to A arriving at A after the original transmission 
of data 200 but before the PTO-triggred retrinsmission of data 200 as long 
as they trigger pure ACKs of which two become CE_marked. As shown in the 
Fig 5 a), if B does not implement DSACK it will result in the AoA (DupACK 
201 with ACE delta = 3) becoming ignored and TLP logic makes incorrect 
desicion as it does not detect the spurious PTO. So, ACE rule seems not 
reliable to me but maybe I am missing something? If B implements DSACK, 
it would include DSACK block for data 200 in the DupACK 201 and TLP logic 
would correctly detect spurious PTO (case 1) unless it is removed in the 
middle.

Fig 5 b) shows a scenario where data receiver does not implement DSACK, 
resulting in A not detecting loss and inappropriately not reacting 
to congestion which is one of the worst violations against the congestion 
control principles. In my reading of the various replies, it seems to me 
that I and Bob agree that DSACK MUST be implemented at a data receiver 
and AoA MUST not include SACK option (requirements in the AccECN draft) 
but Richard seems to disagree(?). So, this is a scenario that Richerd 
requested earlier in this thread.
Even though the ACE rule may work in the scenario in Fig 5 b), I ignore 
it here because it seems unusable as per shown in Fig 5 a). Similarly to 
Fig a), in Fig 5 b) there may be any number of data segments from B to 
A as long as they arrive at B before the TLP probe (rexmit of data 200) 
and they trigger at least three pure ACKS out of which three pure ACKs 
become CE marked. As shown in the figure, the AoA fake DupACK will 
conceal the loss of data 200. If A knows that DSACK is implemented at 
B and B never includes a SACK option in AoAs, the TLP logic at A can be 
modified to ignore DupACKs without a SACK option. With such modification 
AoA (DupACK 201) is ignored and does not end the TLP episode, allowing 
ACK 202 to reveal the loss and A to correctly react to loss.


>> If SACK blocks are removed in the middle, it does not help ECN++ sender
>> even if the other end is required to apply DSACK but additional
>> safeguards are required.
>
> I mentioned this, as it is a (rather uncommon) pathological issue which
> may happen.

[MK] I am not aware of any data showing how common it is today but it is 
known to happen. Do you have any pointers to Internet-wide data?
IMO, a proper stds track spec should take such misbehaviour into 
account.

[MK] And, it is not only that SACK blocks may get removed in the middle. 
There are (or at least have been) TCP implementations that negotiate SACK 
but do not include (any) SACK blocks. This is another reason, why it is 
important require in the AccECN draft that an AccECN implementation 
MUST implement both RFC 2018 and RFC 2883 in full, if it sends Acks of 
Acks.

> With the suggested discrimination of ACE-ACKs, that ACK (and only this
> particular one) would be excluded from subsequent DupACK processing.
> Since genuine DupACKs would not match the ACE-ACK checks, these would
> trigger all other mechanisms - albeit at most 1 ACK "late".

[MK] Pls, Ack if you agree that the ACE rule that you suggested is 
unreliable and cannot be applied (as shown in Fig 5 a).

>
>> I'm not sure if No SACK and ACE<3 rule is good
>> enough, because no full analysis nor experimental evidence exists to
>> judge on.
>
> Which is why I asked repeatedly for some good examples and scenarios to
> fully analyze.

[MK] My suggested rule "if No SACK and ACE delta <3, the turn off 
ECN-capable pure Acks" seems not workable anymore after Bob's 
clarification for the "Increment-Triggered ACKs" rule. But, if there is 
no SACK block in DupACK and ACE delta=0, doesn't it indicate that SACK 
blocks are removed (given that MUST implement RFC 2018 and RFC 2883 and 
reguired in AccECN draft)? The common security practice that IETF has 
followed earlier is that if misbehaviour that brakes the designed logic 
for some feature is detected, then that feature should be disabled.
Or are you suggesting that this is unreasonable practice?

>> Particularly, I am worried about short flows that are very
>> common and with such flows genuine DupACKs may get interpreted
>> incorrectly.
>
> Well, what is your definition of a short flow? Would a flow with a
> congestion window of at least 18 segments, for the minimal theoretical
> possible number of delayed, ECT-enabled ACKs, all of which would get
> CE-marked, count as a short flow? Remember that this would require in
> the order of 30 segments (or 45 kB) at the very least.

[MK] Sorry, I donät follow. The number of ECT-enabled ACKs is not coverned 
by the flow that (potentially) suffers from the AoAs but the flow of data 
from the other direction. So, you may have a data flow from A to B of any 
lenght and and how many AoAs it may encounter depends (almost) solely on 
the data flow from B to A. Almost solelely as AoA ping-pong effect may 
be affected by the flow from A to B itself.
And, your 18 segments minimum rule holds only for the case where three 
DupACKs are needed to trigger false fast rexmit. Actually, the minimum 
cwnd to trigger enough ECT-enabled ACKs for three DupACKs is not 18 
because if there is a pkt loss, it is enough to have 9 out-of-order data 
segments that will trigger 9 ECN-capable DupACKs.
Moreover, breaking the RACK-TLP or F-RTO logic reguires just one fake AoA 
DupACK.

> But at this bare minimum flow size, ending after this, none of any
> possible misclassification or erraneous processing would matter.

[MK] Right, it is not that serious if the flow ends. I was thinking a 
common application limited behaviour that consists of short data bursts 
that are repeated (which maybe should not be called a short flow). If the 
end of flight loss in each such burst is repeatedly misinterpreted (loss 
concealed), it starts to be quite bad behaviour.

> So, the short flow would need to be larger than this. And even (much)
> larger to account for the much lower probability of CE-marked ACKs to
> happen.
>
>> This means that a rule should rather be used in different
>> way than Richard suggested; the stack should immediately start treating
>> DupACKs without SACK as genuine DupACKs for the rest of the connection
>> in addition to turning ECN-capable pure ACKs off.
>
> Whats the definition of your DupACK here? I'm confused. Rather than

[MK] My apologies for missing (not repeating) a crucial point. After 
detecting misbehavior (SACK block are being removed), the stack should 
*disable ECN-capable pure ACKs* and start treating DupACKs without SACK 
as genuine DupACKs in order to allow correct behaviour of all algos 
that depend on the crucial RFC 5681 rule that an ACK MUST NOT be 
generated more than once for any data segment.

Hope this clarifies,

/Markku

  > determining which ACK may NOT be a DupACK to stop confusing earlier
> mechanisms, which don't have an understanding of AccECN and ECT-enabled
> ACKs, suddenly every DupAck (presumably masking out the presence/change
> of the ACE field, as well as any TCP option - including SACK in
> particular) should be accepted as DupACK under all circumstances?
>
> This confuses me even more...
>
>
> Sorry,
>   Richard
>