Re: [tcpm] AccECN and DSACK

[Bob Briscoe <[e-mail]>]

Markku, Richard,

I'll try to summarize your long conversation below (and the subsequent 
emails from Richard responding to specific paragraphs) for the list and 
the chairs...

1) None of this conversation requests any further changes to the AccECN 
draft.

2) I believe the first two thirds of the conversation below (down to 'If 
SACK blocks are removed') can be ignored. Markku thought Richard "seemed 
to disagree" that the AccECN spec should mandate that "DSACK MUST be 
implemented if SACK is implemented". But I understood that Richard had 
already agreed to that change (he is a co-author so I ran it past him).

3) The final third and Richard's subsequent emails concern the case 
where SACK blocks are being removed in the middle (or not set by a host 
that has agreed to set them):

  * Markku believes that, if an ECN++ sender detects that SACK blocks
    are being removed, it should disable ECN-capable pure ACKs.
  * Richard believes that, if an ECN++ sender detects that SACK blocks
    are being removed, further tests can be introduced (Δ(ACE)≥3 and/or
    timestamps) that are not necessarily 100% perfect at distinguishing
    AoAs from DupACKs, but the cases where they don't work are highly
    unlikely, and have 'extremely miniscule' impact even if they do occur.

Given the ECN++ draft is experimental, it seems reasonable to me that it 
could allow implementers to try either, and advice on which approach is 
better would be one potential outcome of the experiment. This is based 
on my single response to Markku below, tagged [BB].


*I hope all the above is agreeable to both of you.*
Assuming it is, I'm about to change the subject line of the thread for 
further discussion of exactly how to write this into the ECN++ spec.



On 20/11/2023 20:33, Markku Kojo wrote:
> Hi Richard, all
>
> On Fri, 17 Nov 2023, [e-mail] wrote:
>
>> Hi Markku,
>>
>>
>> Am 17.11.2023 um 18:26 schrieb Markku Kojo:
>>> Bob, Richard, Yoshi, all,
>>>
>>> On Wed, 15 Nov 2023, Bob Briscoe wrote:
>>>
>>>> Richard,
>>>>
>>>> Thx for the clarification. So the draft will say nothing about
>>>> timestamps.
>>>>
>>>> To confirm, in the editor's copy of ECN++, I'll add the Δ(ACE)≥3
>>>> requirement to the additional DupACK check (In
>>>> deployments where SACK blocks might sometimes be removed, and ideally
>>>> only if no SACK blocks have been seen at all, despite SACK having been
>>>> negotiated).
>>>
>>> I'm afraid that the suggested rule (ACE counter is 3 or more) is
>>> extreamily unreliable. It should be quite easy to see that there are a
>>> multitude of scenarios where the data pkt that triggers a genuine 
>>> DupACK
>>> can itself be CE-marked and preceded by two (or more) CE-marked pure
>>> ACKs, in which case the ACE counter in the DupACK is three (or higher),
>>> resulting in genuine DupACK being ignored. Sounds like a very bad idea
>>> to me.
>>
>> Sorry. You completely lost me.
>>
>> How is an ACK of new data a dupack?
>
> [MK] And you lost me totally, sorry. ;)
> I don't know what made you think that the data pkt could carry new, 
> in-order data? I said "data pkt that triggers a genuine DupACK". How 
> can this data pkt be other than out-of-order or duplicate data pkt if 
> it triggers a DupACK? It is definitely not a new data pkt arriving in 
> order and triggering a new ACK. And, yes, in case of out-of-order data 
> pkt it carries new data but that data won't get acknowledged but 
> triggers a DupACK.
> I thought this should not require a specific scenario to understand 
> it, sorry. I should have been more accurate and say that the data pkt 
> is either duplicate or out-of order pkt as well as the additional 
> details I now add below. Whichever specific scenario you select where 
> a pkt receiver receives the following sequence of pkts that I 
> described, will result in ACE delta being 3. Or maybe I am missing 
> something?
>
> Assume no unacknowledged data at the pkt receiver and all earlier CE 
> marks have been acknowledged:
> 1. A pure ACK marked CE arrives (CE pkt counter incremented by 1, does 
> not trigger an ACK).
> 2. A pure ACK marked CE arrives (CE pkt counter incremented by 1, does 
> not trigger an ACK).
> 3. A duplicate or out-of-order data pkt arrives (CE pkt counter 
> incremented by 1, triggers genuine DupACK with ACE delta = 3).
>
> Please find also a specific scenario that breaks TLP logic (see below).
>
>> Or, since I have to imagine what scenario you are thinking about (please
>> - provide a good depiction of the segment sequences exchanged), how
>> would a discontinous data packet, with the same ACK seq number (to be a
>> dup ack) NOT carry a SACK block?
>
> [MK] I think I have said already earlier that with F-RTO or RACK-TLP 
> we are not considering arrival of out-of-order data pkts that trigger 
> three DupACKs or three AoAs that may be incorrectly interpreted as 
> three DupACKs.
> When a duplicate data segment for the highest seg number already 
> received by the data receiver arrives it triggers DupACK without a 
> SACK option as per RFC 2018. Or, if SACK blocks were removed like Bob 
> mentioned above, then it may be an out-of-order data pkt.
>
>> Furthermore, a singular ACK for a new data packet is commonly not
>> refered to as DupACK, but a regular in-sequence ACK.
>>
>> Or are we suddenly bringing in subsequent data packets with or without
>> loss/reordering?
>>
>> In any case, as described it is *extremely* easy to differentiate from a
>> genuine ACE-ACK (new ACK number, likely more recent timestamp, while not
>> containing SACK).
>>
>>
>>> Maybe the best way is to require the other end to implement and apply
>>> DSACK (MUST), and use DupACKs arriving without SACK and ACE<3 to turn
>>> off ECN-capable pure ACKs for the rest of the connection.
>>
>> Again, I can not follow. How would DSACK help in the above situation?
>>
>> If the new CE-marked data segment is beyond the right edge, it either
>> advances ACK, or carries a SACK (but the scenario didn't contain any
>> losses...). If it is a TLP packet overlapping with the first
>> transmission, by virtue of (still) being at the very right edge, this
>> may be the 1st DupACK (not triggering any DupACK based mechanism), but
>> even if we now conjecture 3 TLPs for some arcane reason, and no DSACK
>> (despite all RACK implementations I know do support DSACK by virtue to
>> improving the RACK mechansims), since these would still be at the
>> rightmost edge, no erraneous (additional) retransmission (with CC
>> response) would be triggered. (Please explain first, how three
>> consecutive TLP transmissions would occur, since these are only
>> happening once).
>
> [MK] Sorry, I don't follow. We are discussing RACK-TLP (TLP part for 
> it) and F-RTO that have been designed to distinguish whether arriving 
> ACKs are due to real loss or duplicate data segment(s). A single extra 
> DupACK (or ignored DupACK) may break the logic.
>
>>
>>> Without the data sender (AoA receiver) knowing that the other end
>>> employs SACK, I cannot find a reliable enough way to distinguish 
>>> genuine
>>> DupACK and AoA.
>>
>> Which is why the guidance is already to only allow the use of
>> ECT-enabled ACKs, when the session does have SACK enabled.
>
> [MK] Yes, and I have been trying to explain that enabling SACK is not 
> enough if the other end (data receiver) does not implement DSACK.
>
>>> Currently there is no RFC I am aware of that says
>>> negotiating SACK would negotiate DSACK as well or that if one 
>>> implements
>>> SACK one SHOULD/MUST implement DSACK as well. As long as this is not
>>> said in any RFC, IMO it would be inappropriate to rely on DSACK being
>>> employed as specified in RFC 2883. That is one important reason (not 
>>> the
>>> only even though I earlier only mentioned this) why RFC 5682 and RFC
>>> 8985 do not rely on DSACK alone but treat the non-SACK DupACKS the same
>>> as DSACK DupACKs to ensure correct operation in the current Internet
>>> (works also with F-RTO even if SACk-blocks are removed).
>>
>> Please provide a fully worked example showing how excluding an ACK which
>> has delta-ACE >= 3, no SACK blocks, which is not advancing snd_una from
>> subsequent DupAck processing breaks existing mechanisms. Ideally in a
>> situation where DSACK would have provided a different outcome.
>
> [MK] Please see the scenarios (Fig 5 a) that I have added and are 
> available at:
>
> https://www.cs.helsinki.fi/u/kojo/IETF/AccECN2023.pdf
>
> For your convenience, the first two added scenarios (Fig 4 a) and b)) 
> are without any AoA interference. That is, they are for the two normal 
> cases (duplicate data segment due to spurious TLP and real loss) that 
> TLP logic has been designed to distinguish from each other even when 
> DSACK is not implemented by the data receiver. See RFC 8985, Sec 7.4.2 
> that I have pointed to already earlier (maybe more than once). Pls, 
> see also the description in the RFC (case 2). In both cases for Fig 4 
> the ACKs are delayed. The only difference is that the end of flight 
> data segment (data 200) is not lost in Fig 4 a) while in Fig 4 b) it 
> is lost.
>
> In the Figures 5 a) and 5 b) I have modified the data traffic from B 
> to A such that an appropriate amount of ECN-capable ACKs may get 
> triggered from A to B and (some of them) marked CE. In Fig 5 a) there 
> may be any number data pkts (>=2) from B to A arriving at A after the 
> original transmission of data 200 but before the PTO-triggred 
> retrinsmission of data 200 as long as they trigger pure ACKs of which 
> two become CE_marked. As shown in the Fig 5 a), if B does not 
> implement DSACK it will result in the AoA (DupACK 201 with ACE delta = 
> 3) becoming ignored and TLP logic makes incorrect desicion as it does 
> not detect the spurious PTO. So, ACE rule seems not reliable to me but 
> maybe I am missing something? If B implements DSACK, it would include 
> DSACK block for data 200 in the DupACK 201 and TLP logic would 
> correctly detect spurious PTO (case 1) unless it is removed in the 
> middle.
>
> Fig 5 b) shows a scenario where data receiver does not implement 
> DSACK, resulting in A not detecting loss and inappropriately not 
> reacting to congestion which is one of the worst violations against 
> the congestion control principles. In my reading of the various 
> replies, it seems to me that I and Bob agree that DSACK MUST be 
> implemented at a data receiver and AoA MUST not include SACK option 
> (requirements in the AccECN draft) but Richard seems to disagree(?). 
> So, this is a scenario that Richerd requested earlier in this thread.
> Even though the ACE rule may work in the scenario in Fig 5 b), I 
> ignore it here because it seems unusable as per shown in Fig 5 a). 
> Similarly to Fig a), in Fig 5 b) there may be any number of data 
> segments from B to A as long as they arrive at B before the TLP probe 
> (rexmit of data 200) and they trigger at least three pure ACKS out of 
> which three pure ACKs become CE marked. As shown in the figure, the 
> AoA fake DupACK will conceal the loss of data 200. If A knows that 
> DSACK is implemented at B and B never includes a SACK option in AoAs, 
> the TLP logic at A can be modified to ignore DupACKs without a SACK 
> option. With such modification AoA (DupACK 201) is ignored and does 
> not end the TLP episode, allowing ACK 202 to reveal the loss and A to 
> correctly react to loss.
>
>
>>> If SACK blocks are removed in the middle, it does not help ECN++ sender
>>> even if the other end is required to apply DSACK but additional
>>> safeguards are required.
>>
>> I mentioned this, as it is a (rather uncommon) pathological issue which
>> may happen.
>
> [MK] I am not aware of any data showing how common it is today but it 
> is known to happen. Do you have any pointers to Internet-wide data?
> IMO, a proper stds track spec should take such misbehaviour into account.
>
> [MK] And, it is not only that SACK blocks may get removed in the 
> middle. There are (or at least have been) TCP implementations that 
> negotiate SACK but do not include (any) SACK blocks. This is another 
> reason, why it is important require in the AccECN draft that an AccECN 
> implementation MUST implement both RFC 2018 and RFC 2883 in full, if 
> it sends Acks of Acks.
>
>> With the suggested discrimination of ACE-ACKs, that ACK (and only this
>> particular one) would be excluded from subsequent DupACK processing.
>> Since genuine DupACKs would not match the ACE-ACK checks, these would
>> trigger all other mechanisms - albeit at most 1 ACK "late".
>
> [MK] Pls, Ack if you agree that the ACE rule that you suggested is 
> unreliable and cannot be applied (as shown in Fig 5 a).
>
>>
>>> I'm not sure if No SACK and ACE<3 rule is good
>>> enough, because no full analysis nor experimental evidence exists to
>>> judge on.
>>
>> Which is why I asked repeatedly for some good examples and scenarios to
>> fully analyze.
>
> [MK] My suggested rule "if No SACK and ACE delta <3, the turn off 
> ECN-capable pure Acks" seems not workable anymore after Bob's 
> clarification for the "Increment-Triggered ACKs" rule. But, if there 
> is no SACK block in DupACK and ACE delta=0, doesn't it indicate that 
> SACK blocks are removed (given that MUST implement RFC 2018 and RFC 
> 2883 and reguired in AccECN draft)? The common security practice that 
> IETF has followed earlier is that if misbehaviour that brakes the 
> designed logic for some feature is detected, then that feature should 
> be disabled.
> Or are you suggesting that this is unreasonable practice?

[BB] Security is not that simple. You are effectively mandating 
vulnerability to a downgrade attack.

If a middlebox strips SACK blocks, it already degrades the performance 
of many of TCP's loss recovery algorithms, and the whole of RACK and TLP 
- except for the one special case you have singled out: 7.4.2. 
<https://datatracker.ietf.org/doc/html/rfc8985#section-7.4.2>Special 
Case: Detecting a Single Loss Repaired by the Loss Probe 
<https://datatracker.ietf.org/doc/html/rfc8985#name-special-case-detecting-a-si>.

If a sender sends ECN-capable pure ACKs, such a middlebox still breaks 
everything it broke before, plus this single special case as well. 
Disabling ECN-capable pure ACKs because it widens the impact of a broken 
middlebox to one more special case seems like overkill. But I agree that 
disabling ECN-capable pure ACKs would be simpler than introducing 
Richard's additional heuristics.


Bob


>
>>> Particularly, I am worried about short flows that are very
>>> common and with such flows genuine DupACKs may get interpreted
>>> incorrectly.
>>
>> Well, what is your definition of a short flow? Would a flow with a
>> congestion window of at least 18 segments, for the minimal theoretical
>> possible number of delayed, ECT-enabled ACKs, all of which would get
>> CE-marked, count as a short flow? Remember that this would require in
>> the order of 30 segments (or 45 kB) at the very least.
>
> [MK] Sorry, I donät follow. The number of ECT-enabled ACKs is not 
> coverned by the flow that (potentially) suffers from the AoAs but the 
> flow of data from the other direction. So, you may have a data flow 
> from A to B of any lenght and and how many AoAs it may encounter 
> depends (almost) solely on the data flow from B to A. Almost solelely 
> as AoA ping-pong effect may be affected by the flow from A to B itself.
> And, your 18 segments minimum rule holds only for the case where three 
> DupACKs are needed to trigger false fast rexmit. Actually, the minimum 
> cwnd to trigger enough ECT-enabled ACKs for three DupACKs is not 18 
> because if there is a pkt loss, it is enough to have 9 out-of-order 
> data segments that will trigger 9 ECN-capable DupACKs.
> Moreover, breaking the RACK-TLP or F-RTO logic reguires just one fake 
> AoA DupACK.
>
>> But at this bare minimum flow size, ending after this, none of any
>> possible misclassification or erraneous processing would matter.
>
> [MK] Right, it is not that serious if the flow ends. I was thinking a 
> common application limited behaviour that consists of short data 
> bursts that are repeated (which maybe should not be called a short 
> flow). If the end of flight loss in each such burst is repeatedly 
> misinterpreted (loss concealed), it starts to be quite bad behaviour.
>
>> So, the short flow would need to be larger than this. And even (much)
>> larger to account for the much lower probability of CE-marked ACKs to
>> happen.
>>
>>> This means that a rule should rather be used in different
>>> way than Richard suggested; the stack should immediately start treating
>>> DupACKs without SACK as genuine DupACKs for the rest of the connection
>>> in addition to turning ECN-capable pure ACKs off.
>>
>> Whats the definition of your DupACK here? I'm confused. Rather than
>
> [MK] My apologies for missing (not repeating) a crucial point. After 
> detecting misbehavior (SACK block are being removed), the stack should 
> *disable ECN-capable pure ACKs* and start treating DupACKs without 
> SACK as genuine DupACKs in order to allow correct behaviour of all 
> algos that depend on the crucial RFC 5681 rule that an ACK MUST NOT be 
> generated more than once for any data segment.
>
> Hope this clarifies,
>
> /Markku
>
>  > determining which ACK may NOT be a DupACK to stop confusing earlier
>> mechanisms, which don't have an understanding of AccECN and ECT-enabled
>> ACKs, suddenly every DupAck (presumably masking out the presence/change
>> of the ACE field, as well as any TCP option - including SACK in
>> particular) should be accepted as DupACK under all circumstances?
>>
>> This confuses me even more...
>>
>>
>> Sorry,
>>   Richard
>>

-- 
________________________________________________________________
Bob Briscoehttp://bobbriscoe.net/