Re: [tcpm] AccECN and DSACK

[Yoshifumi Nishida <[e-mail]>]

Hi folks,

I think there are discussions for ECN++ spec and Accecn spec in this
thread.
I would like to focus on only Accecn spec at first so that we can finalize
the draft and ship it quickly.

As far as I look at Fig 3b and Fig 5 in
https://www.cs.helsinki.fi/u/kojo/IETF/AccECN2023.pdf, A must be an ECN++
node because it sets ECT marks on ACKs. It cannot be just an AccECN node.
I strongly believe the behavior of A should be specified in ECN++ draft,
but not accecn draft.

So, I think we just need to check if we want to mandate D-SACK in accecn
draft here.

I think I understand the case described in Fig 5a. But, it seems to me that
several conditions are required for this scenario. E.g, I think sending
exact 2 ack-on-ack packets right before TLP retransmission is unlikely to
happen very often.
Also, if A doesn't like this scenario, A simply can choose not to set ECT
marks on ACKs, but I think this should be a part of ECN++ spec anyway.
In case of Fig 5b, I think it depends on how to detect Ack-on-ack packets
at A, which should also belong to ECN++ spec.

So, I think a question here would be if we want to mandate D-SACK for
relatively small possibilities or not.
If I overlook something, please let me know

Thanks,
--
Yoshi


On Mon, Nov 20, 2023 at 12:33 PM Markku Kojo <[e-mail]> wrote:

> Hi Richard, all
>
> On Fri, 17 Nov 2023, [e-mail] wrote:
>
> > Hi Markku,
> >
> >
> > Am 17.11.2023 um 18:26 schrieb Markku Kojo:
> >> Bob, Richard, Yoshi, all,
> >>
> >> On Wed, 15 Nov 2023, Bob Briscoe wrote:
> >>
> >>> Richard,
> >>>
> >>> Thx for the clarification. So the draft will say nothing about
> >>> timestamps.
> >>>
> >>> To confirm, in the editor's copy of ECN++, I'll add the Δ(ACE)≥3
> >>> requirement to the additional DupACK check (In
> >>> deployments where SACK blocks might sometimes be removed, and ideally
> >>> only if no SACK blocks have been seen at all, despite SACK having been
> >>> negotiated).
> >>
> >> I'm afraid that the suggested rule (ACE counter is 3 or more) is
> >> extreamily unreliable. It should be quite easy to see that there are a
> >> multitude of scenarios where the data pkt that triggers a genuine DupACK
> >> can itself be CE-marked and preceded by two (or more) CE-marked pure
> >> ACKs, in which case the ACE counter in the DupACK is three (or higher),
> >> resulting in genuine DupACK being ignored. Sounds like a very bad idea
> >> to me.
> >
> > Sorry. You completely lost me.
> >
> > How is an ACK of new data a dupack?
>
> [MK] And you lost me totally, sorry. ;)
> I don't know what made you think that the data pkt could carry new,
> in-order data? I said "data pkt that triggers a genuine DupACK". How can
> this data pkt be other than out-of-order or duplicate data pkt if it
> triggers a DupACK? It is definitely not a new data pkt arriving in order
> and triggering a new ACK. And, yes, in case of out-of-order data pkt it
> carries new data but that data won't get acknowledged but triggers a
> DupACK.
> I thought this should not require a specific scenario to understand it,
> sorry. I should have been more accurate and say that the data pkt
> is either duplicate or out-of order pkt as well as the additional details
> I now add below. Whichever specific scenario you select where a pkt
> receiver receives the following sequence of pkts that I described, will
> result in ACE delta being 3. Or maybe I am missing something?
>
> Assume no unacknowledged data at the pkt receiver and all earlier CE marks
> have been acknowledged:
> 1. A pure ACK marked CE arrives (CE pkt counter incremented by 1, does not
> trigger an ACK).
> 2. A pure ACK marked CE arrives (CE pkt counter incremented by 1, does
> not trigger an ACK).
> 3. A duplicate or out-of-order data pkt arrives (CE pkt counter
> incremented by 1, triggers genuine DupACK with ACE delta = 3).
>
> Please find also a specific scenario that breaks TLP logic (see below).
>
> > Or, since I have to imagine what scenario you are thinking about (please
> > - provide a good depiction of the segment sequences exchanged), how
> > would a discontinous data packet, with the same ACK seq number (to be a
> > dup ack) NOT carry a SACK block?
>
> [MK] I think I have said already earlier that with F-RTO or RACK-TLP we
> are not considering arrival of out-of-order data pkts that trigger three
> DupACKs or three AoAs that may be incorrectly interpreted as three
> DupACKs.
> When a duplicate data segment for the highest seg number already
> received by the data receiver arrives it triggers DupACK without a SACK
> option as per RFC 2018. Or, if SACK blocks were removed like Bob
> mentioned above, then it may be an out-of-order data pkt.
>
> > Furthermore, a singular ACK for a new data packet is commonly not
> > refered to as DupACK, but a regular in-sequence ACK.
> >
> > Or are we suddenly bringing in subsequent data packets with or without
> > loss/reordering?
> >
> > In any case, as described it is *extremely* easy to differentiate from a
> > genuine ACE-ACK (new ACK number, likely more recent timestamp, while not
> > containing SACK).
> >
> >
> >> Maybe the best way is to require the other end to implement and apply
> >> DSACK (MUST), and use DupACKs arriving without SACK and ACE<3 to turn
> >> off ECN-capable pure ACKs for the rest of the connection.
> >
> > Again, I can not follow. How would DSACK help in the above situation?
> >
> > If the new CE-marked data segment is beyond the right edge, it either
> > advances ACK, or carries a SACK (but the scenario didn't contain any
> > losses...). If it is a TLP packet overlapping with the first
> > transmission, by virtue of (still) being at the very right edge, this
> > may be the 1st DupACK (not triggering any DupACK based mechanism), but
> > even if we now conjecture 3 TLPs for some arcane reason, and no DSACK
> > (despite all RACK implementations I know do support DSACK by virtue to
> > improving the RACK mechansims), since these would still be at the
> > rightmost edge, no erraneous (additional) retransmission (with CC
> > response) would be triggered. (Please explain first, how three
> > consecutive TLP transmissions would occur, since these are only
> > happening once).
>
> [MK] Sorry, I don't follow. We are discussing RACK-TLP (TLP part for
> it) and F-RTO that have been designed to distinguish whether arriving
> ACKs are due to real loss or duplicate data segment(s). A single
> extra DupACK (or ignored DupACK) may break the logic.
>
> >
> >> Without the data sender (AoA receiver) knowing that the other end
> >> employs SACK, I cannot find a reliable enough way to distinguish genuine
> >> DupACK and AoA.
> >
> > Which is why the guidance is already to only allow the use of
> > ECT-enabled ACKs, when the session does have SACK enabled.
>
> [MK] Yes, and I have been trying to explain that enabling SACK is not
> enough if the other end (data receiver) does not implement DSACK.
>
> >> Currently there is no RFC I am aware of that says
> >> negotiating SACK would negotiate DSACK as well or that if one implements
> >> SACK one SHOULD/MUST implement DSACK as well. As long as this is not
> >> said in any RFC, IMO it would be inappropriate to rely on DSACK being
> >> employed as specified in RFC 2883. That is one important reason (not the
> >> only even though I earlier only mentioned this) why RFC 5682 and RFC
> >> 8985 do not rely on DSACK alone but treat the non-SACK DupACKS the same
> >> as DSACK DupACKs to ensure correct operation in the current Internet
> >> (works also with F-RTO even if SACk-blocks are removed).
> >
> > Please provide a fully worked example showing how excluding an ACK which
> > has delta-ACE >= 3, no SACK blocks, which is not advancing snd_una from
> > subsequent DupAck processing breaks existing mechanisms. Ideally in a
> > situation where DSACK would have provided a different outcome.
>
> [MK] Please see the scenarios (Fig 5 a) that I have added and are
> available at:
>
>   https://www.cs.helsinki.fi/u/kojo/IETF/AccECN2023.pdf
>
> For your convenience, the first two added scenarios (Fig 4 a) and b)) are
> without any AoA interference. That is, they are for the two normal cases
> (duplicate data segment due to spurious TLP and real loss) that TLP logic
> has been designed to distinguish from each other even when DSACK is not
> implemented by the data receiver. See RFC 8985, Sec 7.4.2 that I have
> pointed to already earlier (maybe more than once). Pls, see also the
> description in the RFC (case 2). In both cases for Fig 4 the ACKs are
> delayed. The only difference is that the end of flight data segment (data
> 200) is not lost in Fig 4 a) while in Fig 4 b) it is lost.
>
> In the Figures 5 a) and 5 b) I have modified the data traffic from B to A
> such that an appropriate amount of ECN-capable ACKs may get triggered from
> A to B and (some of them) marked CE. In Fig 5 a) there may be any number
> data pkts (>=2) from B to A arriving at A after the original transmission
> of data 200 but before the PTO-triggred retrinsmission of data 200 as long
> as they trigger pure ACKs of which two become CE_marked. As shown in the
> Fig 5 a), if B does not implement DSACK it will result in the AoA (DupACK
> 201 with ACE delta = 3) becoming ignored and TLP logic makes incorrect
> desicion as it does not detect the spurious PTO. So, ACE rule seems not
> reliable to me but maybe I am missing something? If B implements DSACK,
> it would include DSACK block for data 200 in the DupACK 201 and TLP logic
> would correctly detect spurious PTO (case 1) unless it is removed in the
> middle.
>
> Fig 5 b) shows a scenario where data receiver does not implement DSACK,
> resulting in A not detecting loss and inappropriately not reacting
> to congestion which is one of the worst violations against the congestion
> control principles. In my reading of the various replies, it seems to me
> that I and Bob agree that DSACK MUST be implemented at a data receiver
> and AoA MUST not include SACK option (requirements in the AccECN draft)
> but Richard seems to disagree(?). So, this is a scenario that Richerd
> requested earlier in this thread.
> Even though the ACE rule may work in the scenario in Fig 5 b), I ignore
> it here because it seems unusable as per shown in Fig 5 a). Similarly to
> Fig a), in Fig 5 b) there may be any number of data segments from B to
> A as long as they arrive at B before the TLP probe (rexmit of data 200)
> and they trigger at least three pure ACKS out of which three pure ACKs
> become CE marked. As shown in the figure, the AoA fake DupACK will
> conceal the loss of data 200. If A knows that DSACK is implemented at
> B and B never includes a SACK option in AoAs, the TLP logic at A can be
> modified to ignore DupACKs without a SACK option. With such modification
> AoA (DupACK 201) is ignored and does not end the TLP episode, allowing
> ACK 202 to reveal the loss and A to correctly react to loss.
>
>
> >> If SACK blocks are removed in the middle, it does not help ECN++ sender
> >> even if the other end is required to apply DSACK but additional
> >> safeguards are required.
> >
> > I mentioned this, as it is a (rather uncommon) pathological issue which
> > may happen.
>
> [MK] I am not aware of any data showing how common it is today but it is
> known to happen. Do you have any pointers to Internet-wide data?
> IMO, a proper stds track spec should take such misbehaviour into
> account.
>
> [MK] And, it is not only that SACK blocks may get removed in the middle.
> There are (or at least have been) TCP implementations that negotiate SACK
> but do not include (any) SACK blocks. This is another reason, why it is
> important require in the AccECN draft that an AccECN implementation
> MUST implement both RFC 2018 and RFC 2883 in full, if it sends Acks of
> Acks.
>
> > With the suggested discrimination of ACE-ACKs, that ACK (and only this
> > particular one) would be excluded from subsequent DupACK processing.
> > Since genuine DupACKs would not match the ACE-ACK checks, these would
> > trigger all other mechanisms - albeit at most 1 ACK "late".
>
> [MK] Pls, Ack if you agree that the ACE rule that you suggested is
> unreliable and cannot be applied (as shown in Fig 5 a).
>
> >
> >> I'm not sure if No SACK and ACE<3 rule is good
> >> enough, because no full analysis nor experimental evidence exists to
> >> judge on.
> >
> > Which is why I asked repeatedly for some good examples and scenarios to
> > fully analyze.
>
> [MK] My suggested rule "if No SACK and ACE delta <3, the turn off
> ECN-capable pure Acks" seems not workable anymore after Bob's
> clarification for the "Increment-Triggered ACKs" rule. But, if there is
> no SACK block in DupACK and ACE delta=0, doesn't it indicate that SACK
> blocks are removed (given that MUST implement RFC 2018 and RFC 2883 and
> reguired in AccECN draft)? The common security practice that IETF has
> followed earlier is that if misbehaviour that brakes the designed logic
> for some feature is detected, then that feature should be disabled.
> Or are you suggesting that this is unreasonable practice?
>
> >> Particularly, I am worried about short flows that are very
> >> common and with such flows genuine DupACKs may get interpreted
> >> incorrectly.
> >
> > Well, what is your definition of a short flow? Would a flow with a
> > congestion window of at least 18 segments, for the minimal theoretical
> > possible number of delayed, ECT-enabled ACKs, all of which would get
> > CE-marked, count as a short flow? Remember that this would require in
> > the order of 30 segments (or 45 kB) at the very least.
>
> [MK] Sorry, I donät follow. The number of ECT-enabled ACKs is not coverned
> by the flow that (potentially) suffers from the AoAs but the flow of data
> from the other direction. So, you may have a data flow from A to B of any
> lenght and and how many AoAs it may encounter depends (almost) solely on
> the data flow from B to A. Almost solelely as AoA ping-pong effect may
> be affected by the flow from A to B itself.
> And, your 18 segments minimum rule holds only for the case where three
> DupACKs are needed to trigger false fast rexmit. Actually, the minimum
> cwnd to trigger enough ECT-enabled ACKs for three DupACKs is not 18
> because if there is a pkt loss, it is enough to have 9 out-of-order data
> segments that will trigger 9 ECN-capable DupACKs.
> Moreover, breaking the RACK-TLP or F-RTO logic reguires just one fake AoA
> DupACK.
>
> > But at this bare minimum flow size, ending after this, none of any
> > possible misclassification or erraneous processing would matter.
>
> [MK] Right, it is not that serious if the flow ends. I was thinking a
> common application limited behaviour that consists of short data bursts
> that are repeated (which maybe should not be called a short flow). If the
> end of flight loss in each such burst is repeatedly misinterpreted (loss
> concealed), it starts to be quite bad behaviour.
>
> > So, the short flow would need to be larger than this. And even (much)
> > larger to account for the much lower probability of CE-marked ACKs to
> > happen.
> >
> >> This means that a rule should rather be used in different
> >> way than Richard suggested; the stack should immediately start treating
> >> DupACKs without SACK as genuine DupACKs for the rest of the connection
> >> in addition to turning ECN-capable pure ACKs off.
> >
> > Whats the definition of your DupACK here? I'm confused. Rather than
>
> [MK] My apologies for missing (not repeating) a crucial point. After
> detecting misbehavior (SACK block are being removed), the stack should
> *disable ECN-capable pure ACKs* and start treating DupACKs without SACK
> as genuine DupACKs in order to allow correct behaviour of all algos
> that depend on the crucial RFC 5681 rule that an ACK MUST NOT be
> generated more than once for any data segment.
>
> Hope this clarifies,
>
> /Markku
>
>   > determining which ACK may NOT be a DupACK to stop confusing earlier
> > mechanisms, which don't have an understanding of AccECN and ECT-enabled
> > ACKs, suddenly every DupAck (presumably masking out the presence/change
> > of the ACE field, as well as any TCP option - including SACK in
> > particular) should be accepted as DupACK under all circumstances?
> >
> > This confuses me even more...
> >
> >
> > Sorry,
> >   Richard
> >