Re: [tcpm] AccECN and DSACK

[Bob Briscoe <[e-mail]>]

Yoshi,

On this list, we already agreed to Markku's request to require D-SACK if 
SACK has been implemented (even though at that time we hadn't been given 
a specific scenario where it would be needed, but we agreed to it 
because it would be unlikely that any implementer would actively *not* 
want to implement D-SACK if they had already implemented SACK and AccECN).

The second sentence below was added in rev 28 which was posted last Friday.

    It is RECOMMENDED that the AccECN protocol is implemented alongside
    SACK [RFC2018 <https://www.rfc-editor.org/info/rfc2018>]. *If SACK
    is implemented with AccECN, DSACK [RFC2883
    <https://www.rfc-editor.org/info/rfc2883>] MUST also be implemented.*

I also asked Markku to focus solely on problems that would /not/ be 
solved by the measures now actually written in the drafts.


Bob

On 21/11/2023 03:43, Yoshifumi Nishida wrote:
> Hi folks,
>
> I think there are discussions for ECN++ spec and Accecn spec in this 
> thread.
> I would like to focus on only Accecn spec at first so that we can 
> finalize the draft and ship it quickly.
>
> As far as I look at Fig 3b and Fig 5 in 
> https://www.cs.helsinki.fi/u/kojo/IETF/AccECN2023.pdf, A must be an 
> ECN++ node because it sets ECT marks on ACKs. It cannot be just an 
> AccECN node.
> I strongly believe the behavior of A should be specified in 
> ECN++ draft, but not accecn draft.
>
> So, I think we just need to check if we want to mandate D-SACK in 
> accecn draft here.
>
> I think I understand the case described in Fig 5a. But, it seems to me 
> that several conditions are required for this scenario. E.g, I think 
> sending exact 2 ack-on-ack packets right before TLP retransmission is 
> unlikely to happen very often.
> Also, if A doesn't like this scenario, A simply can choose not to set 
> ECT marks on ACKs, but I think this should be a part of ECN++ spec anyway.
> In case of Fig 5b, I think it depends on how to detect Ack-on-ack 
> packets at A, which should also belong to ECN++ spec.
>
> So, I think a question here would be if we want to mandate D-SACK for 
> relatively small possibilities or not.
> If I overlook something, please let me know
>
> Thanks,
> --
> Yoshi
>
>
> On Mon, Nov 20, 2023 at 12:33 PM Markku Kojo <[e-mail]> wrote:
>
>     Hi Richard, all
>
>     On Fri, 17 Nov 2023, [e-mail] wrote:
>
>     > Hi Markku,
>     >
>     >
>     > Am 17.11.2023 um 18:26 schrieb Markku Kojo:
>     >> Bob, Richard, Yoshi, all,
>     >>
>     >> On Wed, 15 Nov 2023, Bob Briscoe wrote:
>     >>
>     >>> Richard,
>     >>>
>     >>> Thx for the clarification. So the draft will say nothing about
>     >>> timestamps.
>     >>>
>     >>> To confirm, in the editor's copy of ECN++, I'll add the Δ(ACE)≥3
>     >>> requirement to the additional DupACK check (In
>     >>> deployments where SACK blocks might sometimes be removed, and
>     ideally
>     >>> only if no SACK blocks have been seen at all, despite SACK
>     having been
>     >>> negotiated).
>     >>
>     >> I'm afraid that the suggested rule (ACE counter is 3 or more) is
>     >> extreamily unreliable. It should be quite easy to see that
>     there are a
>     >> multitude of scenarios where the data pkt that triggers a
>     genuine DupACK
>     >> can itself be CE-marked and preceded by two (or more) CE-marked
>     pure
>     >> ACKs, in which case the ACE counter in the DupACK is three (or
>     higher),
>     >> resulting in genuine DupACK being ignored. Sounds like a very
>     bad idea
>     >> to me.
>     >
>     > Sorry. You completely lost me.
>     >
>     > How is an ACK of new data a dupack?
>
>     [MK] And you lost me totally, sorry. ;)
>     I don't know what made you think that the data pkt could carry new,
>     in-order data? I said "data pkt that triggers a genuine DupACK".
>     How can
>     this data pkt be other than out-of-order or duplicate data pkt if it
>     triggers a DupACK? It is definitely not a new data pkt arriving in
>     order
>     and triggering a new ACK. And, yes, in case of out-of-order data
>     pkt it
>     carries new data but that data won't get acknowledged but triggers a
>     DupACK.
>     I thought this should not require a specific scenario to
>     understand it,
>     sorry. I should have been more accurate and say that the data pkt
>     is either duplicate or out-of order pkt as well as the additional
>     details
>     I now add below. Whichever specific scenario you select where a pkt
>     receiver receives the following sequence of pkts that I described,
>     will
>     result in ACE delta being 3. Or maybe I am missing something?
>
>     Assume no unacknowledged data at the pkt receiver and all earlier
>     CE marks
>     have been acknowledged:
>     1. A pure ACK marked CE arrives (CE pkt counter incremented by 1,
>     does not
>     trigger an ACK).
>     2. A pure ACK marked CE arrives (CE pkt counter incremented by 1,
>     does
>     not trigger an ACK).
>     3. A duplicate or out-of-order data pkt arrives (CE pkt counter
>     incremented by 1, triggers genuine DupACK with ACE delta = 3).
>
>     Please find also a specific scenario that breaks TLP logic (see
>     below).
>
>     > Or, since I have to imagine what scenario you are thinking about
>     (please
>     > - provide a good depiction of the segment sequences exchanged), how
>     > would a discontinous data packet, with the same ACK seq number
>     (to be a
>     > dup ack) NOT carry a SACK block?
>
>     [MK] I think I have said already earlier that with F-RTO or
>     RACK-TLP we
>     are not considering arrival of out-of-order data pkts that trigger
>     three
>     DupACKs or three AoAs that may be incorrectly interpreted as three
>     DupACKs.
>     When a duplicate data segment for the highest seg number already
>     received by the data receiver arrives it triggers DupACK without a
>     SACK
>     option as per RFC 2018. Or, if SACK blocks were removed like Bob
>     mentioned above, then it may be an out-of-order data pkt.
>
>     > Furthermore, a singular ACK for a new data packet is commonly not
>     > refered to as DupACK, but a regular in-sequence ACK.
>     >
>     > Or are we suddenly bringing in subsequent data packets with or
>     without
>     > loss/reordering?
>     >
>     > In any case, as described it is *extremely* easy to
>     differentiate from a
>     > genuine ACE-ACK (new ACK number, likely more recent timestamp,
>     while not
>     > containing SACK).
>     >
>     >
>     >> Maybe the best way is to require the other end to implement and
>     apply
>     >> DSACK (MUST), and use DupACKs arriving without SACK and ACE<3
>     to turn
>     >> off ECN-capable pure ACKs for the rest of the connection.
>     >
>     > Again, I can not follow. How would DSACK help in the above
>     situation?
>     >
>     > If the new CE-marked data segment is beyond the right edge, it
>     either
>     > advances ACK, or carries a SACK (but the scenario didn't contain any
>     > losses...). If it is a TLP packet overlapping with the first
>     > transmission, by virtue of (still) being at the very right edge,
>     this
>     > may be the 1st DupACK (not triggering any DupACK based
>     mechanism), but
>     > even if we now conjecture 3 TLPs for some arcane reason, and no
>     DSACK
>     > (despite all RACK implementations I know do support DSACK by
>     virtue to
>     > improving the RACK mechansims), since these would still be at the
>     > rightmost edge, no erraneous (additional) retransmission (with CC
>     > response) would be triggered. (Please explain first, how three
>     > consecutive TLP transmissions would occur, since these are only
>     > happening once).
>
>     [MK] Sorry, I don't follow. We are discussing RACK-TLP (TLP part for
>     it) and F-RTO that have been designed to distinguish whether arriving
>     ACKs are due to real loss or duplicate data segment(s). A single
>     extra DupACK (or ignored DupACK) may break the logic.
>
>     >
>     >> Without the data sender (AoA receiver) knowing that the other end
>     >> employs SACK, I cannot find a reliable enough way to
>     distinguish genuine
>     >> DupACK and AoA.
>     >
>     > Which is why the guidance is already to only allow the use of
>     > ECT-enabled ACKs, when the session does have SACK enabled.
>
>     [MK] Yes, and I have been trying to explain that enabling SACK is not
>     enough if the other end (data receiver) does not implement DSACK.
>
>     >> Currently there is no RFC I am aware of that says
>     >> negotiating SACK would negotiate DSACK as well or that if one
>     implements
>     >> SACK one SHOULD/MUST implement DSACK as well. As long as this
>     is not
>     >> said in any RFC, IMO it would be inappropriate to rely on DSACK
>     being
>     >> employed as specified in RFC 2883. That is one important reason
>     (not the
>     >> only even though I earlier only mentioned this) why RFC 5682
>     and RFC
>     >> 8985 do not rely on DSACK alone but treat the non-SACK DupACKS
>     the same
>     >> as DSACK DupACKs to ensure correct operation in the current
>     Internet
>     >> (works also with F-RTO even if SACk-blocks are removed).
>     >
>     > Please provide a fully worked example showing how excluding an
>     ACK which
>     > has delta-ACE >= 3, no SACK blocks, which is not advancing
>     snd_una from
>     > subsequent DupAck processing breaks existing mechanisms. Ideally
>     in a
>     > situation where DSACK would have provided a different outcome.
>
>     [MK] Please see the scenarios (Fig 5 a) that I have added and are
>     available at:
>
>     https://www.cs.helsinki.fi/u/kojo/IETF/AccECN2023.pdf
>
>     For your convenience, the first two added scenarios (Fig 4 a) and
>     b)) are
>     without any AoA interference. That is, they are for the two normal
>     cases
>     (duplicate data segment due to spurious TLP and real loss) that
>     TLP logic
>     has been designed to distinguish from each other even when DSACK
>     is not
>     implemented by the data receiver. See RFC 8985, Sec 7.4.2 that I have
>     pointed to already earlier (maybe more than once). Pls, see also the
>     description in the RFC (case 2). In both cases for Fig 4 the ACKs are
>     delayed. The only difference is that the end of flight data
>     segment (data
>     200) is not lost in Fig 4 a) while in Fig 4 b) it is lost.
>
>     In the Figures 5 a) and 5 b) I have modified the data traffic from
>     B to A
>     such that an appropriate amount of ECN-capable ACKs may get
>     triggered from
>     A to B and (some of them) marked CE. In Fig 5 a) there may be any
>     number
>     data pkts (>=2) from B to A arriving at A after the original
>     transmission
>     of data 200 but before the PTO-triggred retrinsmission of data 200
>     as long
>     as they trigger pure ACKs of which two become CE_marked. As shown
>     in the
>     Fig 5 a), if B does not implement DSACK it will result in the AoA
>     (DupACK
>     201 with ACE delta = 3) becoming ignored and TLP logic makes
>     incorrect
>     desicion as it does not detect the spurious PTO. So, ACE rule
>     seems not
>     reliable to me but maybe I am missing something? If B implements
>     DSACK,
>     it would include DSACK block for data 200 in the DupACK 201 and
>     TLP logic
>     would correctly detect spurious PTO (case 1) unless it is removed
>     in the
>     middle.
>
>     Fig 5 b) shows a scenario where data receiver does not implement
>     DSACK,
>     resulting in A not detecting loss and inappropriately not reacting
>     to congestion which is one of the worst violations against the
>     congestion
>     control principles. In my reading of the various replies, it seems
>     to me
>     that I and Bob agree that DSACK MUST be implemented at a data
>     receiver
>     and AoA MUST not include SACK option (requirements in the AccECN
>     draft)
>     but Richard seems to disagree(?). So, this is a scenario that Richerd
>     requested earlier in this thread.
>     Even though the ACE rule may work in the scenario in Fig 5 b), I
>     ignore
>     it here because it seems unusable as per shown in Fig 5 a).
>     Similarly to
>     Fig a), in Fig 5 b) there may be any number of data segments from
>     B to
>     A as long as they arrive at B before the TLP probe (rexmit of data
>     200)
>     and they trigger at least three pure ACKS out of which three pure
>     ACKs
>     become CE marked. As shown in the figure, the AoA fake DupACK will
>     conceal the loss of data 200. If A knows that DSACK is implemented at
>     B and B never includes a SACK option in AoAs, the TLP logic at A
>     can be
>     modified to ignore DupACKs without a SACK option. With such
>     modification
>     AoA (DupACK 201) is ignored and does not end the TLP episode,
>     allowing
>     ACK 202 to reveal the loss and A to correctly react to loss.
>
>
>     >> If SACK blocks are removed in the middle, it does not help
>     ECN++ sender
>     >> even if the other end is required to apply DSACK but additional
>     >> safeguards are required.
>     >
>     > I mentioned this, as it is a (rather uncommon) pathological
>     issue which
>     > may happen.
>
>     [MK] I am not aware of any data showing how common it is today but
>     it is
>     known to happen. Do you have any pointers to Internet-wide data?
>     IMO, a proper stds track spec should take such misbehaviour into
>     account.
>
>     [MK] And, it is not only that SACK blocks may get removed in the
>     middle.
>     There are (or at least have been) TCP implementations that
>     negotiate SACK
>     but do not include (any) SACK blocks. This is another reason, why
>     it is
>     important require in the AccECN draft that an AccECN implementation
>     MUST implement both RFC 2018 and RFC 2883 in full, if it sends
>     Acks of
>     Acks.
>
>     > With the suggested discrimination of ACE-ACKs, that ACK (and
>     only this
>     > particular one) would be excluded from subsequent DupACK processing.
>     > Since genuine DupACKs would not match the ACE-ACK checks, these
>     would
>     > trigger all other mechanisms - albeit at most 1 ACK "late".
>
>     [MK] Pls, Ack if you agree that the ACE rule that you suggested is
>     unreliable and cannot be applied (as shown in Fig 5 a).
>
>     >
>     >> I'm not sure if No SACK and ACE<3 rule is good
>     >> enough, because no full analysis nor experimental evidence
>     exists to
>     >> judge on.
>     >
>     > Which is why I asked repeatedly for some good examples and
>     scenarios to
>     > fully analyze.
>
>     [MK] My suggested rule "if No SACK and ACE delta <3, the turn off
>     ECN-capable pure Acks" seems not workable anymore after Bob's
>     clarification for the "Increment-Triggered ACKs" rule. But, if
>     there is
>     no SACK block in DupACK and ACE delta=0, doesn't it indicate that
>     SACK
>     blocks are removed (given that MUST implement RFC 2018 and RFC
>     2883 and
>     reguired in AccECN draft)? The common security practice that IETF has
>     followed earlier is that if misbehaviour that brakes the designed
>     logic
>     for some feature is detected, then that feature should be disabled.
>     Or are you suggesting that this is unreasonable practice?
>
>     >> Particularly, I am worried about short flows that are very
>     >> common and with such flows genuine DupACKs may get interpreted
>     >> incorrectly.
>     >
>     > Well, what is your definition of a short flow? Would a flow with a
>     > congestion window of at least 18 segments, for the minimal
>     theoretical
>     > possible number of delayed, ECT-enabled ACKs, all of which would get
>     > CE-marked, count as a short flow? Remember that this would
>     require in
>     > the order of 30 segments (or 45 kB) at the very least.
>
>     [MK] Sorry, I donät follow. The number of ECT-enabled ACKs is not
>     coverned
>     by the flow that (potentially) suffers from the AoAs but the flow
>     of data
>     from the other direction. So, you may have a data flow from A to B
>     of any
>     lenght and and how many AoAs it may encounter depends (almost)
>     solely on
>     the data flow from B to A. Almost solelely as AoA ping-pong effect
>     may
>     be affected by the flow from A to B itself.
>     And, your 18 segments minimum rule holds only for the case where
>     three
>     DupACKs are needed to trigger false fast rexmit. Actually, the
>     minimum
>     cwnd to trigger enough ECT-enabled ACKs for three DupACKs is not 18
>     because if there is a pkt loss, it is enough to have 9
>     out-of-order data
>     segments that will trigger 9 ECN-capable DupACKs.
>     Moreover, breaking the RACK-TLP or F-RTO logic reguires just one
>     fake AoA
>     DupACK.
>
>     > But at this bare minimum flow size, ending after this, none of any
>     > possible misclassification or erraneous processing would matter.
>
>     [MK] Right, it is not that serious if the flow ends. I was thinking a
>     common application limited behaviour that consists of short data
>     bursts
>     that are repeated (which maybe should not be called a short flow).
>     If the
>     end of flight loss in each such burst is repeatedly misinterpreted
>     (loss
>     concealed), it starts to be quite bad behaviour.
>
>     > So, the short flow would need to be larger than this. And even
>     (much)
>     > larger to account for the much lower probability of CE-marked
>     ACKs to
>     > happen.
>     >
>     >> This means that a rule should rather be used in different
>     >> way than Richard suggested; the stack should immediately start
>     treating
>     >> DupACKs without SACK as genuine DupACKs for the rest of the
>     connection
>     >> in addition to turning ECN-capable pure ACKs off.
>     >
>     > Whats the definition of your DupACK here? I'm confused. Rather than
>
>     [MK] My apologies for missing (not repeating) a crucial point. After
>     detecting misbehavior (SACK block are being removed), the stack
>     should
>     *disable ECN-capable pure ACKs* and start treating DupACKs without
>     SACK
>     as genuine DupACKs in order to allow correct behaviour of all algos
>     that depend on the crucial RFC 5681 rule that an ACK MUST NOT be
>     generated more than once for any data segment.
>
>     Hope this clarifies,
>
>     /Markku
>
>       > determining which ACK may NOT be a DupACK to stop confusing
>     earlier
>     > mechanisms, which don't have an understanding of AccECN and
>     ECT-enabled
>     > ACKs, suddenly every DupAck (presumably masking out the
>     presence/change
>     > of the ACE field, as well as any TCP option - including SACK in
>     > particular) should be accepted as DupACK under all circumstances?
>     >
>     > This confuses me even more...
>     >
>     >
>     > Sorry,
>     >   Richard
>     >
>

-- 
________________________________________________________________
Bob Briscoehttp://bobbriscoe.net/