[tcpm] Thoughts on Ack-on-ACK Re: draft-ietf-tcpm-accurate-ecn-24 addressingallWGLCcomments

[Yoshifumi Nishida <[e-mail]>]

Hi,
I just would like to check my understanding on ACK-on-ACK.
Here's a list of my thoughts. Please correct me if I am wrong...

* If an endpoint wants to receive ACK-on-ACK feedback through ECN++, it
should set ECT on their ACKs.
* This means If it doesn't want to receive ACK-on-ACK feedback, they can
disable it by not putting ECT on ACKs. So, if the peer doesn't support
SACK, it can control the peer to not send ACK-on-ACK feedback.
* It seems to me that dupacks caused by ACK-on-ACK feedback won't happen
so easily (although I don't say zero). Because in my understanding, 3
dupacks will be generated when all conditions below are met.
    1: an endpoint (say A) needs to receive at least 9 ACKs to generate 3
dupacks (because n=3). This means A needs to send 18 or greater packets in
the previous RTT.
    2: A doesn't have data to be sent in the current RTT otherwise A can
send ECN feedback with new data packets.
    3: During this RTT, the peer (say B) should have outstanding packets.
    4: These outstanding packets from B should not arrive A in this RTT
otherwise A can send ECN feedback with the ACKs for newly arrived data.

Based on this, I am thinking that the risk for ack-on-ack won't be high. An
important point is the endpoints which don't want to take a risk can always
disable features (also, accecn draft already clarifies ACK-on-ACK is an
experimental)

Thanks!
--
Yoshi


On Mon, Sep 25, 2023 at 2:02 AM Yoshifumi Nishida <[e-mail]>
wrote:

> Hi folks,
>
> I think that ECN++ draft explicitly mentions that ECN on pure ACK is
> prohibited without SACK negotiations. (Section 3.2.3.2)
> BTW, I am wondering if we could update the following sentence in the ECN++
> draft
>     "If there is no SACK option on the incoming pure ACK despite SACK
> having been negotiated, it is not a duplicate"¶
> <https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-generalized-ecn-12#section-3.3.3.1-2.1>
> by something like
>   "If there is no SACK option on the incoming pure ACK despite SACK having
> been negotiated, it is not a duplicated and MUST not be used for any packet
> loss or packet duplication detection algorithms."
> Wouldn't it solve the ambiguity on ack on ack at least to some extents?
>
> Thanks,
> --
> Yoshi
>
> On Fri, Sep 22, 2023 at 6:50 PM Markku Kojo <kojo=
> [e-mail]> wrote:
>
>> Hi Michael,
>>
>> Please see answers to your comments/questions inline (tagged [MK5]).
>>
>> Please consider this also as my indication for the 2nd WGLC that my
>> concerns/comments have not been addressed (I think, these remaining
>> concerns for 2nd WGLC are better discussed in this thread where they
>> have already benn discussed for some while).
>>
>> On Sun, 10 Sep 2023, [e-mail] wrote:
>>
>> > Hi Markku,
>> >
>> > some comments in-line.
>> >
>> > Best regards
>> > Michael (as an individual)
>> >
>> >> On 3. Aug 2023, at 09:04, Markku Kojo <kojo=
>> [e-mail]> wrote:
>> >>
>> >> Bob,
>> >>
>> >> TL;DR
>> >>
>> >> 1.  It seems that you missed the point with SACK/DSACK;
>> >>
>> >> 2. Even when SACK is in use there is no reliable way to distinguish
>> real DupAcks from Acks of Acks ('fake' DupAcks). Therefore, existing stds
>> track mechanism like RACK/TLP and F-RTO do no work correctly if one uses
>> the SACK rule in ECN++ draft for trying to distinguish real DupAcks from
>> Acks of Acks.
>> >>
>> >> Please see [MK4] below for details.
>> >>
>> >> On Thu, 27 Jul 2023, Bob Briscoe wrote:
>> >>
>> >>> Markku, pls see [BB4]
>> >>>
>> >>> On 27/07/2023 08:26, Markku Kojo wrote:
>> >>>> Bob,
>> >>>> thank you for resending this with another email address as my
>> regular address bounced (my account erreneously expired last Fri but is now
>> back and working).
>> >>>> Please see below tagged [MK3].
>> >>>> On Wed, 26 Jul 2023, Markku P I Kojo wrote:
>> >>>>>
>> ____________________________________________________________________________________________________________________________
>> From: Bob Briscoe <[e-mail]>
>> >>>>> Sent: 23 July 2023 00:01
>> >>>>> To: Kojo, Markku P I <[e-mail]>
>> >>>>> Subject: Fwd: draft-ietf-tcpm-accurate-ecn-24 addressing all WGLC
>> comments
>> >>>>> Retrying with an address which appears not to bounce.
>> >>>>> Pls add back mail distribution below in any reply.
>> >>>>> -------- Forwarded Message --------
>> >>>>> Subject:
>> >>>>> Re: draft-ietf-tcpm-accurate-ecn-24 addressing all WGLC comments
>> >>>>> Date:
>> >>>>> Sat, 22 Jul 2023 13:33:51 -0700
>> >>>>> From:
>> >>>>> Bob Briscoe <[e-mail]>
>> >>>>> To:
>> >>>>> Markku Kojo <[e-mail]>
>> >>>>> CC:
>> >>>>> [e-mail], Yoshifumi Nishida <[e-mail]>, Ian
>> Swett <[e-mail]>, [e-mail], Bob Briscoe
>> >>>>> [Apple] <[e-mail]>
>> >>>>> Markku,
>> >>>>> I'm top-posting 4 responses to all your points, in order to
>> consolidate everything and avoid all the repetition:
>> >>>>> #1) Generic should not be confused with experimental.
>> >>>>> #2) Completeness
>> >>>>> #3) Does not specify the necessary details?
>> >>>>> #4) Not generic enough?
>> >>>>> #1) GENERIC SHOULD NOT BE CONFUSED WITH EXPERIMENTAL.
>> >>>>> The AccECN spec is the IETF's PS spec for how TCP feeds back ECN on
>> any packet. The requirement for AccECN to provide
>> >>>>> forward compatibility by providing generic feedback was agreed by
>> this WG and set down in RFC7560 (nearly 8 years ago).
>> >>>>> Generic mechanistic reflection is a central design principle of
>> AccECN - to reflect the ECN field of SYNs, SYN-ACKs, ACKs,
>> >>>>> data packets, retransmissions, everything (§2.5 of the draft has
>> said this since its inception). AccECN provides accurate
>> >>>>> feedback irrespective of whether or how it is currently used, so
>> that it can be used if needed.
>> >>>>> Mechanistic reflection is generic, not experimental.
>> >>>>>    Analogy: Senders have conducted many CC experiments using
>> feedback from stds-track rcvrs. But that doesn't make the
>> >>>>> rcvrs experimental; rather, it proves the f/b is generic.
>> >>>> [MK3] All these CC experiments that I am aware of have used stds
>> track receiver feedback that had already existed for a long time before the
>> experiment and had been matured for years (decades). This draft intends to
>> introduce new behaviour (acking ACKs randomly in scale) for upcoming
>> experiments at the same time when the experiments are being specified. That
>> is very different because the new feature has not matured. Besides, the
>> Acks of Acks issue appeared only after the wg had decided this document to
>> become stds track.
>> >>>
>> >>> [BB4] I think the WG understands your point. Where we are talking
>> about possible undefined future problems, it is for the WG to decide
>> whether they want to proceed.
>> >>>
>> >>>>> You say an AccECN receiver automatically becomes part of an
>> experiment as soon as the its peer enables ECN-capable ACKs.
>> >>>>> More constructively, one can say that the peer conducts the
>> experiment by building on the generic behaviour of the receiver.
>> >>>>> The latter is more correct because the behaviour of the receiver
>> was already specified (even though it was not exercised);
>> >>>>> so it predated the experiment, and could be analysed and checked
>> prior to any experiments.
>> >>>> [MK3] It can be well phrases both ways. "it was not exercised" is
>> the key concern here. We do not know enough about potential problems; you
>> say it "could be analysed and checked prior to any experiments" but that
>> IMHO is not good enough to introduce such a new stds track behaviour. Even
>> though it was carefully analysed, the TCP Timestamp condition was not found
>> being incorrect, for example. This clearly shows how tricky issue we are
>> handling and I (and I guess anyone else either) cannot come up with all
>> potential problems easily. That's why we should be evry careful here.
>> >>>> Pls see also my reply to Richard where I explained how this feature
>> of Acks of Acks may become a problem much later (maybe after decade or
>> more), when AccECN possibly have been widely deployed as a stds track spec
>> but ECN++ somehow faded away. How is it possible/guaranteed that another
>> experiment that wants to enable pure Acks with non-SACK connections can get
>> rid of the Acks of Acks it does not need and all harm they produce?
>> >>>> They are coded and deployed all over and it would be hard to get rid
>> of them.
>> >>>
>> >>> [BB4] The code for acking ACKs will indeed be in stds track
>> implementations of AccECN, but it will not be exercised unless a peer is
>> running an experiment like ECN++. Any of the problems you mention can only
>> occur during one of those experiments, where they can also be resolved (or
>> the experiment ended).
>> >>
>> >> [MK4] You did not answer the question. You say that any of the
>> problems can also be resolved in the (upcoming) experiments. I disagree.
>> >> Could you please explain how would it be possible to resolve all the
>> problems that arise in an experiment where an endpoint want's to take
>> advantage of ECN-capable pure ACKs over a non-SACK TCP connection (for ACK
>> CC purposes, for example) but the other end implements the stds track
>> behaviour as currently specified in the draft?
>> >>
>> >> If the goal of AccECN is to provide "generic" feedback for any CC
>> purposes, I find it not very good/mature design if it excludes in advance
>> certain CC approaches and makes the protocol unusable without options
>> (e.g., non-SACK connections cannot leverage from ECN-capable pure ACKs).
>> >> IMO, even this is strong enough reason why I don't think that
>> including Acks of Acks in this stds track spec is a reasonable idea.
>> > So is your concern that TCP implementations which
>> > * DO enable ECN on pure ACKs and
>> > * DONT enable SACK
>> > have to deal with ambiguities?
>>
>> MK5] Not that they have to deal with ambiguities but that they cannot
>> operate correctly with the ambiguities generated by a generic reflector
>> that sends Acks of Acks as currently specified in this draft and that it
>> is directly in conflict with RFC 5681. The packet conservation principle
>> is the cornerstone of the current CC algos (particularly fast recovery
>> algoritms but also many other algos like Limited Transmit). These
>> algorithms rely on the packet concervation principle which the current
>> specifications ensure to work as expected. That is, an arriving Ack
>> indicates that a *data* packet (or data packets) have left the network,
>> including the case when the arriving Ack is dupAck. That is why RFC 5681
>> (Sec 2, last para) explicitly prohibits a receiver from generating more
>> than one ACK for every incoming [data] segment. If this rule is not
>> honored, all these current mechanism stop working correctly. That's why
>> it is "MUST NOT" in RFC 5681.
>> In other words, not because the implementations have to deal with
>> ambiguities but because their normal operation would get distorted.
>>
>> > If an implementation wants to use ECN on pure ACKs, it has to deal with
>> the ambiguity.
>>
>> [MK5] In general true but here this draft, which is intended to be stds
>> track, is
>> creating the problematic ambiguity by specifying experimental operation
>> with ECN-capable pure Acks (ECN++). That is, it specifies in Sec
>> 3.2.2.5.1 behaviour that is needed only when the other end is sending
>> ECN-capable pure Acks. The following sentence in the "Increment-Triggered
>> ACKs" rule:
>>
>>   "If there is no newly delivered data to acknowledge, 'n' SHOULD be 3 and
>>    MUST be no less than 3."
>>
>> is not needed with standards track TCP but only for experimentation.
>> Moreover, we have no real evidence that it is a useful feature. Instead,
>> if anyone implements this rule and follows the other stds track
>> specifications, it results in breaking important TCP behaviours like RTTM
>> computation with timestamps (as was already demonstrated during the
>> 1st WGLC discussions by one of the co-authors for RFC 7323).
>> Therefore, the rule that results in Acks of Acks is premature for stds
>> track and should be specified for experimental use only.
>>
>> > If it knows how the peer reacts, it can implement heuristics or enable
>> SACK.
>>
>> [MK5] I am afraid it cannot. The problem is that there seems to be no way
>> to implement any such heuristics. And, more importantly, even enabling
>> SACK does not solve all ambiguities: SACK may help in preventing false
>> fast retransmits from becoming triggered but it does not solve the
>> ambiguity that breaks RACK-TLP logic, F-RTO logic and DSACK-based
>> detection of spurious retransmissions [RFC 3708] (see more for your other
>> comment/question below).
>>
>> > How many stacks are out there not supporting TCP SACK? How many of them
>> will implement
>> > ECN++?
>>
>> [MK5] I think tens if not hundreds, when all various devices like network
>> printers and stacks in various IoT appliances, vehicles, etc are counted.
>> But as already stated, even SACK is not solving the ambiguity created
>> by Acks of Acks. Morover, we cannot know or estimate how many will
>> implement ECN++ and we need not to. TCP and its features as specified on
>> stds track are intended for generic use that any stack may implement. And
>> it is not the question about ECN++ being the only potential enabler for
>> ECN-capable pure ACKs; it has been proposed already before ECN++ and
>> there are quite likely more to come. Any stds track TCP feature should be
>> designed for various possible usages, not just for a single
>> experimentation like one intending to employ ECN++. For example, there
>> are a plenty of environments with asymmetric network capacity where one
>> would like to benefit from ECN-capable pure Acks for Ack CC only, for
>> example. For such use, Acks of Acks are useless, possibly even
>> detrimental.
>>
>> >>>>> This is protocol layering. Feedback is a 'sub-layer' of the
>> transport layer that lies logically beneath the congestion
>> >>>>> control function of the transport layer and is intended to allow
>> future evolution of sender behaviour. The RTCP protocol
>> >>>>> demonstrates this well, because it was specified separately from
>> RTP.
>> >>>>> You express concern that specifying a case where a stds track
>> AccECN implementation has to ack ACKs gives it no control over
>> >>>>> whether it is dragged into an experiment being conducted by its
>> peer.  But that's the deal for every generic capability. It
>> >>>>> gets used. Our task is to make sure that it can be used in this
>> way. That's why we have to correctly specify the SACK
>> >>>>> condition. That's what we do in the IETF.
>> >>>>> #2) COMPLETENESS
>> >>>>> A protocol spec should say what an implementation has to do in
>> response to all possible inputs. This ensures that there is
>> >>>>> just one behaviour to be checked for flaws (including security
>> flaws), rather than implementers inventing their own
>> >>>>> behaviours, each of which might lead to bugs or vulnerabilities if
>> it receives unexpected inputs.
>> >>>> [MK3] The incompleteness is exactly the problem that I have tried to
>> explain. See (*) under item #3 below.
>> >>>>> #3) DOES NOT SPECIFY THE NECESSARY DETAILS?
>> >>>>> You say this spec doesn't define the necessary safety features. We
>> can't work with "it needs a careful review for potential
>> >>>>> problems" then do nothing while we wait in case we might see
>> another email from you. If you think a safety feature is
>> >>>>> missing, the only way the WG works is constructively: participants
>> identify what's missing, and (ideally) define it.
>> >>>> [MK3] (*) My apologies if I have not been clear enough. What I have
>> tried to say is that the present spec does not specify how to ack an Ack
>> when SACK is enabled. I was now able interpret the intended way to ack an
>> Ack (I guessed this myself already earlier and you hinted it as well but I
>> was not sure because there were contradictory comments on this matter
>> earlier) from the most recent ECN++ spec which implies that an ECN++ peer
>> expects that an Ack of Ack does not carry SACK option:
>> >>>> "If there is no SACK option on the incoming pure ACK despite SACK
>> >>>>  having  been negotiated, it is not a duplicate;"
>> >>>> This way of acking, however, has not been specified in this spec.
>> Instead, you claim that this would be the case if the implementer follows
>> SACK (RFC 2018) and DSACK (RFC 2883). I disagee with this. Instead, the
>> implied way from ECN++ spec and the SACK spec (and DSACK) are in conflict.
>> >>>> An implementer who implements this spec following the rules in this
>> spec as well as SACK (RFC 2018) and DSACK (RFC 2883) has a serious dilemma
>> to solve which is contrary to what you claim in item #2 above.
>> >>>> Let's walk through what the implementer has at hand:
>> >>>> 1. When enough pure ACKs with CE have arrived, this spec requires
>> that the receiving end MUST send an ACK.
>> >>>> 2. The implementer reads SACK spec (RFC 2018) that requires that a
>> SACK block is included in the ACK in certain cases:
>> >>>> "SACK options SHOULD be included in all ACKs which do not ACK
>> >>>> the highest sequence number in the data receiver's queue."
>> >>>> 2. a) If there is no hole in the sequence space, everything is fine;
>> >>>>      No SACK block is included as per RFC 2018 and these Acks of
>> >>>>      Acks are not considered being DupAcks.
>> >>>> 2. b) If there is hole in the sequence space, RFC 2018 requires a
>> SACK
>> >>>>      block being included. The implementer includes a SACK block, but
>> >>>>      has a dilemma to solve: which SACK block to indicate. Whichever
>> >>>>      SACK block is included will result these Acks to be considered
>> >>>>      valid DupAcks as per ECN++ which they are not (they do not
>> >>>>      indicate an arrival of an out-of-order data packet but
>> incorrectly
>> >>>>      repeat some old SACK info. This is against the rule in RFC 5681
>> >>>>      that prohibits for a good reason to send more than one ACK per
>> >>>>      arriving data packet and will result in problems (see below).
>> >>>>      That is, this spec misses the rule for acking ACKs without the
>> >>>>      SACK option and indicating this is different from RFC 2018
>> >>>>      specification. And your completeness reuirement in item #2 is
>> >>>>      not fulfilled.
>> >>>
>> >>> [BB4] There is no dilemma. The receiver of the ACKs of ACKs would i)
>> have already entered loss recovery or, ii) if the ACK of the data was lost,
>> it would enter loss recovery on receipt of the ACKs of ACKs.
>> >>
>> >> [MK4] Of course the receiver of the ACKs of ACKs would either have
>> entered loss recovery or would enter loss recovery. Entering loss recovery
>> is not the problem but what happens during the loss recovery (fast
>> recovery). Pls see below.
>> >>
>> >>> RFC2018 allows no ambiguity on this. The state of the sender is
>> either still in loss recovery, when observing an
>> ACK-with-updated-ACE-plus-SACK, or it should enter loss recovery. There is
>> nothing extra needed or possible in the SACK RFCs.
>> >>
>> >> [MK4] Any SACK block included in the Ack of an Ack as the first SACK
>> block will incorrectly indicate arrival of a duplicate /data/ segment even
>> though no such data segment has arrived. This is forbidden both in RFC 5681
>> as well as in RFC 2883 as I have several times indicated. See more below
>> tagged (**).
>> >>
>> >>>
>> >>>> 3. If the implementation also implements DSACK, the implementer has
>> even
>> >>>>   harder dilemma to solve in case there is a hole in the sequence
>> space.
>> >>>>   RFC 2018 requires that a SACK block is included and DSACK specifies
>> >>>>   that the first SACK block reports "sequence of data received by the
>> >>>>   receiver in the most recent packet".
>> >>>>   Because the pure ACK does not include any data, what should the
>> >>>>   implementer do. For sure there will be various different
>> >>>>   interpretations because this has not been specified in this spec
>> >>>>   (and your item #2 is not fulfilled).
>> >>>>  a) An impelmenter may select a "random" block, maybe the most
>> >>>>     recently SACKED which is incorrect because it breaks DSACK idea
>> >>>>     and is in conflict with RFC 2883.
>> >>>>  b) Another implementer may be innovative and includes some "random"
>> >>>>     sequence number and includes a zero length SACK block with the
>> >>>>     same sequence number as the left and right edge. The consequences
>> >>>>     of this are unknown but certainly not correct.
>> >>>>  c) A more clever inplementer that reads the thoughts of authors
>> >>>>     for this spec (or has followed the discusions or even read ECN++)
>> >>>>     may send an ACK without any SACK info but this is in conflict
>> >>>>     RFC 2018.
>> >>>>  d) Maybe some other interpretation ...
>> >>>> You said earlier (below) that a) is the correct way with DSACK:
>> >>>> "[BB2] Surely those ACKs of ACKs from B to A are just further
>> >>>>  valid DupACKs with a SACK block included as defined in
>> >>>>  RFC2018, still cumulatively ack'ing packet 0 delivered
>> >>>>  before the loss."
>> >>>
>> >>> [BB4] The purpose of DSACK is to report the sequence numbers of the
>> packet that triggered the acknowledgement, which in these cases is a pure
>> ACK.
>> >>
>> >> [MK4] That is not quite correct. RFC 2018 specifies that the first
>> SACK block reports the sequence numbers of the packet that triggered the
>> acknowledgement. DSACK extends RFC 2018 by specifying and clarifying how an
>> arrival of a duplicate data segment is reported.
>> >>
>> >>> So, it just doesn't make sense to use DSACK on an ACK of an ACK at
>> all (I'll call this option e).
>> >>>
>> >>> We can say "no DSACK on these ACKs of ACKs" explicitly in the draft,
>> but I don't see how anyone would think  to use DSACK on one of these ACKs
>> of ACKs.
>> >>>
>> >>> BTW, I can't see how my words could have been interpreted as option
>> 'a' ('random').
>> >>
>> >> [MK4] I'm afraid you seem to make the same mistake as Richard did with
>> TCP timestamps; you suggested that "ACKs of ACKs from B to A are just
>> further valid DupACKs with a SACK block included as defined in RFC2018." As
>> I have pointed out earlier, RFC 2018 does not specify how to ack on arrival
>> of a pure ACK (nor does any other TCP spec). The rules in RFC 2018 are
>> correct only when acking data segments. However, you suggest following the
>> rules in RFC 2018 when acking a CE-marked pure ACK.
>> >> That is, if you include a SACK block as defined in RFC 2018, you will
>> include in the ACK of the ACK SACK blocks by repeating the most recently
>> reported SACK blocks. So, you explicitly suggest using DSACK on ACKs of
>> ACKs; the first SACK block would be incorrectly interpreted as DSACK by the
>> receiver of the ACK of the ACK even though no duplicate data segment never
>> arrived. The first SACK block (DSACK) will repeat the most recent SACK
>> block and is therefore a "random" block because one cannot know in advance
>> what is the latest data segment that arrived out-of-order. It depends on
>> recent packet dynamics of the connection that is more or less random. So,
>> if an author of the draft does not know how to ACK an ACK how would a
>> "random" implementer know if the #COMPLETENESS requirement you brought up
>> is not fulfilled?
>> >>
>> >> [MK4] RFC 5681 and RFC 2883 clearly forbid sending an ACK unless a
>> data segment has arrived. In other words, the rule "no DSACK on these ACKs
>> of ACKs" as you suggest is completely vague and unworkable.
>> >>
>> >> [MK4] The ECN++ rule does not work reliably.
>> >> The major remaining problem is the one you ignored in your recent
>> reply. The suggested rule in ECN++ says:  "If there is no SACK option on
>> the incoming pure ACK despite SACK
>> >>  having  been negotiated, it is not a duplicate;"
>> >> This rule does not reliably distinguish an ACK that indicates an
>> arrival of duplicate data segment from arrival of a CE-marked pure ACK.
>> Stds track specifications RACK/TLP (RFC (8985) and F-RTO (RFC 6582) do
>> handle arrival of a duplicate ACK (as per cumulate Ack in the Ack field)
>> without SACK option and with DSACK the same in certain cases to work
>> correctly.
>> >> These specs have been carefully specified to work correctly and they
>> specify their algos for completeness and the suggested ECN++ rule would
>> make these algos work incorrectly. Please see Sec. 7.4.2 for RACK and my
>> earlier ignored text just here below your previous signiture.
>> >> In addition, also the spurious retransmit detection on RFC 3708 seems
>> not to work correctly with the ECN++ rule even though the spec is a bit
>> vague for checking non-SACK DupAcks (Sec. 3, last para).
>> >>
>> >> Hope I was able the describe the problems clear enough this time.
>> > Would it help if the AccECN document would describe how an ACK for an
>> ACK looks like?
>>
>> [MK5] Sure this draft should describe how an Ack of an Ack is constructed
>> if it is sending such Acks. But it does not help because whichever way
>> you
>> construct the Ack of Ack, it will result in incorrect operation for these
>> widely deployed algos.
>>
>> > For example:
>> > Send the same ACK as the last one, except it contains a DSACK block, in
>> which case the DSACK block is removed.
>>
>> [MK5] Unfortunately I am not sure I am quite able follow what you mean by
>> removing DSACK block. The first SACK block is the DSACK block. You cannot
>> remove a DSACK block separately but the only way to remove DSACK is to
>> not include any SACK info. That would make the Ack of Ack to be a regular
>> dupAck (as per Ack field) and that would be interpreted by RACK-TLP,
>> F-RTO and RFC 3708 to indicate that a duplicate *data* segment had
>> arrived at the other end. Hence, all these algos are lead to make an
>> incorrect decision because no data segment had arrived at the other end
>> as I have already earlier explained. That is one reason why RFC 5681
>> prohibits sending a dupAck unless a data segment has arrived and
>> similarly RFC 2883 (Sec 4, item 2) allows only one DSACK per arriving
>> duplicate data segment.
>>
>> > And wouldn't it help, if the sender of the ACK of an ACK would include
>> a AccECN option?
>>
>> [MK5] Do you mean that an ACK of an ACK would always include an AccECN
>> option and genuine dupAck would never include the AccECN option?
>> That might help if there is never reordering. Otherwise, a regular
>> cumulative Ack cannot include the AccECN option either and that is
>> definitely no go.
>>
>> [MK5] But this is quite useless speculation, I think, because using the
>> AccECN option for distinquishing Acks of Acks from other Acks would
>> require that the AccECN option can always be used but the draft makes it
>> quite clear that this is not the case.
>>
>> Best regards,
>>
>> /Markku
>>
>> > Best regards
>> > Michael
>> >>
>> >> Thanks,
>> >>
>> >> /Markku
>> >>
>> >>> Bob
>> >>>
>> >>>> This would result in incorrect behaviour with DSACK and will confuse
>> any algo that depends on correct behavior of DSACK in correctly reporting
>> arrival of duplicate data segments, including RFC 3708 and (TLP of) RACK
>> (RFC 8985) as I have earlier indicated.
>> >>>> The intention of DSACK is to inform an arrival of duplicate data
>> segment and the spec explicitely prohibits repeating that info in the
>> spirit of the one ACK per data segment rule of RFC5681. RFC 2883, Sec 4,
>> item (2):
>> >>>> "Each duplicate contiguous sequence of data received is reported
>> >>>> in at most one D-SACK block.  (I.e., the receiver sends two identical
>> >>>> D-SACK blocks in subsequent packets only if the receiver receives two
>> >>>> duplicate segments.)"
>> >>>> Even if one selects the intended option (c above), it does not
>> necessarily result in correct outcome. SACK specification in RFC 3517 used
>> the same definition for DupAck as RFC 5681.
>> >>>> The DupAck definition in RFC 6675 holds for the purposes of RFC 6675
>> only as clearly stated in that specification. That is, it depends on an RFC
>> spec which definition of DupAck (RFC 5681 of RFC 6675) it adopts regardless
>> of whether the spec involves SACK or not. For example, F-RTO deliberately
>> uses the DupAck definition of RFC 5681 for both non-SACK version and the
>> SACK-enhanced version of F-RTO. In other words, the SACK-enhanced version
>> of F-RTO uses an arriving DupAck without SACK option (or DupAck with SACK
>> block but with no new SACK information) to declare RTO being not spurious
>> similar to the non-SACK version does (Sec 3.1, step 3 a). Therefore, an
>> arriving Ack of an Ack without SACK info incorrectly declares an RTO not
>> spurious even if it actually was spurious RTO.
>> >>>> (This needs a bit different scenario than what I draw in Fig 3 b) of
>> my scenarios that was for the non-SACK version of F-RTO)
>> >>>> To sum up: In order to partially solve the ambiguity on how to ack
>> ACKs and avoid triggering false fast retransmits, this spec must update RFC
>> 2018 by making an exception to the rule that specifies when a SACK block
>> should be included. However, this does not solve the entire problem with
>> Acks of Acks because there are algos like F-RTO that use the RFC 5681
>> definition of DupAck. Therefore, IMHO sending Acks of Acks should not be
>> included in a stds track spec.
>> >>>>> You repeatedly refer forward to apparent problems with SACK, which
>> sounds like you have identified that the protocol is
>> >>>>> broken. But, having read forward to the end, you agree that the
>> SACK condition is more solid than using TSOpt. And the only
>> >>>>> concrete criticism I can find is that we don't specify what DSACK
>> to send. Richard & I (and others) have worked through all
>> >>>>> the cases you have proposed (and others we have contrived
>> ourselves), and we cannot find any case where an implementation
>> >>>>> has to do anything different from what the DSACK RFCs already say.
>> >>>> [MK3] Please see the explanation above. Hope it clarifies the issue
>> now.
>> >>>>> I had independently found the rule you point out in RFC 5681, Sec
>> 2, against generating more than one ACK for every incoming
>> >>>>> [data] segment. We can say that the potential requirement to ack
>> ACKs contravenes that rule, which is why the SACK condition
>> >>>>> is necessary. I know you are concerned that it might not be as
>> simple as that. But, without evidence, how do we distinguish
>> >>>>> this from over-dramatisation? In the absence of anything specific,
>> we can't just sit on our hands. As Yoshi says, these will
>> >>>>> probably be corner cases. If new problems surface, we will have to
>> deal with them as they arise.
>> >>>> [MK3] Please see above. Definitely not a corner case.
>> >>>>> #4) NOT GENERIC ENOUGH?
>> >>>>> To your point about feedback from pure ACKs and data being
>> combined, that's not driven by any particular CCA. If ACKs are
>> >>>>> ECT, AccECN is designed to make it possible for the Data Sender to
>> estimate how much of the combined per-packet feedback is
>> >>>>> from ACKs and data, by using the per-byte feedback in the option to
>> estimate the number of data packets. This might not be
>> >>>>> perfect, but it was recognised from the start that we would need to
>> decide on compromises,, particularly given very limited
>> >>>>> header space.
>> >>>>> Nothing in the increment-triggered ACK rule was designed around
>> ECN++. Requiring the receiver to emit an ACK after 'n' CE
>> >>>>> marks is a fairly obvious way of  ensuring that the sender  gets
>> timely feedback of multiple CE markings.
>> >>>> [MK3] Please see my reply to Richard where I explained why this
>> feedback with Acks of Ack is in almost all scenarious unnecessary or not
>> timely. In a single case when the other end first is idle but starts
>> sending data it might be useful but even in such scenarious the feedback
>> often comes too late or possibly is stale (if there is longish delay before
>> the other end starts sending).
>> >>>>> 'n' has to be less
>> >>>>> than 7 to prevent AccECN's 3-bit ACE field wrapping. And greater
>> than 2 to avoid too many ACKs. Then, as I said, we noticed
>> >>>>> that could cause ping-pong, so we raised the min to 3 if the ACK
>> would be in response to an ACK. That would be necessary for
>> >>>>> protocol completeness (see above) whether or not an ECN++ draft
>> existed.
>> >>>> [MK3] Raising the min to 3 helps in dampening the ping-pong effect.
>> But still the ping-pong may last several RTTs if the cwnd is large and lot
>> of pkts (most) get CE marked. Not sending feedback for CE-marked DupAcks
>> (and not counting them in the counters) would be much more efficient way to
>> dampen the ping-pong effect. In addition, it would avoid making unnecessary
>> rate reductions in such cases (when DupAcks are arriving) where there will
>> be a notable rate reduction anyway (the increased Ack rate due to DupAcks
>> being sent for every data pkt goes away roughly in one RTT and the end
>> receiving DupAcks will react to congestion and reduce cwnd, resulting in
>> lower ack rate for the next RTTs). Pls see my reply to Richard.
>> >>>> This is another reason why this part of the rule is experimental in
>> my opinion: we don't have any evidence/experience what is the best way to
>> provide feedback/react to CE-marked pure Acks.
>> >>>> It would be useful if you could give one scenario where acking
>> CE-marked DupAcks would give some benefit that cannot be (easily)
>> solved(gained in some other means.
>> >>>>> You raise a concern that allowing Acks of Acks might preclude
>> future algos that rely on the absence of 'fake dupacks'.
>> >>>>> Indeed, you say that contravening the one ACK per data segment rule
>> of RFC5681 "will break all mechanisms that rely on the
>> >>>>> behavior specified in RFC 2883, including RFC 3708 and (TLP of)
>> RACK (RFC 8985) as well as any potential new and useful
>> >>>>> algos that rely on this rule." But I think this is all stated
>> rather over-dramatically, because you admit that the SACK
>> >>>>> condition allows an implementation to identify 'fake dupacks'. So
>> it can exclude them from all such present and future
>> >>>>> algos. Then this all becomes a non-issue.
>> >>>> [MK3] Please see above. My apologies if I was not clear enough for
>> this part. I intended to say that SACK would help in distinguishing "fake
>> dupacks" that will otherwise trigger a false fast rexmit. However, SACK
>> seemingly does not solve all problems (e.g., breaks f-RTO). Unfortunately I
>> have no time to pass through all potential algos that may break. Even less
>> time to consider all potential new algos that could count on the
>> fundamental rule for generating only one DupAck per arriving data segment.
>> I just try to point out the ones that I found being problematic. Even if
>> one existing and (widely) deployed algo gets broken, that should be strong
>> enough warning signal to not proceed as a stds track feature.
>> >>>>> _______________
>> >>>>> I haven't added any further discussion to the thread below (except
>> a couple of minor points). To check that the above 4
>> >>>>> points comprehensively address all your [MK2] points. I've tagged
>> each with [BB3] and pointer(s) to the relevant response(s)
>> >>>>> above.
>> >>>> [MK3] I hope the above clarifications help. I didn't add anything
>> below except one comment (taggd [MK3]). I may pass through the comments
>> below later and comment if need be.
>> >>>> Thanks,
>> >>>> /Markku
>> >>>>> On 11/07/2023 16:48, Markku Kojo wrote:
>> >>>>>      Bob, please see [MK2] inline.
>> >>>>>      And apologies for the long delay in replying, again.
>> >>>>>      On Mon, 5 Jun 2023, Bob Briscoe wrote:
>> >>>>>            Markku, pls see [BB2] inline
>> >>>>>            On 26/05/2023 22:56, Markku Kojo wrote:
>> >>>>>                  Bob,
>> >>>>>                  My apologies you had to wait for the scenarios as
>> it took much longer with my limited cycles
>> >>>>>            than I thought.
>> >>>>>                  Anyways, please see my reply to Richard, some
>> scenarios are also included there.
>> >>>>>                  To keep things easier, it might be good to try to
>> keep the discussion on Acks of Acks (mainly)
>> >>>>>            in the thread
>> >>>>>                  with my reply to Richard.
>> >>>>>                  However, see inline tagged [MK].
>> >>>>>                  On Wed, 24 May 2023, Bob Briscoe wrote:
>> >>>>>                        Markku,
>> >>>>>                        Sorry, it's taken a week to build a
>> comprehensive reply to this long email. See inline tagged [BB]...
>> >>>>>                        On 17/05/2023 12:24, Markku Kojo wrote:
>> >>>>>                              Hi Michael, all,
>> >>>>>                              On Sun, 14 May 2023,
>> [e-mail] wrote:
>> >>>>>                                          On 30. Mar 2023, at 16:53,
>> Bob Briscoe <[e-mail]>
>> >>>>>                                          wrote:
>> >>>>>                                          Michael, Yoshi, Ian (as
>> tcpm chairs),
>> >>>>>                                          To close off the WGLC, I
>> have just posted a new rev of
>> >>>>>                                          accurate-ecn. Hyperlinks
>> quoted at the end.
>> >>>>>                                          You will see the diff is
>> rather extensive. I won't give a
>> >>>>>                                          summary of all the diffs
>> like I usually do. Instead I can just
>> >>>>>                                          refer to the summary I
>> gave in the presentation on Monday:
>> >>>>>
>> https://datatracker.ietf.org/meeting/116/materials/slides-116-tcpm-draft-ietf-tcpm-accurate-ecn
>> >>>>>                                          Thank you again to the
>> people who reviewed this during the WGLC:
>> >>>>>                                          Michael Tüxen, Alex Burr,
>> Gorry Fairhurst and Markku Kojo.
>> >>>>>                                          All changes are editorial,
>> apart from removing the para about
>> >>>>>                                          not mistaking certain ACKs
>> of ACKs for DupACKs, which I will add
>> >>>>>                                          to a rev of the ECN++
>> draft, hopefully later this week.
>> >>>>>                                          On the list, we have seen
>> agreement from all the reviewers to
>> >>>>>                                          these changes, except no
>> response from Markku yet.
>> >>>>>                                          On Monday, I told Markku
>> that I would post the draft in a few
>> >>>>>                                          days, so everyone can see
>> the updates and diff.
>> >>>>>                                    Anyone having additional
>> comments? In particular Markku regarding loss
>> >>>>>                                    recovery?
>> >>>>>                              My apologies for being late with my
>> reply to the author's comments on my review
>> >>>>>            (I've
>> >>>>>                              been extremly busy with other issues
>> since the wg mtng in Yokohama, including the
>> >>>>>            rest
>> >>>>>                              of mtng week).
>> >>>>>                              I don't have much new comments but it
>> seems that my major concern regarding the
>> >>>>>            problem
>> >>>>>                              of sending ACKs of ACKs was not fully
>> understood.
>> >>>>>                              The first thing where I think I was
>> not quite clear is that the major problem with
>> >>>>>            ACKs
>> >>>>>                              of ACKs is not that a pure Ack is made
>> ECN-capable. Instead, the problem is in
>> >>>>>                              generating an Ack of an pure Ack and
>> that is what one should prohibit to avoid
>> >>>>>            problems.
>> >>>>>                              I understand that it might be
>> problematic to formulate rules whether generating an
>> >>>>>            Ack
>> >>>>>                              of an Ack is allowed (and when),
>> instead of just disabling sending ECN-capable
>> >>>>>            ACKs.
>> >>>>>                              I don't have a strong opinion which
>> way the problems with ACKs of ACKs is avoided
>> >>>>>            as
>> >>>>>                              long as they are avoided.
>> >>>>>                        [BB] See later after your similar point
>> (following your 'Why?' heading)...
>> >>>>>                              I am preparing a few scenarios to
>> illustriate the problems ACKs of ACks raise and
>> >>>>>            will
>> >>>>>                              send them shortly once I have
>> formulated a more thorough reasoning why sending
>> >>>>>            ACKs of
>> >>>>>                              ACKs is not really a good idea and
>> even seems to be unnecessary in most if not all
>> >>>>>                              cases, i.e., it just results in
>> sending unnecessary packets with not much useful
>> >>>>>            effect
>> >>>>>                              but creates a notable number of
>> problems.
>> >>>>>                        [BB] Having waited this long, it's rather
>> disappointing to still hear you say "I have an
>> >>>>>                        argument,
>> >>>>>                        but I'll tell you later."
>> >>>>>                  [MK] I understand. My sincere apologies again.
>> >>>>>                              Given that this draft is intended to
>> become a stds track RFC I am concerned of any
>> >>>>>            text
>> >>>>>                              in this document that indicates (or
>> even hints) that TCP could acknowledge pure
>> >>>>>            ACKS
>> >>>>>                              (this holds particularly the rules and
>> text in Sec 3.2.2.5.1 for the
>> >>>>>                              "Increment-Triggered ACKs"). If it is
>> seen necessary that this doc should have
>> >>>>>            such
>> >>>>>                              pieces of rules and text, I am fine if
>> any such text is moved to an appendix as
>> >>>>>            long as
>> >>>>>                              the appendix makes it cristal clear
>> that the text is valid only in case one is
>> >>>>>                              implementing an experiment as per
>> [I-D.ietf-tcpm-generalized-ecn].
>> >>>>>                        [BB] See point below about "Generic
>> (Mechanistic) Reflector".
>> >>>>>                              Why?
>> >>>>>                              1) It is well known that TCP does not
>> acknowledge ACKs and Standards track TCP has
>> >>>>>            not
>> >>>>>                              been specified to acknowledge ACKs.
>> This means that a reader/implementer of this
>> >>>>>            doc
>> >>>>>                              cannot correctly understand the rule
>> for "Increment-Triggered ACKs" unless there
>> >>>>>            is a
>> >>>>>                              normative reference to a spec that
>> specifies ACKs of ACKs (or tells that it is
>> >>>>>            even
>> >>>>>                              possible).
>> >>>>>                        [BB] ACKs of ACKs can indeed be tricky. But
>> there's no need to consider not ACKing ACKs
>> >>>>>            as an
>> >>>>>                        architectural principle. Not Acking ACKs on
>> principle certainly avoids some tricky
>> >>>>>            problems.
>> >>>>>                        However, we have a new situation here where,
>> in limited circumstances, ACKs of ACKs are
>> >>>>>                        necessary.
>> >>>>>                        So the WG has already worked through the
>> tricky problems and they have been addressed in
>> >>>>>            the
>> >>>>>                        draft
>> >>>>>                        (e.g. mistaking ACKs of ACKs for DupACks,
>> infinite ping-pong, etc). We'll discuss below
>> >>>>>            whether
>> >>>>>                        you've found some more trickiness.
>> >>>>>                  (MK] I did not mean to refer to any principle but,
>> as I said, that a reader/implementer cannot
>> >>>>>            correctly
>> >>>>>                  understand the rule for "Increment-Triggered ACKs"
>> because it is well-known to her/him that
>> >>>>>            TCP does not Ack
>> >>>>>                  ACKs. This fact is that one can ack ACKs is not
>> specified in this doc
>> >>>>>            [BB2] The fact that one /can/ ack ACKs is specified in
>> the AccECN draft. It clearly says:
>> >>>>>                "Although TCP normally only ACKs newly delivered
>> data, in this case it is mandatory for A to
>> >>>>>            emit ACKs of ACKs
>> >>>>>            because they feed back new congestion state (useful in
>> case B starts sending)."
>> >>>>>            We'll also make it clearer that "in this case" means "in
>> the case of ECN-capable ACKs". And we'll
>> >>>>>            make it clear that it's
>> >>>>>            conditional on carefully following all the conditions in
>> any draft that specifies ECN-capable ACKs.
>> >>>>>      [MK2] Right, but this (acking Acks) is experimental behaviour,
>> needed only for those participating the
>> >>>>>      experiment. However, this doc specifies the behaviour in the
>> Increment-Triggered ACKs rule: MUST emit an ACK".
>> >>>>>      If one implements this draft but is not interested in ECN++
>> nor any other similar experiment that allows
>> >>>>>      ECN-capable ACKs, one must still implement these Acks of Acks
>> but there is no specification nor normative ref
>> >>>>>      how to do it (correctly, including all the conditions). If
>> then this implementation is deployed in the wild, it
>> >>>>>      automatically becomes part of the experiment once the other
>> endpoint enables ECN-capable ACKs but without the
>> >>>>>      necessary safety procedures. This doc cannot depend on an
>> experimental spec which explains how to do it and what
>> >>>>>      conditions to follow.
>> >>>>> [BB3] See
>> >>>>> #1) Generic should not be confused with experimental.
>> >>>>> #2) Completeness
>> >>>>> #3) Does not specify the necessary details?
>> >>>>>                  nor does this doc give a (normative) reference
>> where it is specified,
>> >>>>>            [BB2] As already explained, there is no normative ref
>> where acking ACKs is specified in general,
>> >>>>>            because they are not
>> >>>>>            needed in general. They are specified whenever they are
>> needed, including in the examples Yoshi
>> >>>>>            pointed to (keepalive,
>> >>>>>            etc).
>> >>>>>      [MK2] An implementation of this draft send Acks of Acks
>> "randomly" and possibly multiple of them in a row but
>> >>>>>      this draft does not specify the necessary details unlike you
>> claim above.
>> >>>>> [BB3] See #4) Does not specify the necessary details
>> >>>>>      [MK2] Pls see also my reply to Yoshi. The specific examples
>> Yoshi mentions are sent in very limited cases and
>> >>>>>      result in sending only a single Ack of an Ack which is not
>> causing any harm to others unlike sending multiple
>> >>>>>      "random" Acks of Acks.
>> >>>>>                  including the details on which TSecr value to add
>> or which SACK info if any to include when
>> >>>>>            acking a pure
>> >>>>>                  Ack.
>> >>>>>            [BB2] OK.  But I've had to read all the through SACK and
>> TS specs to guess what complication you
>> >>>>>            might have seen. If you
>> >>>>>            know what made you ask this question, it would be much
>> easier if you gave us a clue, or even just
>> >>>>>            told us explicitly.
>> >>>>>      [MK2] Sorry for not being clear/detailed enough. Maybe this
>> was too obvious to me. See details below for SACK
>> >>>>>      (and my reply to Richard for Timestamps for which I believe
>> you already figured out the problem).
>> >>>>> [BB3] See #3) Does not specify the necessary details?
>> >>>>>            * For TS, I'm pretty sure the normal rules (for which
>> TSecr value to include) apply.
>> >>>>>            * Similarly, for SACK, I can't see that anything new
>> needs to be said.
>> >>>>>                  It is easy to misinterpret the
>> "Increment-Triggered ACKs", if one doesn't realize that pure
>> >>>>>            Acks may be
>> >>>>>                  acked.
>> >>>>>            [BB2] We'll make it clearer in the rule that 'packet'
>> intentionally includes packets without data.
>> >>>>>      [MK2] But as I said earlier, it would specify experimental
>> behavior in a PS doc.
>> >>>>> [BB3]  See: #1) Generic should not be confused with experimental.
>> >>>>>                        What is the new situation?
>> >>>>>                         *  Until ECN was introduced, TCP ACKs only
>> acknowledged data. So there was no need to
>> >>>>>                        acknowledge
>> >>>>>                            pure ACKs, which contain no data.
>> >>>>>                         *  When ECN was introduced in RFC3168, TCP
>> ACKs also acknowledged ECN markings.
>> >>>>>            However, because
>> >>>>>                            RFC3168 precluded pure ACKs from being
>> ECN-capable, there was still no need to
>> >>>>>            acknowledge
>> >>>>>                        pure
>> >>>>>                            ACKs.
>> >>>>>                         *  RFC5690, and now the ECN++ draft
>> introduce the possibility of ECN-capable pure ACKs.
>> >>>>>            So, in
>> >>>>>                        the
>> >>>>>                            limited circumstances described in the
>> AccECN draft, ECN-capable pure ACKs now need
>> >>>>>            to be
>> >>>>>                            acknowledged, because they contain new
>> information - their ECN field.
>> >>>>>                        Similarly, even though the final ACK of
>> TCP's 3WHS is an ACK of an ACK , it is sent
>> >>>>>            because it is
>> >>>>>                        needed (to prove that the SYN wasn't from a
>> spoofed address).
>> >>>>>                  [MK] All otherwise clear, but I disagree that the
>> final ACK of TCP's 3WHS is an ACK of an ACK.
>> >>>>>            It is required
>> >>>>>                  because SYNACK contains control data that eats one
>> sequence number, i.e., it advances RCV.NXT
>> >>>>>            at the client
>> >>>>>                  end and when the ACK arrives at the server it is
>> needed to advance SND.UNA. Very different
>> >>>>>            from Acks of Acks
>> >>>>>                  in this draft.
>> >>>>>            [BB2] Yes, not the same, but similar - ECN information
>> is also control data. But agreed that it
>> >>>>>            doesn't consume a
>> >>>>>            sequence no. Neither do keepalive probes or their ACKs.
>> >>>>>      [MK2] Right, but the actual purpose of Acks is to acknowledge
>> sequence numbers (that have arrived) and the logic
>> >>>>>      of many crucial TCP algos depend on how packets are acked.
>> >>>>> [BB3] Good protocol specs never define or limit the uses of the
>> protocol (it would be pointless to try anyway).
>> >>>>> If you only wanted ACKs to be used to acknowledge sequence numbers,
>> you should have objected to RFC3168, which used ACKs to
>> >>>>> feed back ECN as well. Or you could go further back and object to
>> SACK because it acknowledges gaps, not sequence numbers
>> >>>>> that have arrived.
>> >>>> [MK3] With both RFC 3168 and SACK the ACKS ack the sequence numbers
>> in the first place. These specs just piggyback additional information on
>> ACKS that would be sent anyway. They do not introduce extra pkts to be sent
>> like this spec does. I think this makes a major difference.
>> >>>> [snipped the rest of msg as the msg was too long for the wg list]
>> >>>
>> >>> --
>> >>> ________________________________________________________________
>> >>> Bob Briscoe                               http://bobbriscoe.net/
>> >>>
>> >> _______________________________________________
>> >> tcpm mailing list
>> >> [e-mail]
>> >> https://www.ietf.org/mailman/listinfo/tcpm
>> >
>> >_______________________________________________
>> tcpm mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/tcpm
>>
>