Re: [tcpm] draft-ietf-tcpm-accurate-ecn-24 addressingallWGLCcomments

[Yoshifumi Nishida <[e-mail]>]

Hi folks,

I think that ECN++ draft explicitly mentions that ECN on pure ACK is
prohibited without SACK negotiations. (Section 3.2.3.2)
BTW, I am wondering if we could update the following sentence in the ECN++
draft
    "If there is no SACK option on the incoming pure ACK despite SACK
having been negotiated, it is not a duplicate"¶
<https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-generalized-ecn-12#section-3.3.3.1-2.1>
by something like
  "If there is no SACK option on the incoming pure ACK despite SACK having
been negotiated, it is not a duplicated and MUST not be used for any packet
loss or packet duplication detection algorithms."
Wouldn't it solve the ambiguity on ack on ack at least to some extents?

Thanks,
--
Yoshi

On Fri, Sep 22, 2023 at 6:50 PM Markku Kojo <kojo=
[e-mail]> wrote:

> Hi Michael,
>
> Please see answers to your comments/questions inline (tagged [MK5]).
>
> Please consider this also as my indication for the 2nd WGLC that my
> concerns/comments have not been addressed (I think, these remaining
> concerns for 2nd WGLC are better discussed in this thread where they
> have already benn discussed for some while).
>
> On Sun, 10 Sep 2023, [e-mail] wrote:
>
> > Hi Markku,
> >
> > some comments in-line.
> >
> > Best regards
> > Michael (as an individual)
> >
> >> On 3. Aug 2023, at 09:04, Markku Kojo <kojo=
> [e-mail]> wrote:
> >>
> >> Bob,
> >>
> >> TL;DR
> >>
> >> 1.  It seems that you missed the point with SACK/DSACK;
> >>
> >> 2. Even when SACK is in use there is no reliable way to distinguish
> real DupAcks from Acks of Acks ('fake' DupAcks). Therefore, existing stds
> track mechanism like RACK/TLP and F-RTO do no work correctly if one uses
> the SACK rule in ECN++ draft for trying to distinguish real DupAcks from
> Acks of Acks.
> >>
> >> Please see [MK4] below for details.
> >>
> >> On Thu, 27 Jul 2023, Bob Briscoe wrote:
> >>
> >>> Markku, pls see [BB4]
> >>>
> >>> On 27/07/2023 08:26, Markku Kojo wrote:
> >>>> Bob,
> >>>> thank you for resending this with another email address as my regular
> address bounced (my account erreneously expired last Fri but is now back
> and working).
> >>>> Please see below tagged [MK3].
> >>>> On Wed, 26 Jul 2023, Markku P I Kojo wrote:
> >>>>>
> ____________________________________________________________________________________________________________________________
> From: Bob Briscoe <[e-mail]>
> >>>>> Sent: 23 July 2023 00:01
> >>>>> To: Kojo, Markku P I <[e-mail]>
> >>>>> Subject: Fwd: draft-ietf-tcpm-accurate-ecn-24 addressing all WGLC
> comments
> >>>>> Retrying with an address which appears not to bounce.
> >>>>> Pls add back mail distribution below in any reply.
> >>>>> -------- Forwarded Message --------
> >>>>> Subject:
> >>>>> Re: draft-ietf-tcpm-accurate-ecn-24 addressing all WGLC comments
> >>>>> Date:
> >>>>> Sat, 22 Jul 2023 13:33:51 -0700
> >>>>> From:
> >>>>> Bob Briscoe <[e-mail]>
> >>>>> To:
> >>>>> Markku Kojo <[e-mail]>
> >>>>> CC:
> >>>>> [e-mail], Yoshifumi Nishida <[e-mail]>, Ian
> Swett <[e-mail]>, [e-mail], Bob Briscoe
> >>>>> [Apple] <[e-mail]>
> >>>>> Markku,
> >>>>> I'm top-posting 4 responses to all your points, in order to
> consolidate everything and avoid all the repetition:
> >>>>> #1) Generic should not be confused with experimental.
> >>>>> #2) Completeness
> >>>>> #3) Does not specify the necessary details?
> >>>>> #4) Not generic enough?
> >>>>> #1) GENERIC SHOULD NOT BE CONFUSED WITH EXPERIMENTAL.
> >>>>> The AccECN spec is the IETF's PS spec for how TCP feeds back ECN on
> any packet. The requirement for AccECN to provide
> >>>>> forward compatibility by providing generic feedback was agreed by
> this WG and set down in RFC7560 (nearly 8 years ago).
> >>>>> Generic mechanistic reflection is a central design principle of
> AccECN - to reflect the ECN field of SYNs, SYN-ACKs, ACKs,
> >>>>> data packets, retransmissions, everything (§2.5 of the draft has
> said this since its inception). AccECN provides accurate
> >>>>> feedback irrespective of whether or how it is currently used, so
> that it can be used if needed.
> >>>>> Mechanistic reflection is generic, not experimental.
> >>>>>    Analogy: Senders have conducted many CC experiments using
> feedback from stds-track rcvrs. But that doesn't make the
> >>>>> rcvrs experimental; rather, it proves the f/b is generic.
> >>>> [MK3] All these CC experiments that I am aware of have used stds
> track receiver feedback that had already existed for a long time before the
> experiment and had been matured for years (decades). This draft intends to
> introduce new behaviour (acking ACKs randomly in scale) for upcoming
> experiments at the same time when the experiments are being specified. That
> is very different because the new feature has not matured. Besides, the
> Acks of Acks issue appeared only after the wg had decided this document to
> become stds track.
> >>>
> >>> [BB4] I think the WG understands your point. Where we are talking
> about possible undefined future problems, it is for the WG to decide
> whether they want to proceed.
> >>>
> >>>>> You say an AccECN receiver automatically becomes part of an
> experiment as soon as the its peer enables ECN-capable ACKs.
> >>>>> More constructively, one can say that the peer conducts the
> experiment by building on the generic behaviour of the receiver.
> >>>>> The latter is more correct because the behaviour of the receiver was
> already specified (even though it was not exercised);
> >>>>> so it predated the experiment, and could be analysed and checked
> prior to any experiments.
> >>>> [MK3] It can be well phrases both ways. "it was not exercised" is the
> key concern here. We do not know enough about potential problems; you say
> it "could be analysed and checked prior to any experiments" but that IMHO
> is not good enough to introduce such a new stds track behaviour. Even
> though it was carefully analysed, the TCP Timestamp condition was not found
> being incorrect, for example. This clearly shows how tricky issue we are
> handling and I (and I guess anyone else either) cannot come up with all
> potential problems easily. That's why we should be evry careful here.
> >>>> Pls see also my reply to Richard where I explained how this feature
> of Acks of Acks may become a problem much later (maybe after decade or
> more), when AccECN possibly have been widely deployed as a stds track spec
> but ECN++ somehow faded away. How is it possible/guaranteed that another
> experiment that wants to enable pure Acks with non-SACK connections can get
> rid of the Acks of Acks it does not need and all harm they produce?
> >>>> They are coded and deployed all over and it would be hard to get rid
> of them.
> >>>
> >>> [BB4] The code for acking ACKs will indeed be in stds track
> implementations of AccECN, but it will not be exercised unless a peer is
> running an experiment like ECN++. Any of the problems you mention can only
> occur during one of those experiments, where they can also be resolved (or
> the experiment ended).
> >>
> >> [MK4] You did not answer the question. You say that any of the problems
> can also be resolved in the (upcoming) experiments. I disagree.
> >> Could you please explain how would it be possible to resolve all the
> problems that arise in an experiment where an endpoint want's to take
> advantage of ECN-capable pure ACKs over a non-SACK TCP connection (for ACK
> CC purposes, for example) but the other end implements the stds track
> behaviour as currently specified in the draft?
> >>
> >> If the goal of AccECN is to provide "generic" feedback for any CC
> purposes, I find it not very good/mature design if it excludes in advance
> certain CC approaches and makes the protocol unusable without options
> (e.g., non-SACK connections cannot leverage from ECN-capable pure ACKs).
> >> IMO, even this is strong enough reason why I don't think that including
> Acks of Acks in this stds track spec is a reasonable idea.
> > So is your concern that TCP implementations which
> > * DO enable ECN on pure ACKs and
> > * DONT enable SACK
> > have to deal with ambiguities?
>
> MK5] Not that they have to deal with ambiguities but that they cannot
> operate correctly with the ambiguities generated by a generic reflector
> that sends Acks of Acks as currently specified in this draft and that it
> is directly in conflict with RFC 5681. The packet conservation principle
> is the cornerstone of the current CC algos (particularly fast recovery
> algoritms but also many other algos like Limited Transmit). These
> algorithms rely on the packet concervation principle which the current
> specifications ensure to work as expected. That is, an arriving Ack
> indicates that a *data* packet (or data packets) have left the network,
> including the case when the arriving Ack is dupAck. That is why RFC 5681
> (Sec 2, last para) explicitly prohibits a receiver from generating more
> than one ACK for every incoming [data] segment. If this rule is not
> honored, all these current mechanism stop working correctly. That's why
> it is "MUST NOT" in RFC 5681.
> In other words, not because the implementations have to deal with
> ambiguities but because their normal operation would get distorted.
>
> > If an implementation wants to use ECN on pure ACKs, it has to deal with
> the ambiguity.
>
> [MK5] In general true but here this draft, which is intended to be stds
> track, is
> creating the problematic ambiguity by specifying experimental operation
> with ECN-capable pure Acks (ECN++). That is, it specifies in Sec
> 3.2.2.5.1 behaviour that is needed only when the other end is sending
> ECN-capable pure Acks. The following sentence in the "Increment-Triggered
> ACKs" rule:
>
>   "If there is no newly delivered data to acknowledge, 'n' SHOULD be 3 and
>    MUST be no less than 3."
>
> is not needed with standards track TCP but only for experimentation.
> Moreover, we have no real evidence that it is a useful feature. Instead,
> if anyone implements this rule and follows the other stds track
> specifications, it results in breaking important TCP behaviours like RTTM
> computation with timestamps (as was already demonstrated during the
> 1st WGLC discussions by one of the co-authors for RFC 7323).
> Therefore, the rule that results in Acks of Acks is premature for stds
> track and should be specified for experimental use only.
>
> > If it knows how the peer reacts, it can implement heuristics or enable
> SACK.
>
> [MK5] I am afraid it cannot. The problem is that there seems to be no way
> to implement any such heuristics. And, more importantly, even enabling
> SACK does not solve all ambiguities: SACK may help in preventing false
> fast retransmits from becoming triggered but it does not solve the
> ambiguity that breaks RACK-TLP logic, F-RTO logic and DSACK-based
> detection of spurious retransmissions [RFC 3708] (see more for your other
> comment/question below).
>
> > How many stacks are out there not supporting TCP SACK? How many of them
> will implement
> > ECN++?
>
> [MK5] I think tens if not hundreds, when all various devices like network
> printers and stacks in various IoT appliances, vehicles, etc are counted.
> But as already stated, even SACK is not solving the ambiguity created
> by Acks of Acks. Morover, we cannot know or estimate how many will
> implement ECN++ and we need not to. TCP and its features as specified on
> stds track are intended for generic use that any stack may implement. And
> it is not the question about ECN++ being the only potential enabler for
> ECN-capable pure ACKs; it has been proposed already before ECN++ and
> there are quite likely more to come. Any stds track TCP feature should be
> designed for various possible usages, not just for a single
> experimentation like one intending to employ ECN++. For example, there
> are a plenty of environments with asymmetric network capacity where one
> would like to benefit from ECN-capable pure Acks for Ack CC only, for
> example. For such use, Acks of Acks are useless, possibly even
> detrimental.
>
> >>>>> This is protocol layering. Feedback is a 'sub-layer' of the
> transport layer that lies logically beneath the congestion
> >>>>> control function of the transport layer and is intended to allow
> future evolution of sender behaviour. The RTCP protocol
> >>>>> demonstrates this well, because it was specified separately from RTP.
> >>>>> You express concern that specifying a case where a stds track AccECN
> implementation has to ack ACKs gives it no control over
> >>>>> whether it is dragged into an experiment being conducted by its
> peer.  But that's the deal for every generic capability. It
> >>>>> gets used. Our task is to make sure that it can be used in this way.
> That's why we have to correctly specify the SACK
> >>>>> condition. That's what we do in the IETF.
> >>>>> #2) COMPLETENESS
> >>>>> A protocol spec should say what an implementation has to do in
> response to all possible inputs. This ensures that there is
> >>>>> just one behaviour to be checked for flaws (including security
> flaws), rather than implementers inventing their own
> >>>>> behaviours, each of which might lead to bugs or vulnerabilities if
> it receives unexpected inputs.
> >>>> [MK3] The incompleteness is exactly the problem that I have tried to
> explain. See (*) under item #3 below.
> >>>>> #3) DOES NOT SPECIFY THE NECESSARY DETAILS?
> >>>>> You say this spec doesn't define the necessary safety features. We
> can't work with "it needs a careful review for potential
> >>>>> problems" then do nothing while we wait in case we might see another
> email from you. If you think a safety feature is
> >>>>> missing, the only way the WG works is constructively: participants
> identify what's missing, and (ideally) define it.
> >>>> [MK3] (*) My apologies if I have not been clear enough. What I have
> tried to say is that the present spec does not specify how to ack an Ack
> when SACK is enabled. I was now able interpret the intended way to ack an
> Ack (I guessed this myself already earlier and you hinted it as well but I
> was not sure because there were contradictory comments on this matter
> earlier) from the most recent ECN++ spec which implies that an ECN++ peer
> expects that an Ack of Ack does not carry SACK option:
> >>>> "If there is no SACK option on the incoming pure ACK despite SACK
> >>>>  having  been negotiated, it is not a duplicate;"
> >>>> This way of acking, however, has not been specified in this spec.
> Instead, you claim that this would be the case if the implementer follows
> SACK (RFC 2018) and DSACK (RFC 2883). I disagee with this. Instead, the
> implied way from ECN++ spec and the SACK spec (and DSACK) are in conflict.
> >>>> An implementer who implements this spec following the rules in this
> spec as well as SACK (RFC 2018) and DSACK (RFC 2883) has a serious dilemma
> to solve which is contrary to what you claim in item #2 above.
> >>>> Let's walk through what the implementer has at hand:
> >>>> 1. When enough pure ACKs with CE have arrived, this spec requires
> that the receiving end MUST send an ACK.
> >>>> 2. The implementer reads SACK spec (RFC 2018) that requires that a
> SACK block is included in the ACK in certain cases:
> >>>> "SACK options SHOULD be included in all ACKs which do not ACK
> >>>> the highest sequence number in the data receiver's queue."
> >>>> 2. a) If there is no hole in the sequence space, everything is fine;
> >>>>      No SACK block is included as per RFC 2018 and these Acks of
> >>>>      Acks are not considered being DupAcks.
> >>>> 2. b) If there is hole in the sequence space, RFC 2018 requires a SACK
> >>>>      block being included. The implementer includes a SACK block, but
> >>>>      has a dilemma to solve: which SACK block to indicate. Whichever
> >>>>      SACK block is included will result these Acks to be considered
> >>>>      valid DupAcks as per ECN++ which they are not (they do not
> >>>>      indicate an arrival of an out-of-order data packet but
> incorrectly
> >>>>      repeat some old SACK info. This is against the rule in RFC 5681
> >>>>      that prohibits for a good reason to send more than one ACK per
> >>>>      arriving data packet and will result in problems (see below).
> >>>>      That is, this spec misses the rule for acking ACKs without the
> >>>>      SACK option and indicating this is different from RFC 2018
> >>>>      specification. And your completeness reuirement in item #2 is
> >>>>      not fulfilled.
> >>>
> >>> [BB4] There is no dilemma. The receiver of the ACKs of ACKs would i)
> have already entered loss recovery or, ii) if the ACK of the data was lost,
> it would enter loss recovery on receipt of the ACKs of ACKs.
> >>
> >> [MK4] Of course the receiver of the ACKs of ACKs would either have
> entered loss recovery or would enter loss recovery. Entering loss recovery
> is not the problem but what happens during the loss recovery (fast
> recovery). Pls see below.
> >>
> >>> RFC2018 allows no ambiguity on this. The state of the sender is either
> still in loss recovery, when observing an ACK-with-updated-ACE-plus-SACK,
> or it should enter loss recovery. There is nothing extra needed or possible
> in the SACK RFCs.
> >>
> >> [MK4] Any SACK block included in the Ack of an Ack as the first SACK
> block will incorrectly indicate arrival of a duplicate /data/ segment even
> though no such data segment has arrived. This is forbidden both in RFC 5681
> as well as in RFC 2883 as I have several times indicated. See more below
> tagged (**).
> >>
> >>>
> >>>> 3. If the implementation also implements DSACK, the implementer has
> even
> >>>>   harder dilemma to solve in case there is a hole in the sequence
> space.
> >>>>   RFC 2018 requires that a SACK block is included and DSACK specifies
> >>>>   that the first SACK block reports "sequence of data received by the
> >>>>   receiver in the most recent packet".
> >>>>   Because the pure ACK does not include any data, what should the
> >>>>   implementer do. For sure there will be various different
> >>>>   interpretations because this has not been specified in this spec
> >>>>   (and your item #2 is not fulfilled).
> >>>>  a) An impelmenter may select a "random" block, maybe the most
> >>>>     recently SACKED which is incorrect because it breaks DSACK idea
> >>>>     and is in conflict with RFC 2883.
> >>>>  b) Another implementer may be innovative and includes some "random"
> >>>>     sequence number and includes a zero length SACK block with the
> >>>>     same sequence number as the left and right edge. The consequences
> >>>>     of this are unknown but certainly not correct.
> >>>>  c) A more clever inplementer that reads the thoughts of authors
> >>>>     for this spec (or has followed the discusions or even read ECN++)
> >>>>     may send an ACK without any SACK info but this is in conflict
> >>>>     RFC 2018.
> >>>>  d) Maybe some other interpretation ...
> >>>> You said earlier (below) that a) is the correct way with DSACK:
> >>>> "[BB2] Surely those ACKs of ACKs from B to A are just further
> >>>>  valid DupACKs with a SACK block included as defined in
> >>>>  RFC2018, still cumulatively ack'ing packet 0 delivered
> >>>>  before the loss."
> >>>
> >>> [BB4] The purpose of DSACK is to report the sequence numbers of the
> packet that triggered the acknowledgement, which in these cases is a pure
> ACK.
> >>
> >> [MK4] That is not quite correct. RFC 2018 specifies that the first SACK
> block reports the sequence numbers of the packet that triggered the
> acknowledgement. DSACK extends RFC 2018 by specifying and clarifying how an
> arrival of a duplicate data segment is reported.
> >>
> >>> So, it just doesn't make sense to use DSACK on an ACK of an ACK at all
> (I'll call this option e).
> >>>
> >>> We can say "no DSACK on these ACKs of ACKs" explicitly in the draft,
> but I don't see how anyone would think  to use DSACK on one of these ACKs
> of ACKs.
> >>>
> >>> BTW, I can't see how my words could have been interpreted as option
> 'a' ('random').
> >>
> >> [MK4] I'm afraid you seem to make the same mistake as Richard did with
> TCP timestamps; you suggested that "ACKs of ACKs from B to A are just
> further valid DupACKs with a SACK block included as defined in RFC2018." As
> I have pointed out earlier, RFC 2018 does not specify how to ack on arrival
> of a pure ACK (nor does any other TCP spec). The rules in RFC 2018 are
> correct only when acking data segments. However, you suggest following the
> rules in RFC 2018 when acking a CE-marked pure ACK.
> >> That is, if you include a SACK block as defined in RFC 2018, you will
> include in the ACK of the ACK SACK blocks by repeating the most recently
> reported SACK blocks. So, you explicitly suggest using DSACK on ACKs of
> ACKs; the first SACK block would be incorrectly interpreted as DSACK by the
> receiver of the ACK of the ACK even though no duplicate data segment never
> arrived. The first SACK block (DSACK) will repeat the most recent SACK
> block and is therefore a "random" block because one cannot know in advance
> what is the latest data segment that arrived out-of-order. It depends on
> recent packet dynamics of the connection that is more or less random. So,
> if an author of the draft does not know how to ACK an ACK how would a
> "random" implementer know if the #COMPLETENESS requirement you brought up
> is not fulfilled?
> >>
> >> [MK4] RFC 5681 and RFC 2883 clearly forbid sending an ACK unless a data
> segment has arrived. In other words, the rule "no DSACK on these ACKs of
> ACKs" as you suggest is completely vague and unworkable.
> >>
> >> [MK4] The ECN++ rule does not work reliably.
> >> The major remaining problem is the one you ignored in your recent
> reply. The suggested rule in ECN++ says:  "If there is no SACK option on
> the incoming pure ACK despite SACK
> >>  having  been negotiated, it is not a duplicate;"
> >> This rule does not reliably distinguish an ACK that indicates an
> arrival of duplicate data segment from arrival of a CE-marked pure ACK.
> Stds track specifications RACK/TLP (RFC (8985) and F-RTO (RFC 6582) do
> handle arrival of a duplicate ACK (as per cumulate Ack in the Ack field)
> without SACK option and with DSACK the same in certain cases to work
> correctly.
> >> These specs have been carefully specified to work correctly and they
> specify their algos for completeness and the suggested ECN++ rule would
> make these algos work incorrectly. Please see Sec. 7.4.2 for RACK and my
> earlier ignored text just here below your previous signiture.
> >> In addition, also the spurious retransmit detection on RFC 3708 seems
> not to work correctly with the ECN++ rule even though the spec is a bit
> vague for checking non-SACK DupAcks (Sec. 3, last para).
> >>
> >> Hope I was able the describe the problems clear enough this time.
> > Would it help if the AccECN document would describe how an ACK for an
> ACK looks like?
>
> [MK5] Sure this draft should describe how an Ack of an Ack is constructed
> if it is sending such Acks. But it does not help because whichever way you
> construct the Ack of Ack, it will result in incorrect operation for these
> widely deployed algos.
>
> > For example:
> > Send the same ACK as the last one, except it contains a DSACK block, in
> which case the DSACK block is removed.
>
> [MK5] Unfortunately I am not sure I am quite able follow what you mean by
> removing DSACK block. The first SACK block is the DSACK block. You cannot
> remove a DSACK block separately but the only way to remove DSACK is to
> not include any SACK info. That would make the Ack of Ack to be a regular
> dupAck (as per Ack field) and that would be interpreted by RACK-TLP,
> F-RTO and RFC 3708 to indicate that a duplicate *data* segment had
> arrived at the other end. Hence, all these algos are lead to make an
> incorrect decision because no data segment had arrived at the other end
> as I have already earlier explained. That is one reason why RFC 5681
> prohibits sending a dupAck unless a data segment has arrived and
> similarly RFC 2883 (Sec 4, item 2) allows only one DSACK per arriving
> duplicate data segment.
>
> > And wouldn't it help, if the sender of the ACK of an ACK would include a
> AccECN option?
>
> [MK5] Do you mean that an ACK of an ACK would always include an AccECN
> option and genuine dupAck would never include the AccECN option?
> That might help if there is never reordering. Otherwise, a regular
> cumulative Ack cannot include the AccECN option either and that is
> definitely no go.
>
> [MK5] But this is quite useless speculation, I think, because using the
> AccECN option for distinquishing Acks of Acks from other Acks would
> require that the AccECN option can always be used but the draft makes it
> quite clear that this is not the case.
>
> Best regards,
>
> /Markku
>
> > Best regards
> > Michael
> >>
> >> Thanks,
> >>
> >> /Markku
> >>
> >>> Bob
> >>>
> >>>> This would result in incorrect behaviour with DSACK and will confuse
> any algo that depends on correct behavior of DSACK in correctly reporting
> arrival of duplicate data segments, including RFC 3708 and (TLP of) RACK
> (RFC 8985) as I have earlier indicated.
> >>>> The intention of DSACK is to inform an arrival of duplicate data
> segment and the spec explicitely prohibits repeating that info in the
> spirit of the one ACK per data segment rule of RFC5681. RFC 2883, Sec 4,
> item (2):
> >>>> "Each duplicate contiguous sequence of data received is reported
> >>>> in at most one D-SACK block.  (I.e., the receiver sends two identical
> >>>> D-SACK blocks in subsequent packets only if the receiver receives two
> >>>> duplicate segments.)"
> >>>> Even if one selects the intended option (c above), it does not
> necessarily result in correct outcome. SACK specification in RFC 3517 used
> the same definition for DupAck as RFC 5681.
> >>>> The DupAck definition in RFC 6675 holds for the purposes of RFC 6675
> only as clearly stated in that specification. That is, it depends on an RFC
> spec which definition of DupAck (RFC 5681 of RFC 6675) it adopts regardless
> of whether the spec involves SACK or not. For example, F-RTO deliberately
> uses the DupAck definition of RFC 5681 for both non-SACK version and the
> SACK-enhanced version of F-RTO. In other words, the SACK-enhanced version
> of F-RTO uses an arriving DupAck without SACK option (or DupAck with SACK
> block but with no new SACK information) to declare RTO being not spurious
> similar to the non-SACK version does (Sec 3.1, step 3 a). Therefore, an
> arriving Ack of an Ack without SACK info incorrectly declares an RTO not
> spurious even if it actually was spurious RTO.
> >>>> (This needs a bit different scenario than what I draw in Fig 3 b) of
> my scenarios that was for the non-SACK version of F-RTO)
> >>>> To sum up: In order to partially solve the ambiguity on how to ack
> ACKs and avoid triggering false fast retransmits, this spec must update RFC
> 2018 by making an exception to the rule that specifies when a SACK block
> should be included. However, this does not solve the entire problem with
> Acks of Acks because there are algos like F-RTO that use the RFC 5681
> definition of DupAck. Therefore, IMHO sending Acks of Acks should not be
> included in a stds track spec.
> >>>>> You repeatedly refer forward to apparent problems with SACK, which
> sounds like you have identified that the protocol is
> >>>>> broken. But, having read forward to the end, you agree that the SACK
> condition is more solid than using TSOpt. And the only
> >>>>> concrete criticism I can find is that we don't specify what DSACK to
> send. Richard & I (and others) have worked through all
> >>>>> the cases you have proposed (and others we have contrived
> ourselves), and we cannot find any case where an implementation
> >>>>> has to do anything different from what the DSACK RFCs already say.
> >>>> [MK3] Please see the explanation above. Hope it clarifies the issue
> now.
> >>>>> I had independently found the rule you point out in RFC 5681, Sec 2,
> against generating more than one ACK for every incoming
> >>>>> [data] segment. We can say that the potential requirement to ack
> ACKs contravenes that rule, which is why the SACK condition
> >>>>> is necessary. I know you are concerned that it might not be as
> simple as that. But, without evidence, how do we distinguish
> >>>>> this from over-dramatisation? In the absence of anything specific,
> we can't just sit on our hands. As Yoshi says, these will
> >>>>> probably be corner cases. If new problems surface, we will have to
> deal with them as they arise.
> >>>> [MK3] Please see above. Definitely not a corner case.
> >>>>> #4) NOT GENERIC ENOUGH?
> >>>>> To your point about feedback from pure ACKs and data being combined,
> that's not driven by any particular CCA. If ACKs are
> >>>>> ECT, AccECN is designed to make it possible for the Data Sender to
> estimate how much of the combined per-packet feedback is
> >>>>> from ACKs and data, by using the per-byte feedback in the option to
> estimate the number of data packets. This might not be
> >>>>> perfect, but it was recognised from the start that we would need to
> decide on compromises,, particularly given very limited
> >>>>> header space.
> >>>>> Nothing in the increment-triggered ACK rule was designed around
> ECN++. Requiring the receiver to emit an ACK after 'n' CE
> >>>>> marks is a fairly obvious way of  ensuring that the sender  gets
> timely feedback of multiple CE markings.
> >>>> [MK3] Please see my reply to Richard where I explained why this
> feedback with Acks of Ack is in almost all scenarious unnecessary or not
> timely. In a single case when the other end first is idle but starts
> sending data it might be useful but even in such scenarious the feedback
> often comes too late or possibly is stale (if there is longish delay before
> the other end starts sending).
> >>>>> 'n' has to be less
> >>>>> than 7 to prevent AccECN's 3-bit ACE field wrapping. And greater
> than 2 to avoid too many ACKs. Then, as I said, we noticed
> >>>>> that could cause ping-pong, so we raised the min to 3 if the ACK
> would be in response to an ACK. That would be necessary for
> >>>>> protocol completeness (see above) whether or not an ECN++ draft
> existed.
> >>>> [MK3] Raising the min to 3 helps in dampening the ping-pong effect.
> But still the ping-pong may last several RTTs if the cwnd is large and lot
> of pkts (most) get CE marked. Not sending feedback for CE-marked DupAcks
> (and not counting them in the counters) would be much more efficient way to
> dampen the ping-pong effect. In addition, it would avoid making unnecessary
> rate reductions in such cases (when DupAcks are arriving) where there will
> be a notable rate reduction anyway (the increased Ack rate due to DupAcks
> being sent for every data pkt goes away roughly in one RTT and the end
> receiving DupAcks will react to congestion and reduce cwnd, resulting in
> lower ack rate for the next RTTs). Pls see my reply to Richard.
> >>>> This is another reason why this part of the rule is experimental in
> my opinion: we don't have any evidence/experience what is the best way to
> provide feedback/react to CE-marked pure Acks.
> >>>> It would be useful if you could give one scenario where acking
> CE-marked DupAcks would give some benefit that cannot be (easily)
> solved(gained in some other means.
> >>>>> You raise a concern that allowing Acks of Acks might preclude future
> algos that rely on the absence of 'fake dupacks'.
> >>>>> Indeed, you say that contravening the one ACK per data segment rule
> of RFC5681 "will break all mechanisms that rely on the
> >>>>> behavior specified in RFC 2883, including RFC 3708 and (TLP of) RACK
> (RFC 8985) as well as any potential new and useful
> >>>>> algos that rely on this rule." But I think this is all stated rather
> over-dramatically, because you admit that the SACK
> >>>>> condition allows an implementation to identify 'fake dupacks'. So it
> can exclude them from all such present and future
> >>>>> algos. Then this all becomes a non-issue.
> >>>> [MK3] Please see above. My apologies if I was not clear enough for
> this part. I intended to say that SACK would help in distinguishing "fake
> dupacks" that will otherwise trigger a false fast rexmit. However, SACK
> seemingly does not solve all problems (e.g., breaks f-RTO). Unfortunately I
> have no time to pass through all potential algos that may break. Even less
> time to consider all potential new algos that could count on the
> fundamental rule for generating only one DupAck per arriving data segment.
> I just try to point out the ones that I found being problematic. Even if
> one existing and (widely) deployed algo gets broken, that should be strong
> enough warning signal to not proceed as a stds track feature.
> >>>>> _______________
> >>>>> I haven't added any further discussion to the thread below (except a
> couple of minor points). To check that the above 4
> >>>>> points comprehensively address all your [MK2] points. I've tagged
> each with [BB3] and pointer(s) to the relevant response(s)
> >>>>> above.
> >>>> [MK3] I hope the above clarifications help. I didn't add anything
> below except one comment (taggd [MK3]). I may pass through the comments
> below later and comment if need be.
> >>>> Thanks,
> >>>> /Markku
> >>>>> On 11/07/2023 16:48, Markku Kojo wrote:
> >>>>>      Bob, please see [MK2] inline.
> >>>>>      And apologies for the long delay in replying, again.
> >>>>>      On Mon, 5 Jun 2023, Bob Briscoe wrote:
> >>>>>            Markku, pls see [BB2] inline
> >>>>>            On 26/05/2023 22:56, Markku Kojo wrote:
> >>>>>                  Bob,
> >>>>>                  My apologies you had to wait for the scenarios as
> it took much longer with my limited cycles
> >>>>>            than I thought.
> >>>>>                  Anyways, please see my reply to Richard, some
> scenarios are also included there.
> >>>>>                  To keep things easier, it might be good to try to
> keep the discussion on Acks of Acks (mainly)
> >>>>>            in the thread
> >>>>>                  with my reply to Richard.
> >>>>>                  However, see inline tagged [MK].
> >>>>>                  On Wed, 24 May 2023, Bob Briscoe wrote:
> >>>>>                        Markku,
> >>>>>                        Sorry, it's taken a week to build a
> comprehensive reply to this long email. See inline tagged [BB]...
> >>>>>                        On 17/05/2023 12:24, Markku Kojo wrote:
> >>>>>                              Hi Michael, all,
> >>>>>                              On Sun, 14 May 2023,
> [e-mail] wrote:
> >>>>>                                          On 30. Mar 2023, at 16:53,
> Bob Briscoe <[e-mail]>
> >>>>>                                          wrote:
> >>>>>                                          Michael, Yoshi, Ian (as
> tcpm chairs),
> >>>>>                                          To close off the WGLC, I
> have just posted a new rev of
> >>>>>                                          accurate-ecn. Hyperlinks
> quoted at the end.
> >>>>>                                          You will see the diff is
> rather extensive. I won't give a
> >>>>>                                          summary of all the diffs
> like I usually do. Instead I can just
> >>>>>                                          refer to the summary I gave
> in the presentation on Monday:
> >>>>>
> https://datatracker.ietf.org/meeting/116/materials/slides-116-tcpm-draft-ietf-tcpm-accurate-ecn
> >>>>>                                          Thank you again to the
> people who reviewed this during the WGLC:
> >>>>>                                          Michael Tüxen, Alex Burr,
> Gorry Fairhurst and Markku Kojo.
> >>>>>                                          All changes are editorial,
> apart from removing the para about
> >>>>>                                          not mistaking certain ACKs
> of ACKs for DupACKs, which I will add
> >>>>>                                          to a rev of the ECN++
> draft, hopefully later this week.
> >>>>>                                          On the list, we have seen
> agreement from all the reviewers to
> >>>>>                                          these changes, except no
> response from Markku yet.
> >>>>>                                          On Monday, I told Markku
> that I would post the draft in a few
> >>>>>                                          days, so everyone can see
> the updates and diff.
> >>>>>                                    Anyone having additional
> comments? In particular Markku regarding loss
> >>>>>                                    recovery?
> >>>>>                              My apologies for being late with my
> reply to the author's comments on my review
> >>>>>            (I've
> >>>>>                              been extremly busy with other issues
> since the wg mtng in Yokohama, including the
> >>>>>            rest
> >>>>>                              of mtng week).
> >>>>>                              I don't have much new comments but it
> seems that my major concern regarding the
> >>>>>            problem
> >>>>>                              of sending ACKs of ACKs was not fully
> understood.
> >>>>>                              The first thing where I think I was not
> quite clear is that the major problem with
> >>>>>            ACKs
> >>>>>                              of ACKs is not that a pure Ack is made
> ECN-capable. Instead, the problem is in
> >>>>>                              generating an Ack of an pure Ack and
> that is what one should prohibit to avoid
> >>>>>            problems.
> >>>>>                              I understand that it might be
> problematic to formulate rules whether generating an
> >>>>>            Ack
> >>>>>                              of an Ack is allowed (and when),
> instead of just disabling sending ECN-capable
> >>>>>            ACKs.
> >>>>>                              I don't have a strong opinion which way
> the problems with ACKs of ACKs is avoided
> >>>>>            as
> >>>>>                              long as they are avoided.
> >>>>>                        [BB] See later after your similar point
> (following your 'Why?' heading)...
> >>>>>                              I am preparing a few scenarios to
> illustriate the problems ACKs of ACks raise and
> >>>>>            will
> >>>>>                              send them shortly once I have
> formulated a more thorough reasoning why sending
> >>>>>            ACKs of
> >>>>>                              ACKs is not really a good idea and even
> seems to be unnecessary in most if not all
> >>>>>                              cases, i.e., it just results in sending
> unnecessary packets with not much useful
> >>>>>            effect
> >>>>>                              but creates a notable number of
> problems.
> >>>>>                        [BB] Having waited this long, it's rather
> disappointing to still hear you say "I have an
> >>>>>                        argument,
> >>>>>                        but I'll tell you later."
> >>>>>                  [MK] I understand. My sincere apologies again.
> >>>>>                              Given that this draft is intended to
> become a stds track RFC I am concerned of any
> >>>>>            text
> >>>>>                              in this document that indicates (or
> even hints) that TCP could acknowledge pure
> >>>>>            ACKS
> >>>>>                              (this holds particularly the rules and
> text in Sec 3.2.2.5.1 for the
> >>>>>                              "Increment-Triggered ACKs"). If it is
> seen necessary that this doc should have
> >>>>>            such
> >>>>>                              pieces of rules and text, I am fine if
> any such text is moved to an appendix as
> >>>>>            long as
> >>>>>                              the appendix makes it cristal clear
> that the text is valid only in case one is
> >>>>>                              implementing an experiment as per
> [I-D.ietf-tcpm-generalized-ecn].
> >>>>>                        [BB] See point below about "Generic
> (Mechanistic) Reflector".
> >>>>>                              Why?
> >>>>>                              1) It is well known that TCP does not
> acknowledge ACKs and Standards track TCP has
> >>>>>            not
> >>>>>                              been specified to acknowledge ACKs.
> This means that a reader/implementer of this
> >>>>>            doc
> >>>>>                              cannot correctly understand the rule
> for "Increment-Triggered ACKs" unless there
> >>>>>            is a
> >>>>>                              normative reference to a spec that
> specifies ACKs of ACKs (or tells that it is
> >>>>>            even
> >>>>>                              possible).
> >>>>>                        [BB] ACKs of ACKs can indeed be tricky. But
> there's no need to consider not ACKing ACKs
> >>>>>            as an
> >>>>>                        architectural principle. Not Acking ACKs on
> principle certainly avoids some tricky
> >>>>>            problems.
> >>>>>                        However, we have a new situation here where,
> in limited circumstances, ACKs of ACKs are
> >>>>>                        necessary.
> >>>>>                        So the WG has already worked through the
> tricky problems and they have been addressed in
> >>>>>            the
> >>>>>                        draft
> >>>>>                        (e.g. mistaking ACKs of ACKs for DupACks,
> infinite ping-pong, etc). We'll discuss below
> >>>>>            whether
> >>>>>                        you've found some more trickiness.
> >>>>>                  (MK] I did not mean to refer to any principle but,
> as I said, that a reader/implementer cannot
> >>>>>            correctly
> >>>>>                  understand the rule for "Increment-Triggered ACKs"
> because it is well-known to her/him that
> >>>>>            TCP does not Ack
> >>>>>                  ACKs. This fact is that one can ack ACKs is not
> specified in this doc
> >>>>>            [BB2] The fact that one /can/ ack ACKs is specified in
> the AccECN draft. It clearly says:
> >>>>>                "Although TCP normally only ACKs newly delivered
> data, in this case it is mandatory for A to
> >>>>>            emit ACKs of ACKs
> >>>>>            because they feed back new congestion state (useful in
> case B starts sending)."
> >>>>>            We'll also make it clearer that "in this case" means "in
> the case of ECN-capable ACKs". And we'll
> >>>>>            make it clear that it's
> >>>>>            conditional on carefully following all the conditions in
> any draft that specifies ECN-capable ACKs.
> >>>>>      [MK2] Right, but this (acking Acks) is experimental behaviour,
> needed only for those participating the
> >>>>>      experiment. However, this doc specifies the behaviour in the
> Increment-Triggered ACKs rule: MUST emit an ACK".
> >>>>>      If one implements this draft but is not interested in ECN++ nor
> any other similar experiment that allows
> >>>>>      ECN-capable ACKs, one must still implement these Acks of Acks
> but there is no specification nor normative ref
> >>>>>      how to do it (correctly, including all the conditions). If then
> this implementation is deployed in the wild, it
> >>>>>      automatically becomes part of the experiment once the other
> endpoint enables ECN-capable ACKs but without the
> >>>>>      necessary safety procedures. This doc cannot depend on an
> experimental spec which explains how to do it and what
> >>>>>      conditions to follow.
> >>>>> [BB3] See
> >>>>> #1) Generic should not be confused with experimental.
> >>>>> #2) Completeness
> >>>>> #3) Does not specify the necessary details?
> >>>>>                  nor does this doc give a (normative) reference
> where it is specified,
> >>>>>            [BB2] As already explained, there is no normative ref
> where acking ACKs is specified in general,
> >>>>>            because they are not
> >>>>>            needed in general. They are specified whenever they are
> needed, including in the examples Yoshi
> >>>>>            pointed to (keepalive,
> >>>>>            etc).
> >>>>>      [MK2] An implementation of this draft send Acks of Acks
> "randomly" and possibly multiple of them in a row but
> >>>>>      this draft does not specify the necessary details unlike you
> claim above.
> >>>>> [BB3] See #4) Does not specify the necessary details
> >>>>>      [MK2] Pls see also my reply to Yoshi. The specific examples
> Yoshi mentions are sent in very limited cases and
> >>>>>      result in sending only a single Ack of an Ack which is not
> causing any harm to others unlike sending multiple
> >>>>>      "random" Acks of Acks.
> >>>>>                  including the details on which TSecr value to add
> or which SACK info if any to include when
> >>>>>            acking a pure
> >>>>>                  Ack.
> >>>>>            [BB2] OK.  But I've had to read all the through SACK and
> TS specs to guess what complication you
> >>>>>            might have seen. If you
> >>>>>            know what made you ask this question, it would be much
> easier if you gave us a clue, or even just
> >>>>>            told us explicitly.
> >>>>>      [MK2] Sorry for not being clear/detailed enough. Maybe this was
> too obvious to me. See details below for SACK
> >>>>>      (and my reply to Richard for Timestamps for which I believe you
> already figured out the problem).
> >>>>> [BB3] See #3) Does not specify the necessary details?
> >>>>>            * For TS, I'm pretty sure the normal rules (for which
> TSecr value to include) apply.
> >>>>>            * Similarly, for SACK, I can't see that anything new
> needs to be said.
> >>>>>                  It is easy to misinterpret the "Increment-Triggered
> ACKs", if one doesn't realize that pure
> >>>>>            Acks may be
> >>>>>                  acked.
> >>>>>            [BB2] We'll make it clearer in the rule that 'packet'
> intentionally includes packets without data.
> >>>>>      [MK2] But as I said earlier, it would specify experimental
> behavior in a PS doc.
> >>>>> [BB3]  See: #1) Generic should not be confused with experimental.
> >>>>>                        What is the new situation?
> >>>>>                         *  Until ECN was introduced, TCP ACKs only
> acknowledged data. So there was no need to
> >>>>>                        acknowledge
> >>>>>                            pure ACKs, which contain no data.
> >>>>>                         *  When ECN was introduced in RFC3168, TCP
> ACKs also acknowledged ECN markings.
> >>>>>            However, because
> >>>>>                            RFC3168 precluded pure ACKs from being
> ECN-capable, there was still no need to
> >>>>>            acknowledge
> >>>>>                        pure
> >>>>>                            ACKs.
> >>>>>                         *  RFC5690, and now the ECN++ draft
> introduce the possibility of ECN-capable pure ACKs.
> >>>>>            So, in
> >>>>>                        the
> >>>>>                            limited circumstances described in the
> AccECN draft, ECN-capable pure ACKs now need
> >>>>>            to be
> >>>>>                            acknowledged, because they contain new
> information - their ECN field.
> >>>>>                        Similarly, even though the final ACK of TCP's
> 3WHS is an ACK of an ACK , it is sent
> >>>>>            because it is
> >>>>>                        needed (to prove that the SYN wasn't from a
> spoofed address).
> >>>>>                  [MK] All otherwise clear, but I disagree that the
> final ACK of TCP's 3WHS is an ACK of an ACK.
> >>>>>            It is required
> >>>>>                  because SYNACK contains control data that eats one
> sequence number, i.e., it advances RCV.NXT
> >>>>>            at the client
> >>>>>                  end and when the ACK arrives at the server it is
> needed to advance SND.UNA. Very different
> >>>>>            from Acks of Acks
> >>>>>                  in this draft.
> >>>>>            [BB2] Yes, not the same, but similar - ECN information is
> also control data. But agreed that it
> >>>>>            doesn't consume a
> >>>>>            sequence no. Neither do keepalive probes or their ACKs.
> >>>>>      [MK2] Right, but the actual purpose of Acks is to acknowledge
> sequence numbers (that have arrived) and the logic
> >>>>>      of many crucial TCP algos depend on how packets are acked.
> >>>>> [BB3] Good protocol specs never define or limit the uses of the
> protocol (it would be pointless to try anyway).
> >>>>> If you only wanted ACKs to be used to acknowledge sequence numbers,
> you should have objected to RFC3168, which used ACKs to
> >>>>> feed back ECN as well. Or you could go further back and object to
> SACK because it acknowledges gaps, not sequence numbers
> >>>>> that have arrived.
> >>>> [MK3] With both RFC 3168 and SACK the ACKS ack the sequence numbers
> in the first place. These specs just piggyback additional information on
> ACKS that would be sent anyway. They do not introduce extra pkts to be sent
> like this spec does. I think this makes a major difference.
> >>>> [snipped the rest of msg as the msg was too long for the wg list]
> >>>
> >>> --
> >>> ________________________________________________________________
> >>> Bob Briscoe                               http://bobbriscoe.net/
> >>>
> >> _______________________________________________
> >> tcpm mailing list
> >> [e-mail]
> >> https://www.ietf.org/mailman/listinfo/tcpm
> >
> >_______________________________________________
> tcpm mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/tcpm
>