Re: [tcpm] draft-ietf-tcpm-accurate-ecn-24 addressing all WGLC comments

[Markku Kojo <[e-mail]>]

Hi Yoshi,

thank you for your comments. Pls see inline.

On Mon, 5 Jun 2023, Yoshifumi Nishida wrote:

> Hi folks,
> 
> I just would like to put my personal views on ack on ack discussions here.
> First, I think ack on ack has already has some precedences. 
> keep-alive logic can sent a pure ack for proving and expect an ack for it. 

Keep-alives are very much different in their potential to cause any harm. 
Only a single segment is sent when there is no traffic in flight, by 
default no traffic for the recent two hours (repeated infrequently
a few times, if no reply). So, quite impossible to cause any harm for 
ongoing data communication.

> MPTCP uses 4 WHS and the 4th segment is an ack for the third ACK which is a pure ACK.
> So, I'm not sure if we need to define some rules for ack-on-ack in the doc.

This is also a single pure Ack sent in a very specific situation only: 
immediately after the last Ack of 3whs. Very hard to find any potential 
harm it may cause. Particularly, this pure Ack is not sent "randomly" at 
any point of an ongoing communication nor is it sent possibly multiple 
times per RTT.

> Also, I'm a bit hesitant to define a detailed logic about how to distinguish an ack that carries ECN signals and a dup
> ack, such as using TS or SACK blocks.
> I think such things are the part of experiments and should be described in other docs such as ECN++ or ackcc, etc. 
> I personally prefer the doc simply describes the possibilities of such mechanisms and provides general principles and
> guidelines. 

I pretty much agree. As currently written in -24 that I am commenting what 
happens is that if the other end for whatever reason makes its ACKs 
ECN-capable, it will easily result in automatic generation of these 
excess dupacks, which are MUST NOT in RFC 5681, Sec. 4.2, last para.
Furthermore, sending Acks for CE-marked pure Acks is also part of 
experiment too. It is not needed otherwise, so it should be specified in 
other docs, not in this PS to be. Thereby, the feature (acking Acks) would 
be in the same doc as the necessary measures to make it safe.

> As far as I think, the Markku's examples that triggers false retransmissions makes sense. I think they are good examples
> to show how ack-on-ack can be tricky.
> However, these examples are the case where both sides exchanges data simultaneously and I think there're other cases
> where we don't have to worry about it. 

No, they are examples of cases where communication is bidirectional but 
not simultaneous. Instead, they are scenarios where the end points send 
data in turns, i.e., the data flow direction alternates. Such 
request-reply app. communication scenarios are very common, if not the 
most common communication pattern over TCP in the Internet (e.g., most 
Web-based apps). Of course, there are scenarios that won't create 
problems, but even if there is just one problematic scenario, it must be 
addressed properly.

> For example, I think bulk transfer or request-response type traffic can be these examples. 

Bulk transfer, yes, request-response type traffic definitely not.

Please see Fig 1 of my scenarios but ignore data 0 from B to A and 
corresponding ACK from A to B. I just added this (data 0 and the 
corrsponding ACK) there to indicate which was the last cumulatively acked 
segment from B to A (I added it as the last thing after drawing other 
stuff, so there was not room to place it earlier, sorry). You may freely 
move that excange to occur before A sends the segment 20(000) which makes 
this typical data exchange of a request-reply app.

> In these cases, the endpoint which receives acks for acks doesn't have outstanding data. Hence, although these acks are
> duplicate acks, they won't trigger retransmissions.

I am afraid this is incorrect. Please see Fig 1 of my scenarios: when 
dupacks (Acks of Acks, marked red) start to arrive at B, B has sent out 
data 1..100, i.e., all these segments are outstanding and "fake" dupacks 
will trigger false Fast Retransmit and unnucessary retransmissions in the 
current window of outstanding data. All the retransmissions are 
unnecessary excess data segments sent out during the same RTT as their 
original transmissions! Obviously very bad behaviour that must be avoided.

> So, I am thinking that it would be good to provide a certain guideline about when to enable this feature and potential
> risks in some docs.
> But, I am also thinking we should do it outside of accecn doc.

I would suggest putting anything that concerns Acking ACKs outside of 
AccECN doc, because such behaviour is experimental and AccECN is 
targetted as PS.

Thanks,

/Markku

> Thanks,
> --
> Yoshi
> 
> 
> On Fri, May 26, 2023 at 2:56 PM Markku Kojo <[e-mail]> wrote:
>       Bob,
>
>       My apologies you had to wait for the scenarios as it took much longer
>       with my limited cycles than I thought. Anyways, please see my reply to
>       Richard, some scenarios are also included there.
>
>       To keep things easier, it might be good to try to keep the discussion on
>       Acks of Acks (mainly) in the thread with my reply to Richard.
>
>       However, see inline tagged [MK].
>
>       On Wed, 24 May 2023, Bob Briscoe wrote:
>
>       > Markku,
>       >
>       > Sorry, it's taken a week to build a comprehensive reply to this long email. See inline tagged
>       > [BB]...
>       >
>       > On 17/05/2023 12:24, Markku Kojo wrote:
>       >       Hi Michael, all,
>       >
>       >       On Sun, 14 May 2023, [e-mail] wrote:
>       >
>       >                   On 30. Mar 2023, at 16:53, Bob Briscoe <[e-mail]>
>       >                   wrote:
>       >
>       >                   Michael, Yoshi, Ian (as tcpm chairs),
>       >
>       >                   To close off the WGLC, I have just posted a new rev of
>       >                   accurate-ecn. Hyperlinks quoted at the end.
>       >                   You will see the diff is rather extensive. I won't give a
>       >                   summary of all the diffs like I usually do. Instead I can just
>       >                   refer to the summary I gave in the presentation on Monday:
>       >                 
>        https://datatracker.ietf.org/meeting/116/materials/slides-116-tcpm-draft-ietf-tcpm-accurate-ecn
>       >
>       >                   Thank you again to the people who reviewed this during the WGLC:
>       >                   Michael Tüxen, Alex Burr, Gorry Fairhurst and Markku Kojo.
>       >
>       >                   All changes are editorial, apart from removing the para about
>       >                   not mistaking certain ACKs of ACKs for DupACKs, which I will add
>       >                   to a rev of the ECN++ draft, hopefully later this week.
>       >
>       >                   On the list, we have seen agreement from all the reviewers to
>       >                   these changes, except no response from Markku yet.
>       >                   On Monday, I told Markku that I would post the draft in a few
>       >                   days, so everyone can see the updates and diff.
>       >
>       >             Anyone having additional comments? In particular Markku regarding loss
>       >             recovery?
>       >
>       >
>       >       My apologies for being late with my reply to the author's comments on my review (I've
>       >       been extremly busy with other issues since the wg mtng in Yokohama, including the rest
>       >       of mtng week).
>       >
>       >       I don't have much new comments but it seems that my major concern regarding the problem
>       >       of sending ACKs of ACKs was not fully understood.
>       >
>       >       The first thing where I think I was not quite clear is that the major problem with ACKs
>       >       of ACKs is not that a pure Ack is made ECN-capable. Instead, the problem is in
>       >       generating an Ack of an pure Ack and that is what one should prohibit to avoid problems.
>       >       I understand that it might be problematic to formulate rules whether generating an Ack
>       >       of an Ack is allowed (and when), instead of just disabling sending ECN-capable ACKs.
>       >       I don't have a strong opinion which way the problems with ACKs of ACKs is avoided as
>       >       long as they are avoided.
>       >
>       >
>       > [BB] See later after your similar point (following your 'Why?' heading)...
>       >
>       >
>       >       I am preparing a few scenarios to illustriate the problems ACKs of ACks raise and will
>       >       send them shortly once I have formulated a more thorough reasoning why sending ACKs of
>       >       ACKs is not really a good idea and even seems to be unnecessary in most if not all
>       >       cases, i.e., it just results in sending unnecessary packets with not much useful effect
>       >       but creates a notable number of problems.
>       >
>       >
>       > [BB] Having waited this long, it's rather disappointing to still hear you say "I have an argument,
>       > but I'll tell you later."
>
>       [MK] I understand. My sincere apologies again.
>
>       >       It also seems not have been carefully enough considered in terms of the very basic
>       >       rubustness principle of "be conservative in what you send ..."
>       >
>       >
>       > [BB] The WG has been careful to ensure that ACKs of ACKs are unambiguous (cannot be mistaken for a
>       > DupACK), which is what the robustness principle requires. It's just that you think we have missed
>       > cases where they will be ambiguous. If you think that, we need to hear them all. 
>       >
>       > The robustness principle does not advocate sending nothing just in case some unknown factor might
>       > make it ambiguous. Especially given /not/ feeding back congestion notifications has potential to
>       > cause harm to others. Also, "no feedback" is much more ambiguous.
>
>       [MK] Please see my reply to Richard and let's continue from there. See
>       also what I meant with "be conservative in what you send ...", that is,
>       in context of CC: avoid sending unnecessary packets or be careful in
>       sending packets just because they might be sometimes useful, send the
>       monly when they are useful.
>
>       >       Given that this draft is intended to become a stds track RFC I am concerned of any text
>       >       in this document that indicates (or even hints) that TCP could acknowledge pure ACKS
>       >       (this holds particularly the rules and text in Sec 3.2.2.5.1 for the
>       >       "Increment-Triggered ACKs"). If it is seen necessary that this doc should have such
>       >       pieces of rules and text, I am fine if any such text is moved to an appendix as long as
>       >       the appendix makes it cristal clear that the text is valid only in case one is
>       >       implementing an experiment as per [I-D.ietf-tcpm-generalized-ecn].
>       >
>       >
>       > [BB] See point below about "Generic (Mechanistic) Reflector".
>       >
>       >
>       >       Why?
>       >
>       >       1) It is well known that TCP does not acknowledge ACKs and Standards track TCP has not
>       >       been specified to acknowledge ACKs. This means that a reader/implementer of this doc
>       >       cannot correctly understand the rule for "Increment-Triggered ACKs" unless there is a
>       >       normative reference to a spec that specifies ACKs of ACKs (or tells that it is even
>       >       possible).
>       >
>       >
>       > [BB] ACKs of ACKs can indeed be tricky. But there's no need to consider not ACKing ACKs as an
>       > architectural principle. Not Acking ACKs on principle certainly avoids some tricky problems.
>       > However, we have a new situation here where, in limited circumstances, ACKs of ACKs are necessary.
>       > So the WG has already worked through the tricky problems and they have been addressed in the draft
>       > (e.g. mistaking ACKs of ACKs for DupACks, infinite ping-pong, etc). We'll discuss below whether
>       > you've found some more trickiness.
>
>       (MK] I did not mean to refer to any principle but, as I said, that a
>       reader/implementer cannot correctly understand the rule for
>       "Increment-Triggered ACKs" because it is well-known to her/him that TCP
>       does not Ack ACKs. This fact is that one can ack ACKs is not specified in
>       this doc nor does this doc give a (normative) reference where it is
>       specified, including the details on which TSecr value to add or which
>       SACK info if any to include when acking a pure Ack. It is easy to
>       misinterpret the "Increment-Triggered ACKs", if one doesn't realize that
>       pure Acks may be acked.
>
>       > What is the new situation?
>       >  *  Until ECN was introduced, TCP ACKs only acknowledged data. So there was no need to acknowledge
>       >     pure ACKs, which contain no data.
>       >  *  When ECN was introduced in RFC3168, TCP ACKs also acknowledged ECN markings. However, because
>       >     RFC3168 precluded pure ACKs from being ECN-capable, there was still no need to acknowledge pure
>       >     ACKs.
>       >  *  RFC5690, and now the ECN++ draft introduce the possibility of ECN-capable pure ACKs. So, in the
>       >     limited circumstances described in the AccECN draft, ECN-capable pure ACKs now need to be
>       >     acknowledged, because they contain new information - their ECN field.
>       > Similarly, even though the final ACK of TCP's 3WHS is an ACK of an ACK , it is sent because it is
>       > needed (to prove that the SYN wasn't from a spoofed address).
>
>       [MK] All otherwise clear, but I disagree that the final ACK of TCP's 3WHS
>       is an ACK of an ACK. It is required because SYNACK contains control data
>       that eats one sequence number, i.e., it advances RCV.NXT at the client
>       end and when the ACK arrives at the server it is needed to advance
>       SND.UNA. Very different from Acks of Acks in this draft.
>
>       > It is true that not ACKing ACKs is well-known. However, whether it's well-known as a /principle/, or
>       > just as a current /feature/ of TCP is not clear. Anyway, the IETF's job is to update RFCs that are
>       > "well-known". We don't have to jump through any special procedural hoops to do something different
>       > from what is "well-known". Even if it were prohibited in a stds track RFC, we just have to specify
>       > what has to be done instead; in another stds track RFC.
>
>       [MK] Again, I didn't mention it as a /principle/ but as a crucial
>       piece of information that the reader needs to be noted, that is, the
>       things are now different from what is well-known.
>
>       Sure IETF's job is to update RFCs, but if one changes what is prohibited
>       in a stds track RFC, one needs to understand the consequences and explain
>       them as well as give the justification why the change can be done (without
>       problems), instead of just specifying the change.
>
>       > If there are any tutorials, course notes or text books out there that say that not ACKing ACKs is a
>       > well-known principle, that's not the IETF's problem. It is the job of the tutors, lecturers and text
>       > book authors who wrote those materials to update them.
>       >
>       >
>       >       2) ACKs of ACKs tend to trigger duplicate Acks. There are tons of algorithms that rely
>       >       on the packet conservation principle and the fact that TCP never injects a dupAck unless
>       >       a *data* packet has arrived and left the network. This is enforced with "MUST NOT" in
>       >       RFC 5681, Sec 4.2, because not conforming to this rule makes any algorithm that rely on
>       >       the rule to work incorrectly. These algorithms include (triggering) Fast Retransmit,
>       >       (controlling packet rate during) Fast Recovery, (detecting spurious RTOs in) F-RTO,
>       >       (calculating PipeAck in) RFC 7661, (calculating DeliveredData in) PRR, etc. Furthermore,
>       >       it would make imposible to come up with any new algorihms that rely on this important
>       >       basic rule. In most cases such extra dupAcks make these algorithms too aggressive
>       >       because any extra dupAck is likely to inject extra packet(s) to the network.
>       >
>       >       So, it should be cristal clear that without SACK (or Timestamps) a TCP *MUST NOT* send
>       >       ACKs of ACKs.
>       >
>       >
>       > [BB] Constraining the /Data Receiver/ as you propose would create an interop problem.
>       > Explanation: Consider host A and B are not using SACK or timestamps. Nonetheless, with your
>       > approach, host A can still send ECN-capable pure ACKs to host B. Then, your rule puts host B in an
>       > impossible position, where it gets congestion notifications on ECN-capable pure ACKs, but it is not
>       > allowed to send any feedback about them.
>       >
>       > Instead, if neither timestamps nor SACK are in use for the connection, we need to constrain the
>       > /Data Sender/ of a half connection from sending ECN-capable ACKs in the first place. This is the
>       > approach the WG has adopted in the AccECN and ECN++ specs.
>
>       [MK] I think I said in the beginning that I have no strong opinion which
>       way Acks of Acks are disabled. However, I apologize that I didn't explain
>       why I phrased MUST NOT send ACKs of ACKs. This is because it might be
>       still useful to allow CE-marked pure Acks and take care of Ack CC by some
>       other means than Acks of Acks. Currently the draft mandates Acks of Acks
>       as the only way to report Ack congestion and I think it is too
>       restrictive in a stds track doc, e.g., it rules out reducing Ack rate
>       simply by reducing data send rate which would solve the interop problem in
>       a very simple way. Moreover, When B gets congestion notifications on
>       ECN-capable pure ACKs, not sending Acks of Acks does not prevent sending
>       feedback; such feedback need not to be delivered immediately but by the
>       time needed. Please see more on this in my reply to Richard.
>
>       > Specifically:
>       >  *  The WG makes sure that RFCs about the /Data Sender/ of a half connection (e.g. the ECN++
>       >     experiment or other future RFCs) specify that sending ECN-capable pure ACKs is conditional on
>       >     having another way to distinguish DupACKs, e.g. negotiating SACK or timestamps (and I will
>       >     respond to your later points on the details of these).
>       >  *  The AccECN spec (which primarily specifies the feedback behaviour of a /Data Receiver/ in a
>       >     half-connection) then only needs to define the Increment-triggered ACK rule.
>       > The two together lead to the same outcome you want. But without the interop hole of your approach.
>       >
>       > This is consistent with the "Generic (Mechanistic) Reflector" approach of the AccECN spec which
>       > says:
>       > "AccECN is designed to be a generic reflector of whatever ECN markings it sees, whether or not they
>       > are compliant with a current standard."
>       >
>       > These ACKs of ACKs are generically necessary to feed back congestion notifications from possible
>       > incoming packet patterns, not specifically for ECN++ or AckCC [RFC5690], or any other future RFC
>       > (forward compatibility). We'll edit the reference to ECN++ to make it clearer that it's one example,
>       > not the only case.
>       >
>       > Here's another example of the generic reflector approach, already in the draft:
>       > "Although RFC 3168 prohibits an ECN-capable SYN, providing feedback of ECN marking on the SYN
>       > supports future scenarios in which SYNs might be ECN-enabled (without prejudging whether they ought
>       > to be). ... "
>       >
>       >
>       >       So, it should be cristal clear that without SACK (or Timestamps) a TCP *MUST NOT* send
>       >       ACKs of ACKs.
>       >
>       >
>       > I understand that you want this but, as just explained, without SACK or timestamps, the correct
>       > approach is to prevent the Data Sender putting the Data Receiver in the position where it would have
>       > to ACK ACKs in the first place.
>       >
>       > In a connection without SACK or timestamps, if the Data Receiver were to get lots of congestion
>       > notifications on ECN-capable ACKs, it would face a difficult dilemma. Which would be more important:
>       > Signalling congestion by ACKing ACKs? or ensuring the performance improvements like Fast Retransmit,
>       > Fast Recovery etc. work well? The former would prevent harm to others, the latter would prevent harm
>       > to self.
>
>       [MK] Please see my reply to Richard to understand why Acks of Acks
>       cause Fast Retransmit, Fast Recovery, etc to cause harm to others in the
>       first place. And also why most of the Acks of Acks seem to be just
>       unnecessary load to the network, that is, they harm others without (much)
>       benefits.
>
>       > Nonetheless, I will add some text to the AccECN draft that explains why it is important for other
>       > RFCs not to put a Data Receiver in the position where it has to ACK ACKs iff there is no way to
>       > distinguish them from DupACKs.
>       >
>       >
>       >       3) Even with SACK or Timestamps enabled it is not clear what an
>       >       implementer should do. With SACK the AckECN authors seem to make an assumption, which
>       >       seems obvious but is not, that an ACK of ACK would would never include SACK option and
>       >       hence it could be distinguished from a dupAck. However, RFC 2018 specifies: "If sent at
>       >       all, SACK options SHOULD be included in all ACKs which do not ACK the highest sequence
>       >       number in the data receiver's queue. So, if there is a hole in the receiver's queue, the
>       >       assumption is incorrect and it is unclear which SACK info to include into the SACK
>       >       option. Whatever one selects to include, it makes DSACK (RFC 2883] void and breaks any
>       >       DSACK-based algorithms unless RFC 2018 is updated.
>       >
>       >
>       > [BB]
>       > Reading the draft, it is very clear that there is no such assumption. The text said solely what it
>       > meant:
>       >
>       >    ... a host in AccECN
>       >    mode that is sending ECN-capable pure ACKs SHOULD add one of the
>       >    following additional checks when it tests whether an incoming pure
>       >    ACK is a duplicate:
>       >
>       >    o  If SACK has been negotiated for the connection, but there is no
>       >       SACK option on the incoming pure ACK, it is not a duplicate;
>       > That is, if an incoming ACK were a duplicate, it would have a SACK option on it. This /relies/ on
>       > the rule in RFC2018 that you quote. So there is no need (nor intention) to change SACK behaviour in
>       > any way.
>
>       [MK] Sorry I don't understand your claim. Assume A sends data pkts
>       0,1,2,3,..,10 in the current window to B and pkt 1 is dropped.
>       Simultaneously, B sends data to A such that the data pkts arrive at A
>       after A has injected pkt 10 to the network. These pkts trigger pure
>       cumulative Acks from A to B that follow A's data pkts 1..10 and enough of
>       the Acks get CE-marked due to congestion path A to B. When pkts 2..10
>       arrive at B, each of them trigger a valid dupAck with a SACK block
>       included. When the CE-marked Acks that follow data pkts 2..10 arrive at
>       B, B needs to feedback congestion info on them in Acks of Acks. These
>       Acks of Acks cannot cumulatively ACK the highest sequence number in the
>       data receiver's queue (pkt 10) since pkt 0 is the highest pkt
>       arrived insequence, so the Ack of Ack must include a SACK block as per
>       RFC 2018. What is the SACK block info that the implementer, who follows
>       carefully advice in RFC 2018 and there is no other advice, should
>       include in these Acks unless RFC 2018 is not changed? How does the
>       implementer know how to Ack, if it not specified?
>
>       > (Note: to check this text, you'll need to refer to the previous AccECN draft here:
>       >     https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accurate-ecn-23#section-3.2.2.5.1
>       > It had been there since Jul 2021 (since -15). But, as explained above, it is longer in the latest
>       > AccECN draft (-24), because it has been moved to the editor's copy of the ECN++ draft, which I wrote
>       > on 4 Apr, but I've been waiting for your reply before submitting it.)
>       >
>       >
>       >       With Timestamps some algorithms like Eifel detection breaks. Moreover, there are other
>       >       existing or potentially to-be-created heauristics, including various measurement tools,
>       >       that rely on the fact that TCP does not echo a later Timestamp in a pure Ack than what
>       >       arrived with the latest data packet. Any such mechanisms are subject to break.
>       >
>       >
>       > [BB] I don't fully understand what you're saying here. Can you be clearer please?
>       > And can you please bear in mind that we are in the WGLC processing stage now. So review comments
>       > ought to be suggesting very specific changes to the text under review.
>
>       [MK] Again, the same problem as with SACK. It is unspecified which
>       TSecr value to put in Acks of Acks. That has not been specified
>       anywhere, so how can I or anyone else check what breaks if anything, and
>       how can an implementer know which TSecr value to include in Acks of Acks?
>
>       It is hard to propose specific changes to text that does not exist or to a
>       problem that seems to be a missing piece of design or a design flaw, until
>       the text exists or the intended design is known or the potentiel design
>       flaw is mutually agreed whether there is a flaw or not.
>
>       (Please note also that the Eifel problem seems not to be serious, but
>       instead the timestamps rule you proposed to distinguish dupAcks from Acks
>       of Acks seems suspicious and calls for clarification).
>
>       Please see more details in my reply to Richard.
>
>       > Again, this text about extra DupACK checks has now been removed from AccECN and will shortly appear
>       > in the ECN++ draft instead. I shall post the new ECN++ draft shortly.
>       >
>       >
>       >       It might be good to hold discussing any details on what breaks and how/why and what are
>       >       the consequences until I have sent my reply with scenarios to Richard.
>       >
>       >
>       > [BB] Please try to prioritize any comments about the text that is now left in the AccECN draft. The
>       > WGLC of AccECN is waiting for no-one else at the moment.
>
>       [MK] Unfortunately I was not able to do this, sorry. I got confused
>       already earlier when I was reviewing AccECN and ended up checking ECN++
>       quickly as I noted that AccECN draft cited it for these issues. When
>       reading ECN++ I found Sec 3.3.3 and read:
>
>         "The question of whether and how the receiver of pure ACKs is required to
>          feed back any CE marks on them is outside the scope of the present
>          specification because it is a matter for the relevant feedback
>          specification ([RFC3168] or [I-D.ietf-tcpm-accurate-ecn])."
>
>       This got me totally confused together with the later discussion on the mic
>       at Yokohama mtng because that was what I had read, and I was not able to
>       understand what was intended to go where. And I still not quite know
>       but I think I have a better hunch. The above text still reads in
>       ECN++ draft, but hopefully not in editor's copy?
>
>       Anyways, the major problem is not with any certain text phrases but
>       whether DupAck vs. Ack of Ack problem is solvable and whether
>       injecting Acks of Acks really is a needed and mature enough feature that
>       can be part of a stds track protocol.
>
>       > But also bear in mind that the chairs plan to take ECN++ into WGLC once AccECN WGLC has been
>       > cleared. So we need to hear your actual argument about the DupACK text that has been moved to ECN++
>       > urgently too. We can't work with "I have an argument, but I'll tell you later".
>
>       [MK] Apologies for the delay again. Please see my reply to Richard for my
>       arguments.
>
>       >       One additional comment regarding the "Change-Triggered ACKs" rule is that it would be
>       >       useful to make it more clear how this plays with delayed Acks and how it alters
>       >       acknowledgement rate.
>       >       I am not sure that what the draft currently says is quite correct:
>       >
>       >        "The approach can lead to some additional ACKs but it feeds back
>       >         the timing and the order in which ECN marks are received with minimal
>       >         additional complexity. If CE marks are infrequent, as is the case for
>       >         most AQMs at the time of writing, or there are multiple marks in a row,
>       >         the additional load will be low.
>       >
>       >       For example, consider a scenario with bidirectional traffic between A and B where B has
>       >       a hole in sequence resulting in every data packet in the current RTT to become acked
>       >       (pure duplicate Acks). This may result in a packet flow from A to B where every second
>       >       packet is a pure (duplicate) Ack. If there is congestion on the path from A to B such
>       >       that a significant number of (data) packets get marked, it may result in acking every
>       >       data packet from A to B. This does not necessarily result in low additional load?
>       >
>       >
>       > [BB] I don't quite understand how every second packet from A to B is a pure duplicate ACK, but I
>       > don't think I need to - I'll assume it's somehow possible.
>       >
>       > Then I think you've somehow assumed that the data packets get CE-marked, but the interspersed pure
>       > ACKs don't (perhaps you're assuming that the pure ACKs in this case are not ECN-capable? Or perhaps
>       > you're assuming size-based packet marking?). Whatever, I agree that, if this scenario did occur,
>       > then the change-triggered ACK rule would indeed lead to B ACKing every data packet that arrives,
>       > with no delayed ACKs. {Note 1}
>       >
>       > Nonetheless, the draft is quite open about the implications of the change-triggered ACK rule on ACK
>       > rate. In the sentence straight after the ones you quote, it says:
>       >     "However, marking patterns with numerous non-contiguous CE marks could increase the load
>       > significantly."
>       > And a little earlier it starts out by saying:
>       >     "...the 'Change-Triggered ACKs' rule could sometimes cause the ACK rate to be problematic for
>       > high performance"
>       >
>       > I don't think we need to include examples of how non-contiguous CE marking could occur. And if we
>       > did, I'd prefer to use one that was less complex to explain, e.g. a high level of probabilistic AQM
>       > marking. But thank you for this point anyway.
>
>       [MK] Thanks for pointing these additional sentences. I somehow missed
>       them and/or did not manage to relate them to the text I quoted. I think
>       this is good enough to address the case I raised, particularly "numerous
>       non-contiguous CE marks could increase the load significantly."
>
>       > {Note 1}: It's ironic that the existing behaviour "where B has a hole in sequence" also results "in
>       > every data packet in the current RTT to become acked" (by A). I'm not giving this as an excuse for
>       > introducing another case with the same bad behaviour. I'm just highlighting the irony.
>
>       [MK] Maybe ironic but the fact that a hole in sequence results in every
>       data packet in RTT to become acked has a very good reason being as it is
>       because those dupAcks directly control the data rate per the packet
>       conservation principle in various important algos at the data sender,
>       such as Fast Recovery, and this behaviour is therefore a crucial part of
>       such congestion control algos.
>
>       Best regards,
>
>       /Markku
>
>       >
>       > Regards
>       >
>       >
>       > Bob
>       >
>       >
>       >       Best regards,
>       >
>       >       /Markku
>       >
>       >
>       >             Best regards
>       >             Michael
>       >
>       >                   Cheers
>       >
>       >
>       >                   Bob
>       >
>       >                   On 30/03/2023 15:15, [e-mail] wrote:
>       >                         A New Internet-Draft is available from the on-line
>       >                         Internet-Drafts
>       >                         directories. This Internet-Draft is a work item of
>       >                         the TCP Maintenance and
>       >                         Minor Extensions (TCPM) WG of the IETF.
>       >
>       >                            Title           : More Accurate Explicit
>       >                         Congestion Notification (ECN) Feedback in TCP
>       >                            Authors         : Bob Briscoe
>       >                                              Mirja Kühlewind
>       >                                              Richard Scheffenegger
>       >                            Filename        :
>       >                         draft-ietf-tcpm-accurate-ecn-24.txt
>       >                            Pages           : 64
>       >                            Date            : 2023-03-30
>       >
>       >                         Abstract:
>       >                            Explicit Congestion Notification (ECN) is a
>       >                         mechanism where network
>       >                            nodes can mark IP packets instead of dropping
>       >                         them to indicate
>       >                            incipient congestion to the endpoints.  Receivers
>       >                         with an ECN-capable
>       >                            transport protocol feed back this information to
>       >                         the sender.  ECN was
>       >                            originally specified for TCP in such a way that
>       >                         only one feedback
>       >                            signal can be transmitted per Round-Trip Time
>       >                         (RTT).  Recent new TCP
>       >                            mechanisms like Congestion Exposure (ConEx), Data
>       >                         Center TCP (DCTCP)
>       >                            or Low Latency, Low Loss, and Scalable Throughput
>       >                         (L4S) need more
>       >                            accurate ECN feedback information whenever more
>       >                         than one marking is
>       >                            received in one RTT.  This document updates the
>       >                         original ECN
>       >                            specification in RFC 3168 to specify a scheme
>       >                         that provides more than
>       >                            one feedback signal per RTT in the TCP header. 
>       >                         Given TCP header
>       >                            space is scarce, it allocates a reserved header
>       >                         bit previously
>       >                            assigned to the ECN-Nonce.  It also overloads the
>       >                         two existing ECN
>       >                            flags in the TCP header.  The resulting extra
>       >                         space is exploited to
>       >                            feed back the IP-ECN field received during the
>       >                         3-way handshake as
>       >                            well.  Supplementary feedback information can
>       >                         optionally be provided
>       >                            in two new TCP option alternatives, which are
>       >                         never used on the TCP
>       >                            SYN.  The document also specifies the treatment
>       >                         of this updated TCP
>       >                            wire protocol by middleboxes.
>       >
>       >                         The IETF datatracker status page for this
>       >                         Internet-Draft is:
>       >                         https://datatracker.ietf.org/doc/draft-ietf-tcpm-accurate-ecn/
>       >
>       >                         There is also an htmlized version available at:
>       >                         https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accurate-ecn-24
>       >
>       >                         A diff from the previous version is available at:
>       >                         https://author-tools.ietf.org/iddiff?url2=draft-ietf-tcpm-accurate-ecn-24
>       >
>       >                         Internet-Drafts are also available by rsync at
>       >                         rsync.ietf.org::internet-drafts
>       >
>       >
>       >                         _______________________________________________
>       >                         tcpm mailing list
>       >                         [e-mail]
>       >                         https://www.ietf.org/mailman/listinfo/tcpm
>       >
>       >
>       >                   --
>       >                   ________________________________________________________________
>       >                   Bob Briscoe                               http://bobbriscoe.net/
>       >
>       >
>       >
>       >
>       > --
>       > ________________________________________________________________
>       > Bob Briscoe                               http://bobbriscoe.net/
>       >
>       >
> 
> 
>