Re: [tcpm] draft-ietf-tcpm-accurate-ecn-24 addressing all WGLC comments

[[e-mail]]

> On 19. Jun 2023, at 12:38, Bob Briscoe (IC) <[e-mail]> wrote:
> 
> Michael,
> 
> The rule in RFC2026 is "Standards track specifications normally must not depend on other standards track specifications which are at a lower maturity level or on non standards track specifications". My thinking for making SACK and ECN++ informative refs was that recommendation in itself does not create a dependency.
This is also my understanding of what is the plan.
> The more clear-cut test is that the AccECN spec is defined for the case without ECN++ and without SACK, so I think that means it doesn’t depend on them.
OK. So can you use a term different from "RECOMMENDED"?
> 
> 
> Nonetheless, following a promise to Markku (also on the list), I have added the following to the local copy of AccECN after this ref to ECN++:
>     "...or any equivalent future protocol that allows the ECN capability to be used on TCP control packets"
> And I've similarly generalized all the refs to ECN++.
> 
> Does that resolve this concern?
My concern is about the wording similar to

It is RECOMMENDED that AccECN is implemented alongside [I-D.ietf-tcpm-generalized-ecn].

RECOMMENDED is like SHOULD and therefore I'm not sure the reference can be Informative.
I can double check with the AD on Wednesday...

Best regards
Michael
> 
> 
> Bob
> 
>> On 18 Jun 2023, at 22:03, [e-mail] wrote:
>> 
>>> On 5. Jun 2023, at 10:33, Yoshifumi Nishida <[e-mail]> wrote:
>>> 
>>> Hi folks,
>>> 
>>> I just would like to put my personal views on ack on ack discussions here.
>>> First, I think ack on ack has already has some precedences. 
>>> keep-alive logic can sent a pure ack for proving and expect an ack for it. 
>>> MPTCP uses 4 WHS and the 4th segment is an ack for the third ACK which is a pure ACK.
>>> So, I'm not sure if we need to define some rules for ack-on-ack in the doc.
>>> 
>>> Also, I'm a bit hesitant to define a detailed logic about how to distinguish an ack that carries ECN signals and a dup ack, such as using TS or SACK blocks.
>>> I think such things are the part of experiments and should be described in other docs such as ECN++ or ackcc, etc. 
>>> I personally prefer the doc simply describes the possibilities of such mechanisms and provides general principles and guidelines. 
>>> 
>>> As far as I think, the Markku's examples that triggers false retransmissions makes sense. I think they are good examples to show how ack-on-ack can be tricky.
>>> However, these examples are the case where both sides exchanges data simultaneously and I think there're other cases where we don't have to worry about it. 
>>> For example, I think bulk transfer or request-response type traffic can be these examples. 
>>> In these cases, the endpoint which receives acks for acks doesn't have outstanding data. Hence, although these acks are duplicate acks, they won't trigger retransmissions.
>>> 
>>> So, I am thinking that it would be good to provide a certain guideline about when to enable this feature and potential risks in some docs.
>>> But, I am also thinking we should do it outside of accecn doc.
>> I agree that the ACK of ACK scenario is something which relates to the
>> experimental ECN++ document.
>> 
>> However, I checked again draft-ietf-tcpm-accurate-ecn-24 and found at the
>> paragraph right before Section 1.1:
>> 
>> It is RECOMMENDED that the AccECN protocol is implemented alongside
>> SACK [RFC2018] and the experimental ECN++ protocol
>> [I-D.ietf-tcpm-generalized-ecn], which allows the ECN capability to
>> be used on TCP control packets.
>> 
>> Would using I-D.ietf-tcpm-generalized-ecn in combination with RECOMMENDED
>> not required the ID to be a normative reference? I couldn't find clear rules,
>> but in my view I could not argument against it. Since the intended status
>> of I-D.draft-ietf-tcpm-accurate-ecn is PS and the intended status of
>> I-D.ietf-tcpm-generalized-ecn is Experiemental, this would be a downref.
>> 
>> Is this really intended by the authors or just a leftover?
>> 
>> Best regards
>> Michael
>>> 
>>> Thanks,
>>> --
>>> Yoshi
>>> 
>>> 
>>> On Fri, May 26, 2023 at 2:56 PM Markku Kojo <[e-mail]> wrote:
>>> Bob,
>>> 
>>> My apologies you had to wait for the scenarios as it took much longer 
>>> with my limited cycles than I thought. Anyways, please see my reply to 
>>> Richard, some scenarios are also included there.
>>> 
>>> To keep things easier, it might be good to try to keep the discussion on 
>>> Acks of Acks (mainly) in the thread with my reply to Richard.
>>> 
>>> However, see inline tagged [MK].
>>> 
>>> On Wed, 24 May 2023, Bob Briscoe wrote:
>>> 
>>>> Markku,
>>>> 
>>>> Sorry, it's taken a week to build a comprehensive reply to this long email. See inline tagged
>>>> [BB]...
>>>> 
>>>> On 17/05/2023 12:24, Markku Kojo wrote:
>>>>      Hi Michael, all,
>>>> 
>>>>      On Sun, 14 May 2023, [e-mail] wrote:
>>>> 
>>>>                  On 30. Mar 2023, at 16:53, Bob Briscoe <[e-mail]>
>>>>                  wrote:
>>>> 
>>>>                  Michael, Yoshi, Ian (as tcpm chairs),
>>>> 
>>>>                  To close off the WGLC, I have just posted a new rev of
>>>>                  accurate-ecn. Hyperlinks quoted at the end.
>>>>                  You will see the diff is rather extensive. I won't give a
>>>>                  summary of all the diffs like I usually do. Instead I can just
>>>>                  refer to the summary I gave in the presentation on Monday:
>>>>                  https://datatracker.ietf.org/meeting/116/materials/slides-116-tcpm-draft-ietf-tcpm-accurate-ecn
>>>> 
>>>>                  Thank you again to the people who reviewed this during the WGLC:
>>>>                  Michael Tüxen, Alex Burr, Gorry Fairhurst and Markku Kojo.
>>>> 
>>>>                  All changes are editorial, apart from removing the para about
>>>>                  not mistaking certain ACKs of ACKs for DupACKs, which I will add
>>>>                  to a rev of the ECN++ draft, hopefully later this week.
>>>> 
>>>>                  On the list, we have seen agreement from all the reviewers to
>>>>                  these changes, except no response from Markku yet.
>>>>                  On Monday, I told Markku that I would post the draft in a few
>>>>                  days, so everyone can see the updates and diff.
>>>> 
>>>>            Anyone having additional comments? In particular Markku regarding loss
>>>>            recovery?
>>>> 
>>>> 
>>>>      My apologies for being late with my reply to the author's comments on my review (I've
>>>>      been extremly busy with other issues since the wg mtng in Yokohama, including the rest
>>>>      of mtng week).
>>>> 
>>>>      I don't have much new comments but it seems that my major concern regarding the problem
>>>>      of sending ACKs of ACKs was not fully understood.
>>>> 
>>>>      The first thing where I think I was not quite clear is that the major problem with ACKs
>>>>      of ACKs is not that a pure Ack is made ECN-capable. Instead, the problem is in
>>>>      generating an Ack of an pure Ack and that is what one should prohibit to avoid problems.
>>>>      I understand that it might be problematic to formulate rules whether generating an Ack
>>>>      of an Ack is allowed (and when), instead of just disabling sending ECN-capable ACKs.
>>>>      I don't have a strong opinion which way the problems with ACKs of ACKs is avoided as
>>>>      long as they are avoided.
>>>> 
>>>> 
>>>> [BB] See later after your similar point (following your 'Why?' heading)...
>>>> 
>>>> 
>>>>      I am preparing a few scenarios to illustriate the problems ACKs of ACks raise and will
>>>>      send them shortly once I have formulated a more thorough reasoning why sending ACKs of
>>>>      ACKs is not really a good idea and even seems to be unnecessary in most if not all
>>>>      cases, i.e., it just results in sending unnecessary packets with not much useful effect
>>>>      but creates a notable number of problems.
>>>> 
>>>> 
>>>> [BB] Having waited this long, it's rather disappointing to still hear you say "I have an argument,
>>>> but I'll tell you later."
>>> 
>>> [MK] I understand. My sincere apologies again.
>>> 
>>>>      It also seems not have been carefully enough considered in terms of the very basic
>>>>      rubustness principle of "be conservative in what you send ..."
>>>> 
>>>> 
>>>> [BB] The WG has been careful to ensure that ACKs of ACKs are unambiguous (cannot be mistaken for a
>>>> DupACK), which is what the robustness principle requires. It's just that you think we have missed
>>>> cases where they will be ambiguous. If you think that, we need to hear them all. 
>>>> 
>>>> The robustness principle does not advocate sending nothing just in case some unknown factor might
>>>> make it ambiguous. Especially given /not/ feeding back congestion notifications has potential to
>>>> cause harm to others. Also, "no feedback" is much more ambiguous.
>>> 
>>> [MK] Please see my reply to Richard and let's continue from there. See 
>>> also what I meant with "be conservative in what you send ...", that is, 
>>> in context of CC: avoid sending unnecessary packets or be careful in 
>>> sending packets just because they might be sometimes useful, send the 
>>> monly when they are useful.
>>> 
>>>>      Given that this draft is intended to become a stds track RFC I am concerned of any text
>>>>      in this document that indicates (or even hints) that TCP could acknowledge pure ACKS
>>>>      (this holds particularly the rules and text in Sec 3.2.2.5.1 for the
>>>>      "Increment-Triggered ACKs"). If it is seen necessary that this doc should have such
>>>>      pieces of rules and text, I am fine if any such text is moved to an appendix as long as
>>>>      the appendix makes it cristal clear that the text is valid only in case one is
>>>>      implementing an experiment as per [I-D.ietf-tcpm-generalized-ecn].
>>>> 
>>>> 
>>>> [BB] See point below about "Generic (Mechanistic) Reflector".
>>>> 
>>>> 
>>>>      Why?
>>>> 
>>>>      1) It is well known that TCP does not acknowledge ACKs and Standards track TCP has not
>>>>      been specified to acknowledge ACKs. This means that a reader/implementer of this doc
>>>>      cannot correctly understand the rule for "Increment-Triggered ACKs" unless there is a
>>>>      normative reference to a spec that specifies ACKs of ACKs (or tells that it is even
>>>>      possible).
>>>> 
>>>> 
>>>> [BB] ACKs of ACKs can indeed be tricky. But there's no need to consider not ACKing ACKs as an
>>>> architectural principle. Not Acking ACKs on principle certainly avoids some tricky problems.
>>>> However, we have a new situation here where, in limited circumstances, ACKs of ACKs are necessary.
>>>> So the WG has already worked through the tricky problems and they have been addressed in the draft
>>>> (e.g. mistaking ACKs of ACKs for DupACks, infinite ping-pong, etc). We'll discuss below whether
>>>> you've found some more trickiness.
>>> 
>>> (MK] I did not mean to refer to any principle but, as I said, that a 
>>> reader/implementer cannot correctly understand the rule for 
>>> "Increment-Triggered ACKs" because it is well-known to her/him that TCP 
>>> does not Ack ACKs. This fact is that one can ack ACKs is not specified in 
>>> this doc nor does this doc give a (normative) reference where it is 
>>> specified, including the details on which TSecr value to add or which 
>>> SACK info if any to include when acking a pure Ack. It is easy to 
>>> misinterpret the "Increment-Triggered ACKs", if one doesn't realize that 
>>> pure Acks may be acked.
>>> 
>>>> What is the new situation?
>>>> *  Until ECN was introduced, TCP ACKs only acknowledged data. So there was no need to acknowledge
>>>>    pure ACKs, which contain no data.
>>>> *  When ECN was introduced in RFC3168, TCP ACKs also acknowledged ECN markings. However, because
>>>>    RFC3168 precluded pure ACKs from being ECN-capable, there was still no need to acknowledge pure
>>>>    ACKs.
>>>> *  RFC5690, and now the ECN++ draft introduce the possibility of ECN-capable pure ACKs. So, in the
>>>>    limited circumstances described in the AccECN draft, ECN-capable pure ACKs now need to be
>>>>    acknowledged, because they contain new information - their ECN field.
>>>> Similarly, even though the final ACK of TCP's 3WHS is an ACK of an ACK , it is sent because it is
>>>> needed (to prove that the SYN wasn't from a spoofed address).
>>> 
>>> [MK] All otherwise clear, but I disagree that the final ACK of TCP's 3WHS 
>>> is an ACK of an ACK. It is required because SYNACK contains control data 
>>> that eats one sequence number, i.e., it advances RCV.NXT at the client 
>>> end and when the ACK arrives at the server it is needed to advance 
>>> SND.UNA. Very different from Acks of Acks in this draft.
>>> 
>>>> It is true that not ACKing ACKs is well-known. However, whether it's well-known as a /principle/, or
>>>> just as a current /feature/ of TCP is not clear. Anyway, the IETF's job is to update RFCs that are
>>>> "well-known". We don't have to jump through any special procedural hoops to do something different
>>>> from what is "well-known". Even if it were prohibited in a stds track RFC, we just have to specify
>>>> what has to be done instead; in another stds track RFC.
>>> 
>>> [MK] Again, I didn't mention it as a /principle/ but as a crucial 
>>> piece of information that the reader needs to be noted, that is, the 
>>> things are now different from what is well-known.
>>> 
>>> Sure IETF's job is to update RFCs, but if one changes what is prohibited 
>>> in a stds track RFC, one needs to understand the consequences and explain 
>>> them as well as give the justification why the change can be done (without 
>>> problems), instead of just specifying the change.
>>> 
>>>> If there are any tutorials, course notes or text books out there that say that not ACKing ACKs is a
>>>> well-known principle, that's not the IETF's problem. It is the job of the tutors, lecturers and text
>>>> book authors who wrote those materials to update them.
>>>> 
>>>> 
>>>>      2) ACKs of ACKs tend to trigger duplicate Acks. There are tons of algorithms that rely
>>>>      on the packet conservation principle and the fact that TCP never injects a dupAck unless
>>>>      a *data* packet has arrived and left the network. This is enforced with "MUST NOT" in
>>>>      RFC 5681, Sec 4.2, because not conforming to this rule makes any algorithm that rely on
>>>>      the rule to work incorrectly. These algorithms include (triggering) Fast Retransmit,
>>>>      (controlling packet rate during) Fast Recovery, (detecting spurious RTOs in) F-RTO,
>>>>      (calculating PipeAck in) RFC 7661, (calculating DeliveredData in) PRR, etc. Furthermore,
>>>>      it would make imposible to come up with any new algorihms that rely on this important
>>>>      basic rule. In most cases such extra dupAcks make these algorithms too aggressive
>>>>      because any extra dupAck is likely to inject extra packet(s) to the network.
>>>> 
>>>>      So, it should be cristal clear that without SACK (or Timestamps) a TCP *MUST NOT* send
>>>>      ACKs of ACKs.
>>>> 
>>>> 
>>>> [BB] Constraining the /Data Receiver/ as you propose would create an interop problem.
>>>> Explanation: Consider host A and B are not using SACK or timestamps. Nonetheless, with your
>>>> approach, host A can still send ECN-capable pure ACKs to host B. Then, your rule puts host B in an
>>>> impossible position, where it gets congestion notifications on ECN-capable pure ACKs, but it is not
>>>> allowed to send any feedback about them.
>>>> 
>>>> Instead, if neither timestamps nor SACK are in use for the connection, we need to constrain the
>>>> /Data Sender/ of a half connection from sending ECN-capable ACKs in the first place. This is the
>>>> approach the WG has adopted in the AccECN and ECN++ specs.
>>> 
>>> [MK] I think I said in the beginning that I have no strong opinion which 
>>> way Acks of Acks are disabled. However, I apologize that I didn't explain 
>>> why I phrased MUST NOT send ACKs of ACKs. This is because it might be 
>>> still useful to allow CE-marked pure Acks and take care of Ack CC by some 
>>> other means than Acks of Acks. Currently the draft mandates Acks of Acks 
>>> as the only way to report Ack congestion and I think it is too 
>>> restrictive in a stds track doc, e.g., it rules out reducing Ack rate 
>>> simply by reducing data send rate which would solve the interop problem in 
>>> a very simple way. Moreover, When B gets congestion notifications on 
>>> ECN-capable pure ACKs, not sending Acks of Acks does not prevent sending 
>>> feedback; such feedback need not to be delivered immediately but by the 
>>> time needed. Please see more on this in my reply to Richard.
>>> 
>>>> Specifically:
>>>> *  The WG makes sure that RFCs about the /Data Sender/ of a half connection (e.g. the ECN++
>>>>    experiment or other future RFCs) specify that sending ECN-capable pure ACKs is conditional on
>>>>    having another way to distinguish DupACKs, e.g. negotiating SACK or timestamps (and I will
>>>>    respond to your later points on the details of these).
>>>> *  The AccECN spec (which primarily specifies the feedback behaviour of a /Data Receiver/ in a
>>>>    half-connection) then only needs to define the Increment-triggered ACK rule.
>>>> The two together lead to the same outcome you want. But without the interop hole of your approach.
>>>> 
>>>> This is consistent with the "Generic (Mechanistic) Reflector" approach of the AccECN spec which
>>>> says:
>>>> "AccECN is designed to be a generic reflector of whatever ECN markings it sees, whether or not they
>>>> are compliant with a current standard."
>>>> 
>>>> These ACKs of ACKs are generically necessary to feed back congestion notifications from possible
>>>> incoming packet patterns, not specifically for ECN++ or AckCC [RFC5690], or any other future RFC
>>>> (forward compatibility). We'll edit the reference to ECN++ to make it clearer that it's one example,
>>>> not the only case.
>>>> 
>>>> Here's another example of the generic reflector approach, already in the draft:
>>>> "Although RFC 3168 prohibits an ECN-capable SYN, providing feedback of ECN marking on the SYN
>>>> supports future scenarios in which SYNs might be ECN-enabled (without prejudging whether they ought
>>>> to be). ... "
>>>> 
>>>> 
>>>>      So, it should be cristal clear that without SACK (or Timestamps) a TCP *MUST NOT* send
>>>>      ACKs of ACKs.
>>>> 
>>>> 
>>>> I understand that you want this but, as just explained, without SACK or timestamps, the correct
>>>> approach is to prevent the Data Sender putting the Data Receiver in the position where it would have
>>>> to ACK ACKs in the first place.
>>>> 
>>>> In a connection without SACK or timestamps, if the Data Receiver were to get lots of congestion
>>>> notifications on ECN-capable ACKs, it would face a difficult dilemma. Which would be more important:
>>>> Signalling congestion by ACKing ACKs? or ensuring the performance improvements like Fast Retransmit,
>>>> Fast Recovery etc. work well? The former would prevent harm to others, the latter would prevent harm
>>>> to self.
>>> 
>>> [MK] Please see my reply to Richard to understand why Acks of Acks 
>>> cause Fast Retransmit, Fast Recovery, etc to cause harm to others in the 
>>> first place. And also why most of the Acks of Acks seem to be just 
>>> unnecessary load to the network, that is, they harm others without (much) 
>>> benefits.
>>> 
>>>> Nonetheless, I will add some text to the AccECN draft that explains why it is important for other
>>>> RFCs not to put a Data Receiver in the position where it has to ACK ACKs iff there is no way to
>>>> distinguish them from DupACKs.
>>>> 
>>>> 
>>>>      3) Even with SACK or Timestamps enabled it is not clear what an
>>>>      implementer should do. With SACK the AckECN authors seem to make an assumption, which
>>>>      seems obvious but is not, that an ACK of ACK would would never include SACK option and
>>>>      hence it could be distinguished from a dupAck. However, RFC 2018 specifies: "If sent at
>>>>      all, SACK options SHOULD be included in all ACKs which do not ACK the highest sequence
>>>>      number in the data receiver's queue. So, if there is a hole in the receiver's queue, the
>>>>      assumption is incorrect and it is unclear which SACK info to include into the SACK
>>>>      option. Whatever one selects to include, it makes DSACK (RFC 2883] void and breaks any
>>>>      DSACK-based algorithms unless RFC 2018 is updated.
>>>> 
>>>> 
>>>> [BB]
>>>> Reading the draft, it is very clear that there is no such assumption. The text said solely what it
>>>> meant:
>>>> 
>>>>   ... a host in AccECN
>>>>   mode that is sending ECN-capable pure ACKs SHOULD add one of the
>>>>   following additional checks when it tests whether an incoming pure
>>>>   ACK is a duplicate:
>>>> 
>>>>   o  If SACK has been negotiated for the connection, but there is no
>>>>      SACK option on the incoming pure ACK, it is not a duplicate;
>>>> That is, if an incoming ACK were a duplicate, it would have a SACK option on it. This /relies/ on
>>>> the rule in RFC2018 that you quote. So there is no need (nor intention) to change SACK behaviour in
>>>> any way.
>>> 
>>> [MK] Sorry I don't understand your claim. Assume A sends data pkts 
>>> 0,1,2,3,..,10 in the current window to B and pkt 1 is dropped. 
>>> Simultaneously, B sends data to A such that the data pkts arrive at A 
>>> after A has injected pkt 10 to the network. These pkts trigger pure 
>>> cumulative Acks from A to B that follow A's data pkts 1..10 and enough of 
>>> the Acks get CE-marked due to congestion path A to B. When pkts 2..10 
>>> arrive at B, each of them trigger a valid dupAck with a SACK block 
>>> included. When the CE-marked Acks that follow data pkts 2..10 arrive at 
>>> B, B needs to feedback congestion info on them in Acks of Acks. These 
>>> Acks of Acks cannot cumulatively ACK the highest sequence number in the 
>>> data receiver's queue (pkt 10) since pkt 0 is the highest pkt 
>>> arrived insequence, so the Ack of Ack must include a SACK block as per 
>>> RFC 2018. What is the SACK block info that the implementer, who follows 
>>> carefully advice in RFC 2018 and there is no other advice, should 
>>> include in these Acks unless RFC 2018 is not changed? How does the 
>>> implementer know how to Ack, if it not specified?
>>> 
>>>> (Note: to check this text, you'll need to refer to the previous AccECN draft here:
>>>>    https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accurate-ecn-23#section-3.2.2.5.1
>>>> It had been there since Jul 2021 (since -15). But, as explained above, it is longer in the latest
>>>> AccECN draft (-24), because it has been moved to the editor's copy of the ECN++ draft, which I wrote
>>>> on 4 Apr, but I've been waiting for your reply before submitting it.)
>>>> 
>>>> 
>>>>      With Timestamps some algorithms like Eifel detection breaks. Moreover, there are other
>>>>      existing or potentially to-be-created heauristics, including various measurement tools,
>>>>      that rely on the fact that TCP does not echo a later Timestamp in a pure Ack than what
>>>>      arrived with the latest data packet. Any such mechanisms are subject to break.
>>>> 
>>>> 
>>>> [BB] I don't fully understand what you're saying here. Can you be clearer please?
>>>> And can you please bear in mind that we are in the WGLC processing stage now. So review comments
>>>> ought to be suggesting very specific changes to the text under review.
>>> 
>>> [MK] Again, the same problem as with SACK. It is unspecified which 
>>> TSecr value to put in Acks of Acks. That has not been specified 
>>> anywhere, so how can I or anyone else check what breaks if anything, and 
>>> how can an implementer know which TSecr value to include in Acks of Acks?
>>> 
>>> It is hard to propose specific changes to text that does not exist or to a 
>>> problem that seems to be a missing piece of design or a design flaw, until 
>>> the text exists or the intended design is known or the potentiel design 
>>> flaw is mutually agreed whether there is a flaw or not.
>>> 
>>> (Please note also that the Eifel problem seems not to be serious, but 
>>> instead the timestamps rule you proposed to distinguish dupAcks from Acks 
>>> of Acks seems suspicious and calls for clarification).
>>> 
>>> Please see more details in my reply to Richard.
>>> 
>>>> Again, this text about extra DupACK checks has now been removed from AccECN and will shortly appear
>>>> in the ECN++ draft instead. I shall post the new ECN++ draft shortly.
>>>> 
>>>> 
>>>>      It might be good to hold discussing any details on what breaks and how/why and what are
>>>>      the consequences until I have sent my reply with scenarios to Richard.
>>>> 
>>>> 
>>>> [BB] Please try to prioritize any comments about the text that is now left in the AccECN draft. The
>>>> WGLC of AccECN is waiting for no-one else at the moment.
>>> 
>>> [MK] Unfortunately I was not able to do this, sorry. I got confused 
>>> already earlier when I was reviewing AccECN and ended up checking ECN++ 
>>> quickly as I noted that AccECN draft cited it for these issues. When 
>>> reading ECN++ I found Sec 3.3.3 and read:
>>> 
>>>  "The question of whether and how the receiver of pure ACKs is required to
>>>   feed back any CE marks on them is outside the scope of the present
>>>   specification because it is a matter for the relevant feedback
>>>   specification ([RFC3168] or [I-D.ietf-tcpm-accurate-ecn])."
>>> 
>>> This got me totally confused together with the later discussion on the mic 
>>> at Yokohama mtng because that was what I had read, and I was not able to 
>>> understand what was intended to go where. And I still not quite know 
>>> but I think I have a better hunch. The above text still reads in 
>>> ECN++ draft, but hopefully not in editor's copy?
>>> 
>>> Anyways, the major problem is not with any certain text phrases but 
>>> whether DupAck vs. Ack of Ack problem is solvable and whether 
>>> injecting Acks of Acks really is a needed and mature enough feature that 
>>> can be part of a stds track protocol.
>>> 
>>>> But also bear in mind that the chairs plan to take ECN++ into WGLC once AccECN WGLC has been
>>>> cleared. So we need to hear your actual argument about the DupACK text that has been moved to ECN++
>>>> urgently too. We can't work with "I have an argument, but I'll tell you later".
>>> 
>>> [MK] Apologies for the delay again. Please see my reply to Richard for my 
>>> arguments.
>>> 
>>>>      One additional comment regarding the "Change-Triggered ACKs" rule is that it would be
>>>>      useful to make it more clear how this plays with delayed Acks and how it alters
>>>>      acknowledgement rate.
>>>>      I am not sure that what the draft currently says is quite correct:
>>>> 
>>>>       "The approach can lead to some additional ACKs but it feeds back
>>>>        the timing and the order in which ECN marks are received with minimal
>>>>        additional complexity. If CE marks are infrequent, as is the case for
>>>>        most AQMs at the time of writing, or there are multiple marks in a row,
>>>>        the additional load will be low.
>>>> 
>>>>      For example, consider a scenario with bidirectional traffic between A and B where B has
>>>>      a hole in sequence resulting in every data packet in the current RTT to become acked
>>>>      (pure duplicate Acks). This may result in a packet flow from A to B where every second
>>>>      packet is a pure (duplicate) Ack. If there is congestion on the path from A to B such
>>>>      that a significant number of (data) packets get marked, it may result in acking every
>>>>      data packet from A to B. This does not necessarily result in low additional load?
>>>> 
>>>> 
>>>> [BB] I don't quite understand how every second packet from A to B is a pure duplicate ACK, but I
>>>> don't think I need to - I'll assume it's somehow possible.
>>>> 
>>>> Then I think you've somehow assumed that the data packets get CE-marked, but the interspersed pure
>>>> ACKs don't (perhaps you're assuming that the pure ACKs in this case are not ECN-capable? Or perhaps
>>>> you're assuming size-based packet marking?). Whatever, I agree that, if this scenario did occur,
>>>> then the change-triggered ACK rule would indeed lead to B ACKing every data packet that arrives,
>>>> with no delayed ACKs. {Note 1}
>>>> 
>>>> Nonetheless, the draft is quite open about the implications of the change-triggered ACK rule on ACK
>>>> rate. In the sentence straight after the ones you quote, it says:
>>>>    "However, marking patterns with numerous non-contiguous CE marks could increase the load
>>>> significantly."
>>>> And a little earlier it starts out by saying:
>>>>    "...the 'Change-Triggered ACKs' rule could sometimes cause the ACK rate to be problematic for
>>>> high performance"
>>>> 
>>>> I don't think we need to include examples of how non-contiguous CE marking could occur. And if we
>>>> did, I'd prefer to use one that was less complex to explain, e.g. a high level of probabilistic AQM
>>>> marking. But thank you for this point anyway.
>>> 
>>> [MK] Thanks for pointing these additional sentences. I somehow missed 
>>> them and/or did not manage to relate them to the text I quoted. I think 
>>> this is good enough to address the case I raised, particularly "numerous 
>>> non-contiguous CE marks could increase the load significantly."
>>> 
>>>> {Note 1}: It's ironic that the existing behaviour "where B has a hole in sequence" also results "in
>>>> every data packet in the current RTT to become acked" (by A). I'm not giving this as an excuse for
>>>> introducing another case with the same bad behaviour. I'm just highlighting the irony.
>>> 
>>> [MK] Maybe ironic but the fact that a hole in sequence results in every 
>>> data packet in RTT to become acked has a very good reason being as it is 
>>> because those dupAcks directly control the data rate per the packet 
>>> conservation principle in various important algos at the data sender,
>>> such as Fast Recovery, and this behaviour is therefore a crucial part of 
>>> such congestion control algos.
>>> 
>>> Best regards,
>>> 
>>> /Markku
>>> 
>>>> 
>>>> Regards
>>>> 
>>>> 
>>>> Bob
>>>> 
>>>> 
>>>>      Best regards,
>>>> 
>>>>      /Markku
>>>> 
>>>> 
>>>>            Best regards
>>>>            Michael
>>>> 
>>>>                  Cheers
>>>> 
>>>> 
>>>>                  Bob
>>>> 
>>>>                  On 30/03/2023 15:15, [e-mail] wrote:
>>>>                        A New Internet-Draft is available from the on-line
>>>>                        Internet-Drafts
>>>>                        directories. This Internet-Draft is a work item of
>>>>                        the TCP Maintenance and
>>>>                        Minor Extensions (TCPM) WG of the IETF.
>>>> 
>>>>                           Title           : More Accurate Explicit
>>>>                        Congestion Notification (ECN) Feedback in TCP
>>>>                           Authors         : Bob Briscoe
>>>>                                             Mirja Kühlewind
>>>>                                             Richard Scheffenegger
>>>>                           Filename        :
>>>>                        draft-ietf-tcpm-accurate-ecn-24.txt
>>>>                           Pages           : 64
>>>>                           Date            : 2023-03-30
>>>> 
>>>>                        Abstract:
>>>>                           Explicit Congestion Notification (ECN) is a
>>>>                        mechanism where network
>>>>                           nodes can mark IP packets instead of dropping
>>>>                        them to indicate
>>>>                           incipient congestion to the endpoints.  Receivers
>>>>                        with an ECN-capable
>>>>                           transport protocol feed back this information to
>>>>                        the sender.  ECN was
>>>>                           originally specified for TCP in such a way that
>>>>                        only one feedback
>>>>                           signal can be transmitted per Round-Trip Time
>>>>                        (RTT).  Recent new TCP
>>>>                           mechanisms like Congestion Exposure (ConEx), Data
>>>>                        Center TCP (DCTCP)
>>>>                           or Low Latency, Low Loss, and Scalable Throughput
>>>>                        (L4S) need more
>>>>                           accurate ECN feedback information whenever more
>>>>                        than one marking is
>>>>                           received in one RTT.  This document updates the
>>>>                        original ECN
>>>>                           specification in RFC 3168 to specify a scheme
>>>>                        that provides more than
>>>>                           one feedback signal per RTT in the TCP header. 
>>>>                        Given TCP header
>>>>                           space is scarce, it allocates a reserved header
>>>>                        bit previously
>>>>                           assigned to the ECN-Nonce.  It also overloads the
>>>>                        two existing ECN
>>>>                           flags in the TCP header.  The resulting extra
>>>>                        space is exploited to
>>>>                           feed back the IP-ECN field received during the
>>>>                        3-way handshake as
>>>>                           well.  Supplementary feedback information can
>>>>                        optionally be provided
>>>>                           in two new TCP option alternatives, which are
>>>>                        never used on the TCP
>>>>                           SYN.  The document also specifies the treatment
>>>>                        of this updated TCP
>>>>                           wire protocol by middleboxes.
>>>> 
>>>>                        The IETF datatracker status page for this
>>>>                        Internet-Draft is:
>>>>                        https://datatracker.ietf.org/doc/draft-ietf-tcpm-accurate-ecn/
>>>> 
>>>>                        There is also an htmlized version available at:
>>>>                        https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accurate-ecn-24
>>>> 
>>>>                        A diff from the previous version is available at:
>>>>                        https://author-tools.ietf.org/iddiff?url2=draft-ietf-tcpm-accurate-ecn-24
>>>> 
>>>>                        Internet-Drafts are also available by rsync at
>>>>                        rsync.ietf.org::internet-drafts
>>>> 
>>>> 
>>>>                        _______________________________________________
>>>>                        tcpm mailing list
>>>>                        [e-mail]
>>>>                        https://www.ietf.org/mailman/listinfo/tcpm
>>>> 
>>>> 
>>>>                  --
>>>>                  ________________________________________________________________
>>>>                  Bob Briscoe                               http://bobbriscoe.net/
>>>> 
>>>> 
>>>> 
>>>> 
>>>> -- 
>>>> ________________________________________________________________
>>>> Bob Briscoe                               http://bobbriscoe.net/
>>>> 
>>>> 
>>> _______________________________________________
>>> tcpm mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/tcpm
> 
>