Re: [tcpm] draft-ietf-tcpm-accurate-ecn-24 addressing all WGLC comments

["Bob Briscoe (IC)" <[e-mail]>]

Michael,


> On 19 Jun 2023, at 22:40, [e-mail] wrote:
> 
> 
> 
>> On 19. Jun 2023, at 12:38, Bob Briscoe (IC) <[e-mail]> wrote:
>> 
>> Michael,
>> 
>> The rule in RFC2026 is "Standards track specifications normally must not depend on other standards track specifications which are at a lower maturity level or on non standards track specifications". My thinking for making SACK and ECN++ informative refs was that recommendation in itself does not create a dependency.
> This is also my understanding of what is the plan.
>> The more clear-cut test is that the AccECN spec is defined for the case without ECN++ and without SACK, so I think that means it doesn’t depend on them.
> OK. So can you use a term different from "RECOMMENDED"?
>> 
>> 
>> Nonetheless, following a promise to Markku (also on the list), I have added the following to the local copy of AccECN after this ref to ECN++:
>>    "...or any equivalent future protocol that allows the ECN capability to be used on TCP control packets"
>> And I've similarly generalized all the refs to ECN++.
>> 
>> Does that resolve this concern?
> My concern is about the wording similar to
> 
> It is RECOMMENDED that AccECN is implemented alongside [I-D.ietf-tcpm-generalized-ecn].
> 
> RECOMMENDED is like SHOULD and therefore I'm not sure the reference can be Informative.
> I can double check with the AD on Wednesday...

[BB] I’m sure I can find a word like RECOMMENDED that isn’t actually ‘recommended'. But I’ll wait to see what the AD says first. Let me know if it’s not on the list.

‘Cos I was already aware that RECOMMENDED is defined to mean the same as SHOULD in RFC2119, but I still thought that would be fine and wouldn't require a normative reference. Because it doesn’t create a dependency. Especially, now that we’re going to change the ECN++ citation to “ECN++ [...] or a similar future standards action that allows ECT on TCP control packets”.


Bob


> 
> Best regards
> Michael
>> 
>> 
>> Bob
>> 
>>> On 18 Jun 2023, at 22:03, [e-mail] wrote:
>>> 
>>>> On 5. Jun 2023, at 10:33, Yoshifumi Nishida <[e-mail]> wrote:
>>>> 
>>>> Hi folks,
>>>> 
>>>> I just would like to put my personal views on ack on ack discussions here.
>>>> First, I think ack on ack has already has some precedences. 
>>>> keep-alive logic can sent a pure ack for proving and expect an ack for it. 
>>>> MPTCP uses 4 WHS and the 4th segment is an ack for the third ACK which is a pure ACK.
>>>> So, I'm not sure if we need to define some rules for ack-on-ack in the doc.
>>>> 
>>>> Also, I'm a bit hesitant to define a detailed logic about how to distinguish an ack that carries ECN signals and a dup ack, such as using TS or SACK blocks.
>>>> I think such things are the part of experiments and should be described in other docs such as ECN++ or ackcc, etc. 
>>>> I personally prefer the doc simply describes the possibilities of such mechanisms and provides general principles and guidelines. 
>>>> 
>>>> As far as I think, the Markku's examples that triggers false retransmissions makes sense. I think they are good examples to show how ack-on-ack can be tricky.
>>>> However, these examples are the case where both sides exchanges data simultaneously and I think there're other cases where we don't have to worry about it. 
>>>> For example, I think bulk transfer or request-response type traffic can be these examples. 
>>>> In these cases, the endpoint which receives acks for acks doesn't have outstanding data. Hence, although these acks are duplicate acks, they won't trigger retransmissions.
>>>> 
>>>> So, I am thinking that it would be good to provide a certain guideline about when to enable this feature and potential risks in some docs.
>>>> But, I am also thinking we should do it outside of accecn doc.
>>> I agree that the ACK of ACK scenario is something which relates to the
>>> experimental ECN++ document.
>>> 
>>> However, I checked again draft-ietf-tcpm-accurate-ecn-24 and found at the
>>> paragraph right before Section 1.1:
>>> 
>>> It is RECOMMENDED that the AccECN protocol is implemented alongside
>>> SACK [RFC2018] and the experimental ECN++ protocol
>>> [I-D.ietf-tcpm-generalized-ecn], which allows the ECN capability to
>>> be used on TCP control packets.
>>> 
>>> Would using I-D.ietf-tcpm-generalized-ecn in combination with RECOMMENDED
>>> not required the ID to be a normative reference? I couldn't find clear rules,
>>> but in my view I could not argument against it. Since the intended status
>>> of I-D.draft-ietf-tcpm-accurate-ecn is PS and the intended status of
>>> I-D.ietf-tcpm-generalized-ecn is Experiemental, this would be a downref.
>>> 
>>> Is this really intended by the authors or just a leftover?
>>> 
>>> Best regards
>>> Michael
>>>> 
>>>> Thanks,
>>>> --
>>>> Yoshi
>>>> 
>>>> 
>>>> On Fri, May 26, 2023 at 2:56 PM Markku Kojo <[e-mail]> wrote:
>>>> Bob,
>>>> 
>>>> My apologies you had to wait for the scenarios as it took much longer 
>>>> with my limited cycles than I thought. Anyways, please see my reply to 
>>>> Richard, some scenarios are also included there.
>>>> 
>>>> To keep things easier, it might be good to try to keep the discussion on 
>>>> Acks of Acks (mainly) in the thread with my reply to Richard.
>>>> 
>>>> However, see inline tagged [MK].
>>>> 
>>>> On Wed, 24 May 2023, Bob Briscoe wrote:
>>>> 
>>>>> Markku,
>>>>> 
>>>>> Sorry, it's taken a week to build a comprehensive reply to this long email. See inline tagged
>>>>> [BB]...
>>>>> 
>>>>> On 17/05/2023 12:24, Markku Kojo wrote:
>>>>>     Hi Michael, all,
>>>>> 
>>>>>     On Sun, 14 May 2023, [e-mail] wrote:
>>>>> 
>>>>>                 On 30. Mar 2023, at 16:53, Bob Briscoe <[e-mail]>
>>>>>                 wrote:
>>>>> 
>>>>>                 Michael, Yoshi, Ian (as tcpm chairs),
>>>>> 
>>>>>                 To close off the WGLC, I have just posted a new rev of
>>>>>                 accurate-ecn. Hyperlinks quoted at the end.
>>>>>                 You will see the diff is rather extensive. I won't give a
>>>>>                 summary of all the diffs like I usually do. Instead I can just
>>>>>                 refer to the summary I gave in the presentation on Monday:
>>>>>                 https://datatracker.ietf.org/meeting/116/materials/slides-116-tcpm-draft-ietf-tcpm-accurate-ecn
>>>>> 
>>>>>                 Thank you again to the people who reviewed this during the WGLC:
>>>>>                 Michael Tüxen, Alex Burr, Gorry Fairhurst and Markku Kojo.
>>>>> 
>>>>>                 All changes are editorial, apart from removing the para about
>>>>>                 not mistaking certain ACKs of ACKs for DupACKs, which I will add
>>>>>                 to a rev of the ECN++ draft, hopefully later this week.
>>>>> 
>>>>>                 On the list, we have seen agreement from all the reviewers to
>>>>>                 these changes, except no response from Markku yet.
>>>>>                 On Monday, I told Markku that I would post the draft in a few
>>>>>                 days, so everyone can see the updates and diff.
>>>>> 
>>>>>           Anyone having additional comments? In particular Markku regarding loss
>>>>>           recovery?
>>>>> 
>>>>> 
>>>>>     My apologies for being late with my reply to the author's comments on my review (I've
>>>>>     been extremly busy with other issues since the wg mtng in Yokohama, including the rest
>>>>>     of mtng week).
>>>>> 
>>>>>     I don't have much new comments but it seems that my major concern regarding the problem
>>>>>     of sending ACKs of ACKs was not fully understood.
>>>>> 
>>>>>     The first thing where I think I was not quite clear is that the major problem with ACKs
>>>>>     of ACKs is not that a pure Ack is made ECN-capable. Instead, the problem is in
>>>>>     generating an Ack of an pure Ack and that is what one should prohibit to avoid problems.
>>>>>     I understand that it might be problematic to formulate rules whether generating an Ack
>>>>>     of an Ack is allowed (and when), instead of just disabling sending ECN-capable ACKs.
>>>>>     I don't have a strong opinion which way the problems with ACKs of ACKs is avoided as
>>>>>     long as they are avoided.
>>>>> 
>>>>> 
>>>>> [BB] See later after your similar point (following your 'Why?' heading)...
>>>>> 
>>>>> 
>>>>>     I am preparing a few scenarios to illustriate the problems ACKs of ACks raise and will
>>>>>     send them shortly once I have formulated a more thorough reasoning why sending ACKs of
>>>>>     ACKs is not really a good idea and even seems to be unnecessary in most if not all
>>>>>     cases, i.e., it just results in sending unnecessary packets with not much useful effect
>>>>>     but creates a notable number of problems.
>>>>> 
>>>>> 
>>>>> [BB] Having waited this long, it's rather disappointing to still hear you say "I have an argument,
>>>>> but I'll tell you later."
>>>> 
>>>> [MK] I understand. My sincere apologies again.
>>>> 
>>>>>     It also seems not have been carefully enough considered in terms of the very basic
>>>>>     rubustness principle of "be conservative in what you send ..."
>>>>> 
>>>>> 
>>>>> [BB] The WG has been careful to ensure that ACKs of ACKs are unambiguous (cannot be mistaken for a
>>>>> DupACK), which is what the robustness principle requires. It's just that you think we have missed
>>>>> cases where they will be ambiguous. If you think that, we need to hear them all. 
>>>>> 
>>>>> The robustness principle does not advocate sending nothing just in case some unknown factor might
>>>>> make it ambiguous. Especially given /not/ feeding back congestion notifications has potential to
>>>>> cause harm to others. Also, "no feedback" is much more ambiguous.
>>>> 
>>>> [MK] Please see my reply to Richard and let's continue from there. See 
>>>> also what I meant with "be conservative in what you send ...", that is, 
>>>> in context of CC: avoid sending unnecessary packets or be careful in 
>>>> sending packets just because they might be sometimes useful, send the 
>>>> monly when they are useful.
>>>> 
>>>>>     Given that this draft is intended to become a stds track RFC I am concerned of any text
>>>>>     in this document that indicates (or even hints) that TCP could acknowledge pure ACKS
>>>>>     (this holds particularly the rules and text in Sec 3.2.2.5.1 for the
>>>>>     "Increment-Triggered ACKs"). If it is seen necessary that this doc should have such
>>>>>     pieces of rules and text, I am fine if any such text is moved to an appendix as long as
>>>>>     the appendix makes it cristal clear that the text is valid only in case one is
>>>>>     implementing an experiment as per [I-D.ietf-tcpm-generalized-ecn].
>>>>> 
>>>>> 
>>>>> [BB] See point below about "Generic (Mechanistic) Reflector".
>>>>> 
>>>>> 
>>>>>     Why?
>>>>> 
>>>>>     1) It is well known that TCP does not acknowledge ACKs and Standards track TCP has not
>>>>>     been specified to acknowledge ACKs. This means that a reader/implementer of this doc
>>>>>     cannot correctly understand the rule for "Increment-Triggered ACKs" unless there is a
>>>>>     normative reference to a spec that specifies ACKs of ACKs (or tells that it is even
>>>>>     possible).
>>>>> 
>>>>> 
>>>>> [BB] ACKs of ACKs can indeed be tricky. But there's no need to consider not ACKing ACKs as an
>>>>> architectural principle. Not Acking ACKs on principle certainly avoids some tricky problems.
>>>>> However, we have a new situation here where, in limited circumstances, ACKs of ACKs are necessary.
>>>>> So the WG has already worked through the tricky problems and they have been addressed in the draft
>>>>> (e.g. mistaking ACKs of ACKs for DupACks, infinite ping-pong, etc). We'll discuss below whether
>>>>> you've found some more trickiness.
>>>> 
>>>> (MK] I did not mean to refer to any principle but, as I said, that a 
>>>> reader/implementer cannot correctly understand the rule for 
>>>> "Increment-Triggered ACKs" because it is well-known to her/him that TCP 
>>>> does not Ack ACKs. This fact is that one can ack ACKs is not specified in 
>>>> this doc nor does this doc give a (normative) reference where it is 
>>>> specified, including the details on which TSecr value to add or which 
>>>> SACK info if any to include when acking a pure Ack. It is easy to 
>>>> misinterpret the "Increment-Triggered ACKs", if one doesn't realize that 
>>>> pure Acks may be acked.
>>>> 
>>>>> What is the new situation?
>>>>> *  Until ECN was introduced, TCP ACKs only acknowledged data. So there was no need to acknowledge
>>>>>   pure ACKs, which contain no data.
>>>>> *  When ECN was introduced in RFC3168, TCP ACKs also acknowledged ECN markings. However, because
>>>>>   RFC3168 precluded pure ACKs from being ECN-capable, there was still no need to acknowledge pure
>>>>>   ACKs.
>>>>> *  RFC5690, and now the ECN++ draft introduce the possibility of ECN-capable pure ACKs. So, in the
>>>>>   limited circumstances described in the AccECN draft, ECN-capable pure ACKs now need to be
>>>>>   acknowledged, because they contain new information - their ECN field.
>>>>> Similarly, even though the final ACK of TCP's 3WHS is an ACK of an ACK , it is sent because it is
>>>>> needed (to prove that the SYN wasn't from a spoofed address).
>>>> 
>>>> [MK] All otherwise clear, but I disagree that the final ACK of TCP's 3WHS 
>>>> is an ACK of an ACK. It is required because SYNACK contains control data 
>>>> that eats one sequence number, i.e., it advances RCV.NXT at the client 
>>>> end and when the ACK arrives at the server it is needed to advance 
>>>> SND.UNA. Very different from Acks of Acks in this draft.
>>>> 
>>>>> It is true that not ACKing ACKs is well-known. However, whether it's well-known as a /principle/, or
>>>>> just as a current /feature/ of TCP is not clear. Anyway, the IETF's job is to update RFCs that are
>>>>> "well-known". We don't have to jump through any special procedural hoops to do something different
>>>>> from what is "well-known". Even if it were prohibited in a stds track RFC, we just have to specify
>>>>> what has to be done instead; in another stds track RFC.
>>>> 
>>>> [MK] Again, I didn't mention it as a /principle/ but as a crucial 
>>>> piece of information that the reader needs to be noted, that is, the 
>>>> things are now different from what is well-known.
>>>> 
>>>> Sure IETF's job is to update RFCs, but if one changes what is prohibited 
>>>> in a stds track RFC, one needs to understand the consequences and explain 
>>>> them as well as give the justification why the change can be done (without 
>>>> problems), instead of just specifying the change.
>>>> 
>>>>> If there are any tutorials, course notes or text books out there that say that not ACKing ACKs is a
>>>>> well-known principle, that's not the IETF's problem. It is the job of the tutors, lecturers and text
>>>>> book authors who wrote those materials to update them.
>>>>> 
>>>>> 
>>>>>     2) ACKs of ACKs tend to trigger duplicate Acks. There are tons of algorithms that rely
>>>>>     on the packet conservation principle and the fact that TCP never injects a dupAck unless
>>>>>     a *data* packet has arrived and left the network. This is enforced with "MUST NOT" in
>>>>>     RFC 5681, Sec 4.2, because not conforming to this rule makes any algorithm that rely on
>>>>>     the rule to work incorrectly. These algorithms include (triggering) Fast Retransmit,
>>>>>     (controlling packet rate during) Fast Recovery, (detecting spurious RTOs in) F-RTO,
>>>>>     (calculating PipeAck in) RFC 7661, (calculating DeliveredData in) PRR, etc. Furthermore,
>>>>>     it would make imposible to come up with any new algorihms that rely on this important
>>>>>     basic rule. In most cases such extra dupAcks make these algorithms too aggressive
>>>>>     because any extra dupAck is likely to inject extra packet(s) to the network.
>>>>> 
>>>>>     So, it should be cristal clear that without SACK (or Timestamps) a TCP *MUST NOT* send
>>>>>     ACKs of ACKs.
>>>>> 
>>>>> 
>>>>> [BB] Constraining the /Data Receiver/ as you propose would create an interop problem.
>>>>> Explanation: Consider host A and B are not using SACK or timestamps. Nonetheless, with your
>>>>> approach, host A can still send ECN-capable pure ACKs to host B. Then, your rule puts host B in an
>>>>> impossible position, where it gets congestion notifications on ECN-capable pure ACKs, but it is not
>>>>> allowed to send any feedback about them.
>>>>> 
>>>>> Instead, if neither timestamps nor SACK are in use for the connection, we need to constrain the
>>>>> /Data Sender/ of a half connection from sending ECN-capable ACKs in the first place. This is the
>>>>> approach the WG has adopted in the AccECN and ECN++ specs.
>>>> 
>>>> [MK] I think I said in the beginning that I have no strong opinion which 
>>>> way Acks of Acks are disabled. However, I apologize that I didn't explain 
>>>> why I phrased MUST NOT send ACKs of ACKs. This is because it might be 
>>>> still useful to allow CE-marked pure Acks and take care of Ack CC by some 
>>>> other means than Acks of Acks. Currently the draft mandates Acks of Acks 
>>>> as the only way to report Ack congestion and I think it is too 
>>>> restrictive in a stds track doc, e.g., it rules out reducing Ack rate 
>>>> simply by reducing data send rate which would solve the interop problem in 
>>>> a very simple way. Moreover, When B gets congestion notifications on 
>>>> ECN-capable pure ACKs, not sending Acks of Acks does not prevent sending 
>>>> feedback; such feedback need not to be delivered immediately but by the 
>>>> time needed. Please see more on this in my reply to Richard.
>>>> 
>>>>> Specifically:
>>>>> *  The WG makes sure that RFCs about the /Data Sender/ of a half connection (e.g. the ECN++
>>>>>   experiment or other future RFCs) specify that sending ECN-capable pure ACKs is conditional on
>>>>>   having another way to distinguish DupACKs, e.g. negotiating SACK or timestamps (and I will
>>>>>   respond to your later points on the details of these).
>>>>> *  The AccECN spec (which primarily specifies the feedback behaviour of a /Data Receiver/ in a
>>>>>   half-connection) then only needs to define the Increment-triggered ACK rule.
>>>>> The two together lead to the same outcome you want. But without the interop hole of your approach.
>>>>> 
>>>>> This is consistent with the "Generic (Mechanistic) Reflector" approach of the AccECN spec which
>>>>> says:
>>>>> "AccECN is designed to be a generic reflector of whatever ECN markings it sees, whether or not they
>>>>> are compliant with a current standard."
>>>>> 
>>>>> These ACKs of ACKs are generically necessary to feed back congestion notifications from possible
>>>>> incoming packet patterns, not specifically for ECN++ or AckCC [RFC5690], or any other future RFC
>>>>> (forward compatibility). We'll edit the reference to ECN++ to make it clearer that it's one example,
>>>>> not the only case.
>>>>> 
>>>>> Here's another example of the generic reflector approach, already in the draft:
>>>>> "Although RFC 3168 prohibits an ECN-capable SYN, providing feedback of ECN marking on the SYN
>>>>> supports future scenarios in which SYNs might be ECN-enabled (without prejudging whether they ought
>>>>> to be). ... "
>>>>> 
>>>>> 
>>>>>     So, it should be cristal clear that without SACK (or Timestamps) a TCP *MUST NOT* send
>>>>>     ACKs of ACKs.
>>>>> 
>>>>> 
>>>>> I understand that you want this but, as just explained, without SACK or timestamps, the correct
>>>>> approach is to prevent the Data Sender putting the Data Receiver in the position where it would have
>>>>> to ACK ACKs in the first place.
>>>>> 
>>>>> In a connection without SACK or timestamps, if the Data Receiver were to get lots of congestion
>>>>> notifications on ECN-capable ACKs, it would face a difficult dilemma. Which would be more important:
>>>>> Signalling congestion by ACKing ACKs? or ensuring the performance improvements like Fast Retransmit,
>>>>> Fast Recovery etc. work well? The former would prevent harm to others, the latter would prevent harm
>>>>> to self.
>>>> 
>>>> [MK] Please see my reply to Richard to understand why Acks of Acks 
>>>> cause Fast Retransmit, Fast Recovery, etc to cause harm to others in the 
>>>> first place. And also why most of the Acks of Acks seem to be just 
>>>> unnecessary load to the network, that is, they harm others without (much) 
>>>> benefits.
>>>> 
>>>>> Nonetheless, I will add some text to the AccECN draft that explains why it is important for other
>>>>> RFCs not to put a Data Receiver in the position where it has to ACK ACKs iff there is no way to
>>>>> distinguish them from DupACKs.
>>>>> 
>>>>> 
>>>>>     3) Even with SACK or Timestamps enabled it is not clear what an
>>>>>     implementer should do. With SACK the AckECN authors seem to make an assumption, which
>>>>>     seems obvious but is not, that an ACK of ACK would would never include SACK option and
>>>>>     hence it could be distinguished from a dupAck. However, RFC 2018 specifies: "If sent at
>>>>>     all, SACK options SHOULD be included in all ACKs which do not ACK the highest sequence
>>>>>     number in the data receiver's queue. So, if there is a hole in the receiver's queue, the
>>>>>     assumption is incorrect and it is unclear which SACK info to include into the SACK
>>>>>     option. Whatever one selects to include, it makes DSACK (RFC 2883] void and breaks any
>>>>>     DSACK-based algorithms unless RFC 2018 is updated.
>>>>> 
>>>>> 
>>>>> [BB]
>>>>> Reading the draft, it is very clear that there is no such assumption. The text said solely what it
>>>>> meant:
>>>>> 
>>>>>  ... a host in AccECN
>>>>>  mode that is sending ECN-capable pure ACKs SHOULD add one of the
>>>>>  following additional checks when it tests whether an incoming pure
>>>>>  ACK is a duplicate:
>>>>> 
>>>>>  o  If SACK has been negotiated for the connection, but there is no
>>>>>     SACK option on the incoming pure ACK, it is not a duplicate;
>>>>> That is, if an incoming ACK were a duplicate, it would have a SACK option on it. This /relies/ on
>>>>> the rule in RFC2018 that you quote. So there is no need (nor intention) to change SACK behaviour in
>>>>> any way.
>>>> 
>>>> [MK] Sorry I don't understand your claim. Assume A sends data pkts 
>>>> 0,1,2,3,..,10 in the current window to B and pkt 1 is dropped. 
>>>> Simultaneously, B sends data to A such that the data pkts arrive at A 
>>>> after A has injected pkt 10 to the network. These pkts trigger pure 
>>>> cumulative Acks from A to B that follow A's data pkts 1..10 and enough of 
>>>> the Acks get CE-marked due to congestion path A to B. When pkts 2..10 
>>>> arrive at B, each of them trigger a valid dupAck with a SACK block 
>>>> included. When the CE-marked Acks that follow data pkts 2..10 arrive at 
>>>> B, B needs to feedback congestion info on them in Acks of Acks. These 
>>>> Acks of Acks cannot cumulatively ACK the highest sequence number in the 
>>>> data receiver's queue (pkt 10) since pkt 0 is the highest pkt 
>>>> arrived insequence, so the Ack of Ack must include a SACK block as per 
>>>> RFC 2018. What is the SACK block info that the implementer, who follows 
>>>> carefully advice in RFC 2018 and there is no other advice, should 
>>>> include in these Acks unless RFC 2018 is not changed? How does the 
>>>> implementer know how to Ack, if it not specified?
>>>> 
>>>>> (Note: to check this text, you'll need to refer to the previous AccECN draft here:
>>>>>   https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accurate-ecn-23#section-3.2.2.5.1
>>>>> It had been there since Jul 2021 (since -15). But, as explained above, it is longer in the latest
>>>>> AccECN draft (-24), because it has been moved to the editor's copy of the ECN++ draft, which I wrote
>>>>> on 4 Apr, but I've been waiting for your reply before submitting it.)
>>>>> 
>>>>> 
>>>>>     With Timestamps some algorithms like Eifel detection breaks. Moreover, there are other
>>>>>     existing or potentially to-be-created heauristics, including various measurement tools,
>>>>>     that rely on the fact that TCP does not echo a later Timestamp in a pure Ack than what
>>>>>     arrived with the latest data packet. Any such mechanisms are subject to break.
>>>>> 
>>>>> 
>>>>> [BB] I don't fully understand what you're saying here. Can you be clearer please?
>>>>> And can you please bear in mind that we are in the WGLC processing stage now. So review comments
>>>>> ought to be suggesting very specific changes to the text under review.
>>>> 
>>>> [MK] Again, the same problem as with SACK. It is unspecified which 
>>>> TSecr value to put in Acks of Acks. That has not been specified 
>>>> anywhere, so how can I or anyone else check what breaks if anything, and 
>>>> how can an implementer know which TSecr value to include in Acks of Acks?
>>>> 
>>>> It is hard to propose specific changes to text that does not exist or to a 
>>>> problem that seems to be a missing piece of design or a design flaw, until 
>>>> the text exists or the intended design is known or the potentiel design 
>>>> flaw is mutually agreed whether there is a flaw or not.
>>>> 
>>>> (Please note also that the Eifel problem seems not to be serious, but 
>>>> instead the timestamps rule you proposed to distinguish dupAcks from Acks 
>>>> of Acks seems suspicious and calls for clarification).
>>>> 
>>>> Please see more details in my reply to Richard.
>>>> 
>>>>> Again, this text about extra DupACK checks has now been removed from AccECN and will shortly appear
>>>>> in the ECN++ draft instead. I shall post the new ECN++ draft shortly.
>>>>> 
>>>>> 
>>>>>     It might be good to hold discussing any details on what breaks and how/why and what are
>>>>>     the consequences until I have sent my reply with scenarios to Richard.
>>>>> 
>>>>> 
>>>>> [BB] Please try to prioritize any comments about the text that is now left in the AccECN draft. The
>>>>> WGLC of AccECN is waiting for no-one else at the moment.
>>>> 
>>>> [MK] Unfortunately I was not able to do this, sorry. I got confused 
>>>> already earlier when I was reviewing AccECN and ended up checking ECN++ 
>>>> quickly as I noted that AccECN draft cited it for these issues. When 
>>>> reading ECN++ I found Sec 3.3.3 and read:
>>>> 
>>>> "The question of whether and how the receiver of pure ACKs is required to
>>>>  feed back any CE marks on them is outside the scope of the present
>>>>  specification because it is a matter for the relevant feedback
>>>>  specification ([RFC3168] or [I-D.ietf-tcpm-accurate-ecn])."
>>>> 
>>>> This got me totally confused together with the later discussion on the mic 
>>>> at Yokohama mtng because that was what I had read, and I was not able to 
>>>> understand what was intended to go where. And I still not quite know 
>>>> but I think I have a better hunch. The above text still reads in 
>>>> ECN++ draft, but hopefully not in editor's copy?
>>>> 
>>>> Anyways, the major problem is not with any certain text phrases but 
>>>> whether DupAck vs. Ack of Ack problem is solvable and whether 
>>>> injecting Acks of Acks really is a needed and mature enough feature that 
>>>> can be part of a stds track protocol.
>>>> 
>>>>> But also bear in mind that the chairs plan to take ECN++ into WGLC once AccECN WGLC has been
>>>>> cleared. So we need to hear your actual argument about the DupACK text that has been moved to ECN++
>>>>> urgently too. We can't work with "I have an argument, but I'll tell you later".
>>>> 
>>>> [MK] Apologies for the delay again. Please see my reply to Richard for my 
>>>> arguments.
>>>> 
>>>>>     One additional comment regarding the "Change-Triggered ACKs" rule is that it would be
>>>>>     useful to make it more clear how this plays with delayed Acks and how it alters
>>>>>     acknowledgement rate.
>>>>>     I am not sure that what the draft currently says is quite correct:
>>>>> 
>>>>>      "The approach can lead to some additional ACKs but it feeds back
>>>>>       the timing and the order in which ECN marks are received with minimal
>>>>>       additional complexity. If CE marks are infrequent, as is the case for
>>>>>       most AQMs at the time of writing, or there are multiple marks in a row,
>>>>>       the additional load will be low.
>>>>> 
>>>>>     For example, consider a scenario with bidirectional traffic between A and B where B has
>>>>>     a hole in sequence resulting in every data packet in the current RTT to become acked
>>>>>     (pure duplicate Acks). This may result in a packet flow from A to B where every second
>>>>>     packet is a pure (duplicate) Ack. If there is congestion on the path from A to B such
>>>>>     that a significant number of (data) packets get marked, it may result in acking every
>>>>>     data packet from A to B. This does not necessarily result in low additional load?
>>>>> 
>>>>> 
>>>>> [BB] I don't quite understand how every second packet from A to B is a pure duplicate ACK, but I
>>>>> don't think I need to - I'll assume it's somehow possible.
>>>>> 
>>>>> Then I think you've somehow assumed that the data packets get CE-marked, but the interspersed pure
>>>>> ACKs don't (perhaps you're assuming that the pure ACKs in this case are not ECN-capable? Or perhaps
>>>>> you're assuming size-based packet marking?). Whatever, I agree that, if this scenario did occur,
>>>>> then the change-triggered ACK rule would indeed lead to B ACKing every data packet that arrives,
>>>>> with no delayed ACKs. {Note 1}
>>>>> 
>>>>> Nonetheless, the draft is quite open about the implications of the change-triggered ACK rule on ACK
>>>>> rate. In the sentence straight after the ones you quote, it says:
>>>>>   "However, marking patterns with numerous non-contiguous CE marks could increase the load
>>>>> significantly."
>>>>> And a little earlier it starts out by saying:
>>>>>   "...the 'Change-Triggered ACKs' rule could sometimes cause the ACK rate to be problematic for
>>>>> high performance"
>>>>> 
>>>>> I don't think we need to include examples of how non-contiguous CE marking could occur. And if we
>>>>> did, I'd prefer to use one that was less complex to explain, e.g. a high level of probabilistic AQM
>>>>> marking. But thank you for this point anyway.
>>>> 
>>>> [MK] Thanks for pointing these additional sentences. I somehow missed 
>>>> them and/or did not manage to relate them to the text I quoted. I think 
>>>> this is good enough to address the case I raised, particularly "numerous 
>>>> non-contiguous CE marks could increase the load significantly."
>>>> 
>>>>> {Note 1}: It's ironic that the existing behaviour "where B has a hole in sequence" also results "in
>>>>> every data packet in the current RTT to become acked" (by A). I'm not giving this as an excuse for
>>>>> introducing another case with the same bad behaviour. I'm just highlighting the irony.
>>>> 
>>>> [MK] Maybe ironic but the fact that a hole in sequence results in every 
>>>> data packet in RTT to become acked has a very good reason being as it is 
>>>> because those dupAcks directly control the data rate per the packet 
>>>> conservation principle in various important algos at the data sender,
>>>> such as Fast Recovery, and this behaviour is therefore a crucial part of 
>>>> such congestion control algos.
>>>> 
>>>> Best regards,
>>>> 
>>>> /Markku
>>>> 
>>>>> 
>>>>> Regards
>>>>> 
>>>>> 
>>>>> Bob
>>>>> 
>>>>> 
>>>>>     Best regards,
>>>>> 
>>>>>     /Markku
>>>>> 
>>>>> 
>>>>>           Best regards
>>>>>           Michael
>>>>> 
>>>>>                 Cheers
>>>>> 
>>>>> 
>>>>>                 Bob
>>>>> 
>>>>>                 On 30/03/2023 15:15, [e-mail] wrote:
>>>>>                       A New Internet-Draft is available from the on-line
>>>>>                       Internet-Drafts
>>>>>                       directories. This Internet-Draft is a work item of
>>>>>                       the TCP Maintenance and
>>>>>                       Minor Extensions (TCPM) WG of the IETF.
>>>>> 
>>>>>                          Title           : More Accurate Explicit
>>>>>                       Congestion Notification (ECN) Feedback in TCP
>>>>>                          Authors         : Bob Briscoe
>>>>>                                            Mirja Kühlewind
>>>>>                                            Richard Scheffenegger
>>>>>                          Filename        :
>>>>>                       draft-ietf-tcpm-accurate-ecn-24.txt
>>>>>                          Pages           : 64
>>>>>                          Date            : 2023-03-30
>>>>> 
>>>>>                       Abstract:
>>>>>                          Explicit Congestion Notification (ECN) is a
>>>>>                       mechanism where network
>>>>>                          nodes can mark IP packets instead of dropping
>>>>>                       them to indicate
>>>>>                          incipient congestion to the endpoints.  Receivers
>>>>>                       with an ECN-capable
>>>>>                          transport protocol feed back this information to
>>>>>                       the sender.  ECN was
>>>>>                          originally specified for TCP in such a way that
>>>>>                       only one feedback
>>>>>                          signal can be transmitted per Round-Trip Time
>>>>>                       (RTT).  Recent new TCP
>>>>>                          mechanisms like Congestion Exposure (ConEx), Data
>>>>>                       Center TCP (DCTCP)
>>>>>                          or Low Latency, Low Loss, and Scalable Throughput
>>>>>                       (L4S) need more
>>>>>                          accurate ECN feedback information whenever more
>>>>>                       than one marking is
>>>>>                          received in one RTT.  This document updates the
>>>>>                       original ECN
>>>>>                          specification in RFC 3168 to specify a scheme
>>>>>                       that provides more than
>>>>>                          one feedback signal per RTT in the TCP header. 
>>>>>                       Given TCP header
>>>>>                          space is scarce, it allocates a reserved header
>>>>>                       bit previously
>>>>>                          assigned to the ECN-Nonce.  It also overloads the
>>>>>                       two existing ECN
>>>>>                          flags in the TCP header.  The resulting extra
>>>>>                       space is exploited to
>>>>>                          feed back the IP-ECN field received during the
>>>>>                       3-way handshake as
>>>>>                          well.  Supplementary feedback information can
>>>>>                       optionally be provided
>>>>>                          in two new TCP option alternatives, which are
>>>>>                       never used on the TCP
>>>>>                          SYN.  The document also specifies the treatment
>>>>>                       of this updated TCP
>>>>>                          wire protocol by middleboxes.
>>>>> 
>>>>>                       The IETF datatracker status page for this
>>>>>                       Internet-Draft is:
>>>>>                       https://datatracker.ietf.org/doc/draft-ietf-tcpm-accurate-ecn/
>>>>> 
>>>>>                       There is also an htmlized version available at:
>>>>>                       https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accurate-ecn-24
>>>>> 
>>>>>                       A diff from the previous version is available at:
>>>>>                       https://author-tools.ietf.org/iddiff?url2=draft-ietf-tcpm-accurate-ecn-24
>>>>> 
>>>>>                       Internet-Drafts are also available by rsync at
>>>>>                       rsync.ietf.org::internet-drafts
>>>>> 
>>>>> 
>>>>>                       _______________________________________________
>>>>>                       tcpm mailing list
>>>>>                       [e-mail]
>>>>>                       https://www.ietf.org/mailman/listinfo/tcpm
>>>>> 
>>>>> 
>>>>>                 --
>>>>>                 ________________________________________________________________
>>>>>                 Bob Briscoe                               http://bobbriscoe.net/
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> -- 
>>>>> ________________________________________________________________
>>>>> Bob Briscoe                               http://bobbriscoe.net/
>>>>> 
>>>>> 
>>>> _______________________________________________
>>>> tcpm mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/tcpm
>> 
>> 
>