Re: [tcpm] draft-ietf-tcpm-accurate-ecn-24 addressing all WGLC comments

[Yoshifumi Nishida <[e-mail]>]

Hi Markku,

Thanks for the comments. I put my comments in lines.

On Tue, Jul 11, 2023 at 6:40 AM Markku Kojo <[e-mail]> wrote:

> Hi Yoshi,
>
> thank you for your comments. Pls see inline.
>
> On Mon, 5 Jun 2023, Yoshifumi Nishida wrote:
>
> > Hi folks,
> >
> > I just would like to put my personal views on ack on ack discussions
> here.
> > First, I think ack on ack has already has some precedences.
> > keep-alive logic can sent a pure ack for proving and expect an ack for
> it.
>
> Keep-alives are very much different in their potential to cause any harm.
> Only a single segment is sent when there is no traffic in flight, by
> default no traffic for the recent two hours (repeated infrequently
> a few times, if no reply). So, quite impossible to cause any harm for
> ongoing data communication.
>
> > MPTCP uses 4 WHS and the 4th segment is an ack for the third ACK which
> is a pure ACK.
> > So, I'm not sure if we need to define some rules for ack-on-ack in the
> doc.
>
> This is also a single pure Ack sent in a very specific situation only:
> immediately after the last Ack of 3whs. Very hard to find any potential
> harm it may cause. Particularly, this pure Ack is not sent "randomly" at
> any point of an ongoing communication nor is it sent possibly multiple
> times per RTT.
>

Right, I just tried to mention that ACK on ACK is not a specific thing for
accecn here.
This is one reason that I would like to have most of the discussions for
ACK on ACK outside of this draft.

>
> > Also, I'm a bit hesitant to define a detailed logic about how to
> distinguish an ack that carries ECN signals and a dup
> > ack, such as using TS or SACK blocks.
> > I think such things are the part of experiments and should be described
> in other docs such as ECN++ or ackcc, etc.
> > I personally prefer the doc simply describes the possibilities of such
> mechanisms and provides general principles and
> > guidelines.
>
> I pretty much agree. As currently written in -24 that I am commenting what
> happens is that if the other end for whatever reason makes its ACKs
> ECN-capable, it will easily result in automatic generation of these
> excess dupacks, which are MUST NOT in RFC 5681, Sec. 4.2, last para.
> Furthermore, sending Acks for CE-marked pure Acks is also part of
> experiment too. It is not needed otherwise, so it should be specified in
> other docs, not in this PS to be. Thereby, the feature (acking Acks) would
> be in the same doc as the necessary measures to make it safe.
>
> > As far as I think, the Markku's examples that triggers false
> retransmissions makes sense. I think they are good examples
> > to show how ack-on-ack can be tricky.
> > However, these examples are the case where both sides exchanges data
> simultaneously and I think there're other cases
> > where we don't have to worry about it.
>
> No, they are examples of cases where communication is bidirectional but
> not simultaneous. Instead, they are scenarios where the end points send
> data in turns, i.e., the data flow direction alternates. Such
> request-reply app. communication scenarios are very common, if not the
> most common communication pattern over TCP in the Internet (e.g., most
> Web-based apps). Of course, there are scenarios that won't create
> problems, but even if there is just one problematic scenario, it must be
> addressed properly.
>

Sorry. Yes, you're right. The example is not simultaneous.

>
> > For example, I think bulk transfer or request-response type traffic can
> be these examples.
>
> Bulk transfer, yes, request-response type traffic definitely not.
>
> Please see Fig 1 of my scenarios but ignore data 0 from B to A and
> corresponding ACK from A to B. I just added this (data 0 and the
> corrsponding ACK) there to indicate which was the last cumulatively acked
> segment from B to A (I added it as the last thing after drawing other
> stuff, so there was not room to place it earlier, sorry). You may freely
> move that excange to occur before A sends the segment 20(000) which makes
> this typical data exchange of a request-reply app.


> In these cases, the endpoint which receives acks for acks doesn't have
> outstanding data. Hence, although these acks are
> > duplicate acks, they won't trigger retransmissions.
>
> I am afraid this is incorrect. Please see Fig 1 of my scenarios: when
> dupacks (Acks of Acks, marked red) start to arrive at B, B has sent out
> data 1..100, i.e., all these segments are outstanding and "fake" dupacks
> will trigger false Fast Retransmit and unnucessary retransmissions in the
> current window of outstanding data. All the retransmissions are
> unnecessary excess data segments sent out during the same RTT as their
> original transmissions! Obviously very bad behaviour that must be avoided.
>

Yes, I agree that you are right on Fig 1 example. But, I'm not very sure
this is typical request-response type traffic.

In this example, B starts sending data as soon as it receives data from A.
But, I am guessing B usually needs to process the received data from A on
App level and waits for the next data
from the App. Because of this, I think there are usually some gaps here.

In this case, if the dupacks arrive before B starts sending data, we don't
have any false retransmissions
because there's no outstanding data. In addition, if B is a requester, it
usually doesn't send much data.
So, the number of false retransmissions could be just a few packets even if
it receives dupacks through ACK on ACK.
Hence, I think the chance for false retransmission in request-response type
is not very big although I don't say it won't happen.

>
> > So, I am thinking that it would be good to provide a certain guideline
> about when to enable this feature and potential
> > risks in some docs.
> > But, I am also thinking we should do it outside of accecn doc.
>
> I would suggest putting anything that concerns Acking ACKs outside of
> AccECN doc, because such behaviour is experimental and AccECN is
> targetted as PS.
>

I agree with this as an individual.
--
Yoshi


>
> > Thanks,
> > --
> > Yoshi
> >
> >
> > On Fri, May 26, 2023 at 2:56 PM Markku Kojo <[e-mail]> wrote:
> >       Bob,
> >
> >       My apologies you had to wait for the scenarios as it took much
> longer
> >       with my limited cycles than I thought. Anyways, please see my
> reply to
> >       Richard, some scenarios are also included there.
> >
> >       To keep things easier, it might be good to try to keep the
> discussion on
> >       Acks of Acks (mainly) in the thread with my reply to Richard.
> >
> >       However, see inline tagged [MK].
> >
> >       On Wed, 24 May 2023, Bob Briscoe wrote:
> >
> >       > Markku,
> >       >
> >       > Sorry, it's taken a week to build a comprehensive reply to this
> long email. See inline tagged
> >       > [BB]...
> >       >
> >       > On 17/05/2023 12:24, Markku Kojo wrote:
> >       >       Hi Michael, all,
> >       >
> >       >       On Sun, 14 May 2023, [e-mail] wrote:
> >       >
> >       >                   On 30. Mar 2023, at 16:53, Bob Briscoe <
> [e-mail]>
> >       >                   wrote:
> >       >
> >       >                   Michael, Yoshi, Ian (as tcpm chairs),
> >       >
> >       >                   To close off the WGLC, I have just posted a
> new rev of
> >       >                   accurate-ecn. Hyperlinks quoted at the end.
> >       >                   You will see the diff is rather extensive. I
> won't give a
> >       >                   summary of all the diffs like I usually do.
> Instead I can just
> >       >                   refer to the summary I gave in the
> presentation on Monday:
> >       >
> >
> https://datatracker.ietf.org/meeting/116/materials/slides-116-tcpm-draft-ietf-tcpm-accurate-ecn
> >       >
> >       >                   Thank you again to the people who reviewed
> this during the WGLC:
> >       >                   Michael Tüxen, Alex Burr, Gorry Fairhurst and
> Markku Kojo.
> >       >
> >       >                   All changes are editorial, apart from removing
> the para about
> >       >                   not mistaking certain ACKs of ACKs for
> DupACKs, which I will add
> >       >                   to a rev of the ECN++ draft, hopefully later
> this week.
> >       >
> >       >                   On the list, we have seen agreement from all
> the reviewers to
> >       >                   these changes, except no response from Markku
> yet.
> >       >                   On Monday, I told Markku that I would post the
> draft in a few
> >       >                   days, so everyone can see the updates and diff.
> >       >
> >       >             Anyone having additional comments? In particular
> Markku regarding loss
> >       >             recovery?
> >       >
> >       >
> >       >       My apologies for being late with my reply to the author's
> comments on my review (I've
> >       >       been extremly busy with other issues since the wg mtng in
> Yokohama, including the rest
> >       >       of mtng week).
> >       >
> >       >       I don't have much new comments but it seems that my major
> concern regarding the problem
> >       >       of sending ACKs of ACKs was not fully understood.
> >       >
> >       >       The first thing where I think I was not quite clear is
> that the major problem with ACKs
> >       >       of ACKs is not that a pure Ack is made ECN-capable.
> Instead, the problem is in
> >       >       generating an Ack of an pure Ack and that is what one
> should prohibit to avoid problems.
> >       >       I understand that it might be problematic to formulate
> rules whether generating an Ack
> >       >       of an Ack is allowed (and when), instead of just disabling
> sending ECN-capable ACKs.
> >       >       I don't have a strong opinion which way the problems with
> ACKs of ACKs is avoided as
> >       >       long as they are avoided.
> >       >
> >       >
> >       > [BB] See later after your similar point (following your 'Why?'
> heading)...
> >       >
> >       >
> >       >       I am preparing a few scenarios to illustriate the problems
> ACKs of ACks raise and will
> >       >       send them shortly once I have formulated a more thorough
> reasoning why sending ACKs of
> >       >       ACKs is not really a good idea and even seems to be
> unnecessary in most if not all
> >       >       cases, i.e., it just results in sending unnecessary
> packets with not much useful effect
> >       >       but creates a notable number of problems.
> >       >
> >       >
> >       > [BB] Having waited this long, it's rather disappointing to still
> hear you say "I have an argument,
> >       > but I'll tell you later."
> >
> >       [MK] I understand. My sincere apologies again.
> >
> >       >       It also seems not have been carefully enough considered in
> terms of the very basic
> >       >       rubustness principle of "be conservative in what you send
> ..."
> >       >
> >       >
> >       > [BB] The WG has been careful to ensure that ACKs of ACKs are
> unambiguous (cannot be mistaken for a
> >       > DupACK), which is what the robustness principle requires. It's
> just that you think we have missed
> >       > cases where they will be ambiguous. If you think that, we need
> to hear them all.
> >       >
> >       > The robustness principle does not advocate sending nothing just
> in case some unknown factor might
> >       > make it ambiguous. Especially given /not/ feeding back
> congestion notifications has potential to
> >       > cause harm to others. Also, "no feedback" is much more ambiguous.
> >
> >       [MK] Please see my reply to Richard and let's continue from there.
> See
> >       also what I meant with "be conservative in what you send ...",
> that is,
> >       in context of CC: avoid sending unnecessary packets or be careful
> in
> >       sending packets just because they might be sometimes useful, send
> the
> >       monly when they are useful.
> >
> >       >       Given that this draft is intended to become a stds track
> RFC I am concerned of any text
> >       >       in this document that indicates (or even hints) that TCP
> could acknowledge pure ACKS
> >       >       (this holds particularly the rules and text in Sec
> 3.2.2.5.1 for the
> >       >       "Increment-Triggered ACKs"). If it is seen necessary that
> this doc should have such
> >       >       pieces of rules and text, I am fine if any such text is
> moved to an appendix as long as
> >       >       the appendix makes it cristal clear that the text is valid
> only in case one is
> >       >       implementing an experiment as per
> [I-D.ietf-tcpm-generalized-ecn].
> >       >
> >       >
> >       > [BB] See point below about "Generic (Mechanistic) Reflector".
> >       >
> >       >
> >       >       Why?
> >       >
> >       >       1) It is well known that TCP does not acknowledge ACKs and
> Standards track TCP has not
> >       >       been specified to acknowledge ACKs. This means that a
> reader/implementer of this doc
> >       >       cannot correctly understand the rule for
> "Increment-Triggered ACKs" unless there is a
> >       >       normative reference to a spec that specifies ACKs of ACKs
> (or tells that it is even
> >       >       possible).
> >       >
> >       >
> >       > [BB] ACKs of ACKs can indeed be tricky. But there's no need to
> consider not ACKing ACKs as an
> >       > architectural principle. Not Acking ACKs on principle certainly
> avoids some tricky problems.
> >       > However, we have a new situation here where, in limited
> circumstances, ACKs of ACKs are necessary.
> >       > So the WG has already worked through the tricky problems and
> they have been addressed in the draft
> >       > (e.g. mistaking ACKs of ACKs for DupACks, infinite ping-pong,
> etc). We'll discuss below whether
> >       > you've found some more trickiness.
> >
> >       (MK] I did not mean to refer to any principle but, as I said, that
> a
> >       reader/implementer cannot correctly understand the rule for
> >       "Increment-Triggered ACKs" because it is well-known to her/him
> that TCP
> >       does not Ack ACKs. This fact is that one can ack ACKs is not
> specified in
> >       this doc nor does this doc give a (normative) reference where it is
> >       specified, including the details on which TSecr value to add or
> which
> >       SACK info if any to include when acking a pure Ack. It is easy to
> >       misinterpret the "Increment-Triggered ACKs", if one doesn't
> realize that
> >       pure Acks may be acked.
> >
> >       > What is the new situation?
> >       >  *  Until ECN was introduced, TCP ACKs only acknowledged data.
> So there was no need to acknowledge
> >       >     pure ACKs, which contain no data.
> >       >  *  When ECN was introduced in RFC3168, TCP ACKs also
> acknowledged ECN markings. However, because
> >       >     RFC3168 precluded pure ACKs from being ECN-capable, there
> was still no need to acknowledge pure
> >       >     ACKs.
> >       >  *  RFC5690, and now the ECN++ draft introduce the possibility
> of ECN-capable pure ACKs. So, in the
> >       >     limited circumstances described in the AccECN draft,
> ECN-capable pure ACKs now need to be
> >       >     acknowledged, because they contain new information - their
> ECN field.
> >       > Similarly, even though the final ACK of TCP's 3WHS is an ACK of
> an ACK , it is sent because it is
> >       > needed (to prove that the SYN wasn't from a spoofed address).
> >
> >       [MK] All otherwise clear, but I disagree that the final ACK of
> TCP's 3WHS
> >       is an ACK of an ACK. It is required because SYNACK contains
> control data
> >       that eats one sequence number, i.e., it advances RCV.NXT at the
> client
> >       end and when the ACK arrives at the server it is needed to advance
> >       SND.UNA. Very different from Acks of Acks in this draft.
> >
> >       > It is true that not ACKing ACKs is well-known. However, whether
> it's well-known as a /principle/, or
> >       > just as a current /feature/ of TCP is not clear. Anyway, the
> IETF's job is to update RFCs that are
> >       > "well-known". We don't have to jump through any special
> procedural hoops to do something different
> >       > from what is "well-known". Even if it were prohibited in a stds
> track RFC, we just have to specify
> >       > what has to be done instead; in another stds track RFC.
> >
> >       [MK] Again, I didn't mention it as a /principle/ but as a crucial
> >       piece of information that the reader needs to be noted, that is,
> the
> >       things are now different from what is well-known.
> >
> >       Sure IETF's job is to update RFCs, but if one changes what is
> prohibited
> >       in a stds track RFC, one needs to understand the consequences and
> explain
> >       them as well as give the justification why the change can be done
> (without
> >       problems), instead of just specifying the change.
> >
> >       > If there are any tutorials, course notes or text books out there
> that say that not ACKing ACKs is a
> >       > well-known principle, that's not the IETF's problem. It is the
> job of the tutors, lecturers and text
> >       > book authors who wrote those materials to update them.
> >       >
> >       >
> >       >       2) ACKs of ACKs tend to trigger duplicate Acks. There are
> tons of algorithms that rely
> >       >       on the packet conservation principle and the fact that TCP
> never injects a dupAck unless
> >       >       a *data* packet has arrived and left the network. This is
> enforced with "MUST NOT" in
> >       >       RFC 5681, Sec 4.2, because not conforming to this rule
> makes any algorithm that rely on
> >       >       the rule to work incorrectly. These algorithms include
> (triggering) Fast Retransmit,
> >       >       (controlling packet rate during) Fast Recovery, (detecting
> spurious RTOs in) F-RTO,
> >       >       (calculating PipeAck in) RFC 7661, (calculating
> DeliveredData in) PRR, etc. Furthermore,
> >       >       it would make imposible to come up with any new algorihms
> that rely on this important
> >       >       basic rule. In most cases such extra dupAcks make these
> algorithms too aggressive
> >       >       because any extra dupAck is likely to inject extra
> packet(s) to the network.
> >       >
> >       >       So, it should be cristal clear that without SACK (or
> Timestamps) a TCP *MUST NOT* send
> >       >       ACKs of ACKs.
> >       >
> >       >
> >       > [BB] Constraining the /Data Receiver/ as you propose would
> create an interop problem.
> >       > Explanation: Consider host A and B are not using SACK or
> timestamps. Nonetheless, with your
> >       > approach, host A can still send ECN-capable pure ACKs to host B.
> Then, your rule puts host B in an
> >       > impossible position, where it gets congestion notifications on
> ECN-capable pure ACKs, but it is not
> >       > allowed to send any feedback about them.
> >       >
> >       > Instead, if neither timestamps nor SACK are in use for the
> connection, we need to constrain the
> >       > /Data Sender/ of a half connection from sending ECN-capable ACKs
> in the first place. This is the
> >       > approach the WG has adopted in the AccECN and ECN++ specs.
> >
> >       [MK] I think I said in the beginning that I have no strong opinion
> which
> >       way Acks of Acks are disabled. However, I apologize that I didn't
> explain
> >       why I phrased MUST NOT send ACKs of ACKs. This is because it might
> be
> >       still useful to allow CE-marked pure Acks and take care of Ack CC
> by some
> >       other means than Acks of Acks. Currently the draft mandates Acks
> of Acks
> >       as the only way to report Ack congestion and I think it is too
> >       restrictive in a stds track doc, e.g., it rules out reducing Ack
> rate
> >       simply by reducing data send rate which would solve the interop
> problem in
> >       a very simple way. Moreover, When B gets congestion notifications
> on
> >       ECN-capable pure ACKs, not sending Acks of Acks does not prevent
> sending
> >       feedback; such feedback need not to be delivered immediately but
> by the
> >       time needed. Please see more on this in my reply to Richard.
> >
> >       > Specifically:
> >       >  *  The WG makes sure that RFCs about the /Data Sender/ of a
> half connection (e.g. the ECN++
> >       >     experiment or other future RFCs) specify that sending
> ECN-capable pure ACKs is conditional on
> >       >     having another way to distinguish DupACKs, e.g. negotiating
> SACK or timestamps (and I will
> >       >     respond to your later points on the details of these).
> >       >  *  The AccECN spec (which primarily specifies the feedback
> behaviour of a /Data Receiver/ in a
> >       >     half-connection) then only needs to define the
> Increment-triggered ACK rule.
> >       > The two together lead to the same outcome you want. But without
> the interop hole of your approach.
> >       >
> >       > This is consistent with the "Generic (Mechanistic) Reflector"
> approach of the AccECN spec which
> >       > says:
> >       > "AccECN is designed to be a generic reflector of whatever ECN
> markings it sees, whether or not they
> >       > are compliant with a current standard."
> >       >
> >       > These ACKs of ACKs are generically necessary to feed back
> congestion notifications from possible
> >       > incoming packet patterns, not specifically for ECN++ or AckCC
> [RFC5690], or any other future RFC
> >       > (forward compatibility). We'll edit the reference to ECN++ to
> make it clearer that it's one example,
> >       > not the only case.
> >       >
> >       > Here's another example of the generic reflector approach,
> already in the draft:
> >       > "Although RFC 3168 prohibits an ECN-capable SYN, providing
> feedback of ECN marking on the SYN
> >       > supports future scenarios in which SYNs might be ECN-enabled
> (without prejudging whether they ought
> >       > to be). ... "
> >       >
> >       >
> >       >       So, it should be cristal clear that without SACK (or
> Timestamps) a TCP *MUST NOT* send
> >       >       ACKs of ACKs.
> >       >
> >       >
> >       > I understand that you want this but, as just explained, without
> SACK or timestamps, the correct
> >       > approach is to prevent the Data Sender putting the Data Receiver
> in the position where it would have
> >       > to ACK ACKs in the first place.
> >       >
> >       > In a connection without SACK or timestamps, if the Data Receiver
> were to get lots of congestion
> >       > notifications on ECN-capable ACKs, it would face a difficult
> dilemma. Which would be more important:
> >       > Signalling congestion by ACKing ACKs? or ensuring the
> performance improvements like Fast Retransmit,
> >       > Fast Recovery etc. work well? The former would prevent harm to
> others, the latter would prevent harm
> >       > to self.
> >
> >       [MK] Please see my reply to Richard to understand why Acks of Acks
> >       cause Fast Retransmit, Fast Recovery, etc to cause harm to others
> in the
> >       first place. And also why most of the Acks of Acks seem to be just
> >       unnecessary load to the network, that is, they harm others without
> (much)
> >       benefits.
> >
> >       > Nonetheless, I will add some text to the AccECN draft that
> explains why it is important for other
> >       > RFCs not to put a Data Receiver in the position where it has to
> ACK ACKs iff there is no way to
> >       > distinguish them from DupACKs.
> >       >
> >       >
> >       >       3) Even with SACK or Timestamps enabled it is not clear
> what an
> >       >       implementer should do. With SACK the AckECN authors seem
> to make an assumption, which
> >       >       seems obvious but is not, that an ACK of ACK would would
> never include SACK option and
> >       >       hence it could be distinguished from a dupAck. However,
> RFC 2018 specifies: "If sent at
> >       >       all, SACK options SHOULD be included in all ACKs which do
> not ACK the highest sequence
> >       >       number in the data receiver's queue. So, if there is a
> hole in the receiver's queue, the
> >       >       assumption is incorrect and it is unclear which SACK info
> to include into the SACK
> >       >       option. Whatever one selects to include, it makes DSACK
> (RFC 2883] void and breaks any
> >       >       DSACK-based algorithms unless RFC 2018 is updated.
> >       >
> >       >
> >       > [BB]
> >       > Reading the draft, it is very clear that there is no such
> assumption. The text said solely what it
> >       > meant:
> >       >
> >       >    ... a host in AccECN
> >       >    mode that is sending ECN-capable pure ACKs SHOULD add one of
> the
> >       >    following additional checks when it tests whether an incoming
> pure
> >       >    ACK is a duplicate:
> >       >
> >       >    o  If SACK has been negotiated for the connection, but there
> is no
> >       >       SACK option on the incoming pure ACK, it is not a
> duplicate;
> >       > That is, if an incoming ACK were a duplicate, it would have a
> SACK option on it. This /relies/ on
> >       > the rule in RFC2018 that you quote. So there is no need (nor
> intention) to change SACK behaviour in
> >       > any way.
> >
> >       [MK] Sorry I don't understand your claim. Assume A sends data pkts
> >       0,1,2,3,..,10 in the current window to B and pkt 1 is dropped.
> >       Simultaneously, B sends data to A such that the data pkts arrive
> at A
> >       after A has injected pkt 10 to the network. These pkts trigger pure
> >       cumulative Acks from A to B that follow A's data pkts 1..10 and
> enough of
> >       the Acks get CE-marked due to congestion path A to B. When pkts
> 2..10
> >       arrive at B, each of them trigger a valid dupAck with a SACK block
> >       included. When the CE-marked Acks that follow data pkts 2..10
> arrive at
> >       B, B needs to feedback congestion info on them in Acks of Acks.
> These
> >       Acks of Acks cannot cumulatively ACK the highest sequence number
> in the
> >       data receiver's queue (pkt 10) since pkt 0 is the highest pkt
> >       arrived insequence, so the Ack of Ack must include a SACK block as
> per
> >       RFC 2018. What is the SACK block info that the implementer, who
> follows
> >       carefully advice in RFC 2018 and there is no other advice, should
> >       include in these Acks unless RFC 2018 is not changed? How does the
> >       implementer know how to Ack, if it not specified?
> >
> >       > (Note: to check this text, you'll need to refer to the previous
> AccECN draft here:
> >       >
> https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accurate-ecn-23#section-3.2.2.5.1
> >       > It had been there since Jul 2021 (since -15). But, as explained
> above, it is longer in the latest
> >       > AccECN draft (-24), because it has been moved to the editor's
> copy of the ECN++ draft, which I wrote
> >       > on 4 Apr, but I've been waiting for your reply before submitting
> it.)
> >       >
> >       >
> >       >       With Timestamps some algorithms like Eifel detection
> breaks. Moreover, there are other
> >       >       existing or potentially to-be-created heauristics,
> including various measurement tools,
> >       >       that rely on the fact that TCP does not echo a later
> Timestamp in a pure Ack than what
> >       >       arrived with the latest data packet. Any such mechanisms
> are subject to break.
> >       >
> >       >
> >       > [BB] I don't fully understand what you're saying here. Can you
> be clearer please?
> >       > And can you please bear in mind that we are in the WGLC
> processing stage now. So review comments
> >       > ought to be suggesting very specific changes to the text under
> review.
> >
> >       [MK] Again, the same problem as with SACK. It is unspecified which
> >       TSecr value to put in Acks of Acks. That has not been specified
> >       anywhere, so how can I or anyone else check what breaks if
> anything, and
> >       how can an implementer know which TSecr value to include in Acks
> of Acks?
> >
> >       It is hard to propose specific changes to text that does not exist
> or to a
> >       problem that seems to be a missing piece of design or a design
> flaw, until
> >       the text exists or the intended design is known or the potentiel
> design
> >       flaw is mutually agreed whether there is a flaw or not.
> >
> >       (Please note also that the Eifel problem seems not to be serious,
> but
> >       instead the timestamps rule you proposed to distinguish dupAcks
> from Acks
> >       of Acks seems suspicious and calls for clarification).
> >
> >       Please see more details in my reply to Richard.
> >
> >       > Again, this text about extra DupACK checks has now been removed
> from AccECN and will shortly appear
> >       > in the ECN++ draft instead. I shall post the new ECN++ draft
> shortly.
> >       >
> >       >
> >       >       It might be good to hold discussing any details on what
> breaks and how/why and what are
> >       >       the consequences until I have sent my reply with scenarios
> to Richard.
> >       >
> >       >
> >       > [BB] Please try to prioritize any comments about the text that
> is now left in the AccECN draft. The
> >       > WGLC of AccECN is waiting for no-one else at the moment.
> >
> >       [MK] Unfortunately I was not able to do this, sorry. I got confused
> >       already earlier when I was reviewing AccECN and ended up checking
> ECN++
> >       quickly as I noted that AccECN draft cited it for these issues.
> When
> >       reading ECN++ I found Sec 3.3.3 and read:
> >
> >         "The question of whether and how the receiver of pure ACKs is
> required to
> >          feed back any CE marks on them is outside the scope of the
> present
> >          specification because it is a matter for the relevant feedback
> >          specification ([RFC3168] or [I-D.ietf-tcpm-accurate-ecn])."
> >
> >       This got me totally confused together with the later discussion on
> the mic
> >       at Yokohama mtng because that was what I had read, and I was not
> able to
> >       understand what was intended to go where. And I still not quite
> know
> >       but I think I have a better hunch. The above text still reads in
> >       ECN++ draft, but hopefully not in editor's copy?
> >
> >       Anyways, the major problem is not with any certain text phrases but
> >       whether DupAck vs. Ack of Ack problem is solvable and whether
> >       injecting Acks of Acks really is a needed and mature enough
> feature that
> >       can be part of a stds track protocol.
> >
> >       > But also bear in mind that the chairs plan to take ECN++ into
> WGLC once AccECN WGLC has been
> >       > cleared. So we need to hear your actual argument about the
> DupACK text that has been moved to ECN++
> >       > urgently too. We can't work with "I have an argument, but I'll
> tell you later".
> >
> >       [MK] Apologies for the delay again. Please see my reply to Richard
> for my
> >       arguments.
> >
> >       >       One additional comment regarding the "Change-Triggered
> ACKs" rule is that it would be
> >       >       useful to make it more clear how this plays with delayed
> Acks and how it alters
> >       >       acknowledgement rate.
> >       >       I am not sure that what the draft currently says is quite
> correct:
> >       >
> >       >        "The approach can lead to some additional ACKs but it
> feeds back
> >       >         the timing and the order in which ECN marks are received
> with minimal
> >       >         additional complexity. If CE marks are infrequent, as is
> the case for
> >       >         most AQMs at the time of writing, or there are multiple
> marks in a row,
> >       >         the additional load will be low.
> >       >
> >       >       For example, consider a scenario with bidirectional
> traffic between A and B where B has
> >       >       a hole in sequence resulting in every data packet in the
> current RTT to become acked
> >       >       (pure duplicate Acks). This may result in a packet flow
> from A to B where every second
> >       >       packet is a pure (duplicate) Ack. If there is congestion
> on the path from A to B such
> >       >       that a significant number of (data) packets get marked, it
> may result in acking every
> >       >       data packet from A to B. This does not necessarily result
> in low additional load?
> >       >
> >       >
> >       > [BB] I don't quite understand how every second packet from A to
> B is a pure duplicate ACK, but I
> >       > don't think I need to - I'll assume it's somehow possible.
> >       >
> >       > Then I think you've somehow assumed that the data packets get
> CE-marked, but the interspersed pure
> >       > ACKs don't (perhaps you're assuming that the pure ACKs in this
> case are not ECN-capable? Or perhaps
> >       > you're assuming size-based packet marking?). Whatever, I agree
> that, if this scenario did occur,
> >       > then the change-triggered ACK rule would indeed lead to B ACKing
> every data packet that arrives,
> >       > with no delayed ACKs. {Note 1}
> >       >
> >       > Nonetheless, the draft is quite open about the implications of
> the change-triggered ACK rule on ACK
> >       > rate. In the sentence straight after the ones you quote, it says:
> >       >     "However, marking patterns with numerous non-contiguous CE
> marks could increase the load
> >       > significantly."
> >       > And a little earlier it starts out by saying:
> >       >     "...the 'Change-Triggered ACKs' rule could sometimes cause
> the ACK rate to be problematic for
> >       > high performance"
> >       >
> >       > I don't think we need to include examples of how non-contiguous
> CE marking could occur. And if we
> >       > did, I'd prefer to use one that was less complex to explain,
> e.g. a high level of probabilistic AQM
> >       > marking. But thank you for this point anyway.
> >
> >       [MK] Thanks for pointing these additional sentences. I somehow
> missed
> >       them and/or did not manage to relate them to the text I quoted. I
> think
> >       this is good enough to address the case I raised, particularly
> "numerous
> >       non-contiguous CE marks could increase the load significantly."
> >
> >       > {Note 1}: It's ironic that the existing behaviour "where B has a
> hole in sequence" also results "in
> >       > every data packet in the current RTT to become acked" (by A).
> I'm not giving this as an excuse for
> >       > introducing another case with the same bad behaviour. I'm just
> highlighting the irony.
> >
> >       [MK] Maybe ironic but the fact that a hole in sequence results in
> every
> >       data packet in RTT to become acked has a very good reason being as
> it is
> >       because those dupAcks directly control the data rate per the packet
> >       conservation principle in various important algos at the data
> sender,
> >       such as Fast Recovery, and this behaviour is therefore a crucial
> part of
> >       such congestion control algos.
> >
> >       Best regards,
> >
> >       /Markku
> >
> >       >
> >       > Regards
> >       >
> >       >
> >       > Bob
> >       >
> >       >
> >       >       Best regards,
> >       >
> >       >       /Markku
> >       >
> >       >
> >       >             Best regards
> >       >             Michael
> >       >
> >       >                   Cheers
> >       >
> >       >
> >       >                   Bob
> >       >
> >       >                   On 30/03/2023 15:15, [e-mail]
> wrote:
> >       >                         A New Internet-Draft is available from
> the on-line
> >       >                         Internet-Drafts
> >       >                         directories. This Internet-Draft is a
> work item of
> >       >                         the TCP Maintenance and
> >       >                         Minor Extensions (TCPM) WG of the IETF.
> >       >
> >       >                            Title           : More Accurate
> Explicit
> >       >                         Congestion Notification (ECN) Feedback
> in TCP
> >       >                            Authors         : Bob Briscoe
> >       >                                              Mirja Kühlewind
> >       >                                              Richard
> Scheffenegger
> >       >                            Filename        :
> >       >                         draft-ietf-tcpm-accurate-ecn-24.txt
> >       >                            Pages           : 64
> >       >                            Date            : 2023-03-30
> >       >
> >       >                         Abstract:
> >       >                            Explicit Congestion Notification
> (ECN) is a
> >       >                         mechanism where network
> >       >                            nodes can mark IP packets instead of
> dropping
> >       >                         them to indicate
> >       >                            incipient congestion to the
> endpoints.  Receivers
> >       >                         with an ECN-capable
> >       >                            transport protocol feed back this
> information to
> >       >                         the sender.  ECN was
> >       >                            originally specified for TCP in such
> a way that
> >       >                         only one feedback
> >       >                            signal can be transmitted per
> Round-Trip Time
> >       >                         (RTT).  Recent new TCP
> >       >                            mechanisms like Congestion Exposure
> (ConEx), Data
> >       >                         Center TCP (DCTCP)
> >       >                            or Low Latency, Low Loss, and
> Scalable Throughput
> >       >                         (L4S) need more
> >       >                            accurate ECN feedback information
> whenever more
> >       >                         than one marking is
> >       >                            received in one RTT.  This document
> updates the
> >       >                         original ECN
> >       >                            specification in RFC 3168 to specify
> a scheme
> >       >                         that provides more than
> >       >                            one feedback signal per RTT in the
> TCP header.
> >       >                         Given TCP header
> >       >                            space is scarce, it allocates a
> reserved header
> >       >                         bit previously
> >       >                            assigned to the ECN-Nonce.  It also
> overloads the
> >       >                         two existing ECN
> >       >                            flags in the TCP header.  The
> resulting extra
> >       >                         space is exploited to
> >       >                            feed back the IP-ECN field received
> during the
> >       >                         3-way handshake as
> >       >                            well.  Supplementary feedback
> information can
> >       >                         optionally be provided
> >       >                            in two new TCP option alternatives,
> which are
> >       >                         never used on the TCP
> >       >                            SYN.  The document also specifies the
> treatment
> >       >                         of this updated TCP
> >       >                            wire protocol by middleboxes.
> >       >
> >       >                         The IETF datatracker status page for this
> >       >                         Internet-Draft is:
> >       >
> https://datatracker.ietf.org/doc/draft-ietf-tcpm-accurate-ecn/
> >       >
> >       >                         There is also an htmlized version
> available at:
> >       >
> https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-accurate-ecn-24
> >       >
> >       >                         A diff from the previous version is
> available at:
> >       >
> https://author-tools.ietf.org/iddiff?url2=draft-ietf-tcpm-accurate-ecn-24
> >       >
> >       >                         Internet-Drafts are also available by
> rsync at
> >       >                         rsync.ietf.org::internet-drafts
> >       >
> >       >
> >       >
>  _______________________________________________
> >       >                         tcpm mailing list
> >       >                         [e-mail]
> >       >
> https://www.ietf.org/mailman/listinfo/tcpm
> >       >
> >       >
> >       >                   --
> >       >
>  ________________________________________________________________
> >       >                   Bob Briscoe
> http://bobbriscoe.net/
> >       >
> >       >
> >       >
> >       >
> >       > --
> >       > ________________________________________________________________
> >       > Bob Briscoe                               http://bobbriscoe.net/
> >       >
> >       >
> >
> >
> >