Re: [tcpm] Fw: draft-ietf-tcpm-accurate-ecn-24 addressing allWGLCcomments

Markku Kojo <kojo@cs.helsinki.fi> Thu, 27 July 2023 15:26 UTC

Return-Path: <kojo@cs.helsinki.fi>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 233C7C14CE46 for <tcpm@ietfa.amsl.com>; Thu, 27 Jul 2023 08:26:32 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.007
X-Spam-Level:
X-Spam-Status: No, score=-2.007 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=cs.helsinki.fi
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kVb9M1_ki4pb for <tcpm@ietfa.amsl.com>; Thu, 27 Jul 2023 08:26:26 -0700 (PDT)
Received: from script.cs.helsinki.fi (script.cs.helsinki.fi [128.214.11.1]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4F9E7C14CF0D for <tcpm@ietf.org>; Thu, 27 Jul 2023 08:26:25 -0700 (PDT)
X-DKIM: Courier DKIM Filter v0.50+pk-2017-10-25 mail.cs.helsinki.fi Thu, 27 Jul 2023 18:26:18 +0300
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cs.helsinki.fi; h=date:from:to:cc:subject:message-id:mime-version:content-type :content-id; s=dkim20130528; bh=IzaSIgAMK/ALKQeR7J253QUtV2pzOn4S fIL5AcQTvYA=; b=GeESefXdVdod9TvXQDh27Na/xMIsr5eVf9BKGIcphs6AkfmW z48fyAhYlp8LXwMuX6dCeSJ05h54+IbJC/vDBEJCuNR98EZaqhmxd6O1qNf0jBTq C8y2I0S92taDCNBBGHq4rPoLZnfiCRM/+mh6V0lyiNGzrtJm4GxXwLGrljM=
Received: from hp8x-60.cs.helsinki.fi (85-76-117-182-nat.elisa-mobile.fi [85.76.117.182]) (AUTH: PLAIN kojo, TLS: TLSv1/SSLv3,256bits,AES256-GCM-SHA384) by mail.cs.helsinki.fi with ESMTPSA; Thu, 27 Jul 2023 18:26:17 +0300 id 00000000005A011D.0000000064C28C99.00006E24
Date: Thu, 27 Jul 2023 18:26:16 +0300
From: Markku Kojo <kojo@cs.helsinki.fi>
To: Bob Briscoe <ietf@bobbriscoe.net>
cc: tuexen@fh-muenster.de, Yoshifumi Nishida <nsd.ietf@gmail.com>, Ian Swett <ianswett@google.com>, bob_briscoe@apple.com, tcpm@ietf.org
Message-ID: <556e9011-df92-c163-26c5-512922148289@cs.helsinki.fi>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=_script-28221-1690471578-0001-2"
Content-ID: <a0556c56-48d1-a5ab-1b57-df38af57625@cs.helsinki.fi>
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/NFtnBrqegyrpLn9SWuSzKYVAr0A>
Subject: Re: [tcpm] Fw: draft-ietf-tcpm-accurate-ecn-24 addressing allWGLCcomments
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 27 Jul 2023 15:26:32 -0000

Bob,

thank you for resending this with another email address as my regular address 
bounced (my account erreneously expired last Fri but is now back and working).

Please see below tagged [MK3].

On Wed, 26 Jul 2023, Markku P I Kojo wrote:

> 
> 
> ____________________________________________________________________________________________________________________________
> From: Bob Briscoe <research@bobbriscoe.net>
> Sent: 23 July 2023 00:01
> To: Kojo, Markku P I <markku.kojo@helsinki.fi>
> Subject: Fwd: draft-ietf-tcpm-accurate-ecn-24 addressing all WGLC comments  
> Retrying with an address which appears not to bounce.
> Pls add back mail distribution below in any reply.
> 
> 
> -------- Forwarded Message --------
> Subject:
> Re: draft-ietf-tcpm-accurate-ecn-24 addressing all WGLC comments
> Date:
> Sat, 22 Jul 2023 13:33:51 -0700
> From:
> Bob Briscoe <ietf@bobbriscoe.net>
> To:
> Markku Kojo <kojo@cs.helsinki.fi>
> CC:
> tuexen@fh-muenster.de, Yoshifumi Nishida <nsd.ietf@gmail.com>, Ian Swett 
> <ianswett@google.com>, tcpm@ietf.org, Bob Briscoe
> [Apple] <bob_briscoe@apple.com>
> 
> Markku,
> 
> I'm top-posting 4 responses to all your points, in order to consolidate 
> everything and avoid all the repetition:
> 
> #1) Generic should not be confused with experimental.
> #2) Completeness
> #3) Does not specify the necessary details?
> #4) Not generic enough?
> 
> #1) GENERIC SHOULD NOT BE CONFUSED WITH EXPERIMENTAL.
> 
> 
> The AccECN spec is the IETF's PS spec for how TCP feeds back ECN on any 
> packet. The requirement for AccECN to provide
> forward compatibility by providing generic feedback was agreed by this WG and 
> set down in RFC7560 (nearly 8 years ago).
> Generic mechanistic reflection is a central design principle of AccECN - to 
> reflect the ECN field of SYNs, SYN-ACKs, ACKs,
> data packets, retransmissions, everything (§2.5 of the draft has said this 
> since its inception). AccECN provides accurate
> feedback irrespective of whether or how it is currently used, so that it can 
> be used if needed.
> Mechanistic reflection is generic, not experimental.
>     Analogy: Senders have conducted many CC experiments using feedback from 
> stds-track rcvrs. But that doesn't make the
> rcvrs experimental; rather, it proves the f/b is generic.

[MK3] All these CC experiments that I am aware of have used stds track receiver 
feedback that had already existed for a long time before the experiment and had 
been matured for years (decades). This draft intends to introduce new behaviour 
(acking ACKs randomly in scale) for upcoming experiments at the same time when 
the experiments are being specified. That is very different because the new 
feature has not matured. Besides, the Acks of Acks issue appeared only after 
the wg had decided this document to become stds track.

> You say an AccECN receiver automatically becomes part of an experiment as 
> soon as the its peer enables ECN-capable ACKs.
> More constructively, one can say that the peer conducts the experiment by 
> building on the generic behaviour of the receiver.
> The latter is more correct because the behaviour of the receiver was already 
> specified (even though it was not exercised);
> so it predated the experiment, and could be analysed and checked prior to any 
> experiments.

[MK3] It can be well phrases both ways. "it was not exercised" is the key 
concern here. We do not know enough about potential problems; you say it "could 
be analysed and checked prior to any experiments" but that IMHO is not good 
enough to introduce such a new stds track behaviour. Even though it was 
carefully analysed, the TCP Timestamp condition was not found being incorrect, 
for example. This clearly shows how tricky issue we are handling and I (and I 
guess anyone else either) cannot come up with all potential problems easily. 
That's why we should be evry careful here.
Pls see also my reply to Richard where I explained how this feature of Acks of 
Acks may become a problem much later (maybe after decade or more), when AccECN 
possibly have been widely deployed as a stds track spec but ECN++ somehow faded 
away. How is it possible/guaranteed that another experiment that wants to 
enable pure Acks with non-SACK connections can get rid of the Acks of Acks it 
does not need and all harm they produce?
They are coded and deployed all over and it would be hard to get rid of them.

> This is protocol layering. Feedback is a 'sub-layer' of the transport layer 
> that lies logically beneath the congestion
> control function of the transport layer and is intended to allow future 
> evolution of sender behaviour. The RTCP protocol
> demonstrates this well, because it was specified separately from RTP.
> 
> You express concern that specifying a case where a stds track AccECN 
> implementation has to ack ACKs gives it no control over
> whether it is dragged into an experiment being conducted by its peer.  But 
> that's the deal for every generic capability. It
> gets used. Our task is to make sure that it can be used in this way. That's 
> why we have to correctly specify the SACK
> condition. That's what we do in the IETF.
> 
> #2) COMPLETENESS
> 
> 
> A protocol spec should say what an implementation has to do in response to 
> all possible inputs. This ensures that there is
> just one behaviour to be checked for flaws (including security flaws), rather 
> than implementers inventing their own
> behaviours, each of which might lead to bugs or vulnerabilities if it 
> receives unexpected inputs.

[MK3] The incompleteness is exactly the problem that I have tried to explain. 
See (*) under item #3 below.

> #3) DOES NOT SPECIFY THE NECESSARY DETAILS?
> 
> 
> You say this spec doesn't define the necessary safety features. We can't work 
> with "it needs a careful review for potential
> problems" then do nothing while we wait in case we might see another email 
> from you. If you think a safety feature is
> missing, the only way the WG works is constructively: participants identify 
> what's missing, and (ideally) define it.

[MK3] (*) My apologies if I have not been clear enough. What I have tried to 
say is that the present spec does not specify how to ack an Ack when SACK is 
enabled. I was now able interpret the intended way to ack an Ack (I guessed 
this myself already earlier and you hinted it as well but I was not sure 
because there were contradictory comments on this matter earlier) from the most 
recent ECN++ spec which implies that an ECN++ peer expects that an Ack of Ack 
does not carry SACK option:
  "If there is no SACK option on the incoming pure ACK despite SACK
   having  been negotiated, it is not a duplicate;"

This way of acking, however, has not been specified in this spec. Instead, you 
claim that this would be the case if the implementer follows SACK (RFC 2018) 
and DSACK (RFC 2883). I disagee with this. Instead, the implied way from ECN++ 
spec and the SACK spec (and DSACK) are in conflict.

An implementer who implements this spec following the rules in this spec as 
well as SACK (RFC 2018) and DSACK (RFC 2883) has a serious dilemma to solve 
which is contrary to what you claim in item #2 above.
Let's walk through what the implementer has at hand:

1. When enough pure ACKs with CE have arrived, this spec requires that the 
receiving end MUST send an ACK.

2. The implementer reads SACK spec (RFC 2018) that requires that a SACK block 
is included in the ACK in certain cases:

  "SACK options SHOULD be included in all ACKs which do not ACK
  the highest sequence number in the data receiver's queue."

2. a) If there is no hole in the sequence space, everything is fine;
       No SACK block is included as per RFC 2018 and these Acks of
       Acks are not considered being DupAcks.

2. b) If there is hole in the sequence space, RFC 2018 requires a SACK
       block being included. The implementer includes a SACK block, but
       has a dilemma to solve: which SACK block to indicate. Whichever
       SACK block is included will result these Acks to be considered
       valid DupAcks as per ECN++ which they are not (they do not
       indicate an arrival of an out-of-order data packet but incorrectly
       repeat some old SACK info. This is against the rule in RFC 5681
       that prohibits for a good reason to send more than one ACK per
       arriving data packet and will result in problems (see below).
       That is, this spec misses the rule for acking ACKs without the
       SACK option and indicating this is different from RFC 2018
       specification. And your completeness reuirement in item #2 is
       not fulfilled.

3. If the implementation also implements DSACK, the implementer has even
    harder dilemma to solve in case there is a hole in the sequence space.
    RFC 2018 requires that a SACK block is included and DSACK specifies
    that the first SACK block reports "sequence of data received by the
    receiver in the most recent packet".
    Because the pure ACK does not include any data, what should the
    implementer do. For sure there will be various different
    interpretations because this has not been specified in this spec
    (and your item #2 is not fulfilled).

   a) An impelmenter may select a "random" block, maybe the most
      recently SACKED which is incorrect because it breaks DSACK idea
      and is in conflict with RFC 2883.
   b) Another implementer may be innovative and includes some "random"
      sequence number and includes a zero length SACK block with the
      same sequence number as the left and right edge. The consequences
      of this are unknown but certainly not correct.
   c) A more clever inplementer that reads the thoughts of authors
      for this spec (or has followed the discusions or even read ECN++)
      may send an ACK without any SACK info but this is in conflict
      RFC 2018.
   d) Maybe some other interpretation ...

You said earlier (below) that a) is the correct way with DSACK:

  "[BB2] Surely those ACKs of ACKs from B to A are just further
   valid DupACKs with a SACK block included as defined in
   RFC2018, still cumulatively ack'ing packet 0 delivered
   before the loss."

This would result in incorrect behaviour with DSACK and will confuse any algo 
that depends on correct behavior of DSACK in correctly reporting arrival 
of duplicate data segments, including RFC 3708 and (TLP of) RACK (RFC 
8985) as I have earlier indicated.

The intention of DSACK is to inform an arrival of duplicate data segment and 
the spec explicitely prohibits repeating that info in the spirit of the one ACK 
per data segment rule of RFC5681. RFC 2883, Sec 4, item (2):

  "Each duplicate contiguous sequence of data received is reported
  in at most one D-SACK block.  (I.e., the receiver sends two identical
  D-SACK blocks in subsequent packets only if the receiver receives two
  duplicate segments.)"

Even if one selects the intended option (c above), it does not necessarily 
result in correct outcome. SACK specification in RFC 3517 used the same 
definition for DupAck as RFC 5681.
The DupAck definition in RFC 6675 holds for the purposes of RFC 6675 only as 
clearly stated in that specification. That is, it depends on an RFC spec which 
definition of DupAck (RFC 5681 of RFC 6675) it adopts regardless of whether the 
spec involves SACK or not. For example, F-RTO deliberately uses the DupAck 
definition of RFC 5681 for both non-SACK version and the SACK-enhanced version 
of F-RTO. In other words, the SACK-enhanced version of F-RTO uses an arriving 
DupAck without SACK option (or DupAck with SACK block but with no new SACK 
information) to declare RTO being not spurious similar to the non-SACK version 
does (Sec 3.1, step 3 a). Therefore, an arriving Ack of an Ack without SACK 
info incorrectly declares an RTO not spurious even if it actually was spurious 
RTO.
(This needs a bit different scenario than what I draw in Fig 3 b) of my 
scenarios that was for the non-SACK version of F-RTO)

To sum up: In order to partially solve the ambiguity on how to ack ACKs and 
avoid triggering false fast retransmits, this spec must update RFC 2018 by 
making an exception to the rule that specifies when a SACK block should be 
included. However, this does not solve the entire problem with Acks of Acks 
because there are algos like F-RTO that use the RFC 5681 definition of DupAck. 
Therefore, IMHO sending Acks of Acks should not be included in a stds track 
spec.

> You repeatedly refer forward to apparent problems with SACK, which sounds 
> like you have identified that the protocol is
> broken. But, having read forward to the end, you agree that the SACK 
> condition is more solid than using TSOpt. And the only
> concrete criticism I can find is that we don't specify what DSACK to send. 
> Richard & I (and others) have worked through all
> the cases you have proposed (and others we have contrived ourselves), and we 
> cannot find any case where an implementation
> has to do anything different from what the DSACK RFCs already say.

[MK3] Please see the explanation above. Hope it clarifies the issue now.

> I had independently found the rule you point out in RFC 5681, Sec 2, against 
> generating more than one ACK for every incoming
> [data] segment. We can say that the potential requirement to ack ACKs 
> contravenes that rule, which is why the SACK condition
> is necessary. I know you are concerned that it might not be as simple as 
> that. But, without evidence, how do we distinguish
> this from over-dramatisation? In the absence of anything specific, we can't 
> just sit on our hands. As Yoshi says, these will
> probably be corner cases. If new problems surface, we will have to deal with 
> them as they arise.

[MK3] Please see above. Definitely not a corner case.

> #4) NOT GENERIC ENOUGH?
> 
> 
> To your point about feedback from pure ACKs and data being combined, that's 
> not driven by any particular CCA. If ACKs are
> ECT, AccECN is designed to make it possible for the Data Sender to estimate 
> how much of the combined per-packet feedback is
> from ACKs and data, by using the per-byte feedback in the option to estimate 
> the number of data packets. This might not be
> perfect, but it was recognised from the start that we would need to decide on 
> compromises,, particularly given very limited
> header space.
> 
> Nothing in the increment-triggered ACK rule was designed around ECN++. 
> Requiring the receiver to emit an ACK after 'n' CE
> marks is a fairly obvious way of  ensuring that the sender  gets timely 
> feedback of multiple CE markings.

[MK3] Please see my reply to Richard where I explained why this feedback with 
Acks of Ack is in almost all scenarious unnecessary or not timely. In a single 
case when the other end first is idle but starts sending data it might be 
useful but even in such scenarious the feedback often comes too late or 
possibly is stale (if there is longish delay before the other end starts 
sending).

> 'n' has to be less
> than 7 to prevent AccECN's 3-bit ACE field wrapping. And greater than 2 to 
> avoid too many ACKs. Then, as I said, we noticed
> that could cause ping-pong, so we raised the min to 3 if the ACK would be in 
> response to an ACK. That would be necessary for
> protocol completeness (see above) whether or not an ECN++ draft existed.

[MK3] Raising the min to 3 helps in dampening the ping-pong effect. But still 
the ping-pong may last several RTTs if the cwnd is large and lot of pkts (most) 
get CE marked. Not sending feedback for CE-marked DupAcks (and not counting 
them in the counters) would be much more efficient way to dampen the ping-pong 
effect. In addition, it would avoid making unnecessary rate reductions in such 
cases (when DupAcks are arriving) where there will be a notable rate reduction 
anyway (the increased Ack rate due to DupAcks being sent for every data pkt 
goes away roughly in one RTT and the end receiving DupAcks will react to 
congestion and reduce cwnd, resulting in lower ack rate for the next RTTs). Pls 
see my reply to Richard.
This is another reason why this part of the rule is experimental in my opinion: 
we don't have any evidence/experience what is the best way to provide 
feedback/react to CE-marked pure Acks.
It would be useful if you could give one scenario where acking CE-marked 
DupAcks would give some benefit that cannot be (easily) solved(gained in some 
other means.

> You raise a concern that allowing Acks of Acks might preclude future algos 
> that rely on the absence of 'fake dupacks'.
> Indeed, you say that contravening the one ACK per data segment rule of 
> RFC5681 "will break all mechanisms that rely on the
> behavior specified in RFC 2883, including RFC 3708 and (TLP of) RACK (RFC 
> 8985) as well as any potential new and useful
> algos that rely on this rule." But I think this is all stated rather 
> over-dramatically, because you admit that the SACK
> condition allows an implementation to identify 'fake dupacks'. So it can 
> exclude them from all such present and future
> algos. Then this all becomes a non-issue.

[MK3] Please see above. My apologies if I was not clear enough for this part. I 
intended to say that SACK would help in distinguishing "fake dupacks" that will 
otherwise trigger a false fast rexmit. However, SACK seemingly does not solve 
all problems (e.g., breaks f-RTO). Unfortunately I have no time to pass through 
all potential algos that may break. Even less time to consider all potential 
new algos that could count on the fundamental rule for generating only one 
DupAck per arriving data segment. I just try to point out the ones that I found 
being problematic. Even if one existing and (widely) deployed algo gets broken, 
that should be strong enough warning signal to not proceed as a stds track 
feature.

> _______________
> I haven't added any further discussion to the thread below (except a couple 
> of minor points). To check that the above 4
> points comprehensively address all your [MK2] points. I've tagged each with 
> [BB3] and pointer(s) to the relevant response(s)
> above.

[MK3] I hope the above clarifications help. I didn't add anything below except 
one comment (taggd [MK3]). I may pass through the comments below later and 
comment if need be.

Thanks,

/Markku

> On 11/07/2023 16:48, Markku Kojo wrote:
>       Bob, please see [MK2] inline.
>
>       And apologies for the long delay in replying, again.
>
>       On Mon, 5 Jun 2023, Bob Briscoe wrote:
>
>             Markku, pls see [BB2] inline
>
>             On 26/05/2023 22:56, Markku Kojo wrote:
>                   Bob,
>
>                   My apologies you had to wait for the scenarios as it took 
> much longer with my limited cycles
>             than I thought.
>                   Anyways, please see my reply to Richard, some scenarios are 
> also included there.
>
>                   To keep things easier, it might be good to try to keep the 
> discussion on Acks of Acks (mainly)
>             in the thread
>                   with my reply to Richard.
>
>                   However, see inline tagged [MK].
>
>                   On Wed, 24 May 2023, Bob Briscoe wrote:
>
>                         Markku,
>
>                         Sorry, it's taken a week to build a comprehensive 
> reply to this long email. See inline tagged [BB]...
>
>                         On 17/05/2023 12:24, Markku Kojo wrote:
>                               Hi Michael, all,
>
>                               On Sun, 14 May 2023, tuexen@fh-muenster.de 
> wrote:
>
>                                           On 30. Mar 2023, at 16:53, Bob 
> Briscoe <ietf@bobbriscoe.net>
>                                           wrote:
>
>                                           Michael, Yoshi, Ian (as tcpm 
> chairs),
>
>                                           To close off the WGLC, I have just 
> posted a new rev of
>                                           accurate-ecn. Hyperlinks quoted at 
> the end.
>                                           You will see the diff is rather 
> extensive. I won't give a
>                                           summary of all the diffs like I 
> usually do. Instead I can just
>                                           refer to the summary I gave in the 
> presentation on Monday:
>                                          
>                        
>             https://datatracker.ietf.org/meeting/116/materials/slides-116-tcpm-draft-ietf-tcpm-accurate-ecn
>
>                                           Thank you again to the people who 
> reviewed this during the WGLC:
>                                           Michael Tüxen, Alex Burr, Gorry 
> Fairhurst and Markku Kojo.
>
>                                           All changes are editorial, apart 
> from removing the para about
>                                           not mistaking certain ACKs of ACKs 
> for DupACKs, which I will add
>                                           to a rev of the ECN++ draft, 
> hopefully later this week.
>
>                                           On the list, we have seen agreement 
> from all the reviewers to
>                                           these changes, except no response 
> from Markku yet.
>                                           On Monday, I told Markku that I 
> would post the draft in a few
>                                           days, so everyone can see the 
> updates and diff.
>
>                                     Anyone having additional comments? In 
> particular Markku regarding loss
>                                     recovery?
> 
>
>                               My apologies for being late with my reply to 
> the author's comments on my review
>             (I've
>                               been extremly busy with other issues since the 
> wg mtng in Yokohama, including the
>             rest
>                               of mtng week).
>
>                               I don't have much new comments but it seems 
> that my major concern regarding the
>             problem
>                               of sending ACKs of ACKs was not fully 
> understood.
>
>                               The first thing where I think I was not quite 
> clear is that the major problem with
>             ACKs
>                               of ACKs is not that a pure Ack is made 
> ECN-capable. Instead, the problem is in
>                               generating an Ack of an pure Ack and that is 
> what one should prohibit to avoid
>             problems.
>                               I understand that it might be problematic to 
> formulate rules whether generating an
>             Ack
>                               of an Ack is allowed (and when), instead of 
> just disabling sending ECN-capable
>             ACKs.
>                               I don't have a strong opinion which way the 
> problems with ACKs of ACKs is avoided
>             as
>                               long as they are avoided.
> 
>
>                         [BB] See later after your similar point (following 
> your 'Why?' heading)...
> 
>
>                               I am preparing a few scenarios to illustriate 
> the problems ACKs of ACks raise and
>             will
>                               send them shortly once I have formulated a more 
> thorough reasoning why sending
>             ACKs of
>                               ACKs is not really a good idea and even seems 
> to be unnecessary in most if not all
>                               cases, i.e., it just results in sending 
> unnecessary packets with not much useful
>             effect
>                               but creates a notable number of problems.
> 
>
>                         [BB] Having waited this long, it's rather 
> disappointing to still hear you say "I have an
>                         argument,
>                         but I'll tell you later."
> 
>
>                   [MK] I understand. My sincere apologies again.
>
>                               Given that this draft is intended to become a 
> stds track RFC I am concerned of any
>             text
>                               in this document that indicates (or even hints) 
> that TCP could acknowledge pure
>             ACKS
>                               (this holds particularly the rules and text in 
> Sec 3.2.2.5.1 for the
>                               "Increment-Triggered ACKs"). If it is seen 
> necessary that this doc should have
>             such
>                               pieces of rules and text, I am fine if any such 
> text is moved to an appendix as
>             long as
>                               the appendix makes it cristal clear that the 
> text is valid only in case one is
>                               implementing an experiment as per 
> [I-D.ietf-tcpm-generalized-ecn].
> 
>
>                         [BB] See point below about "Generic (Mechanistic) 
> Reflector".
> 
>
>                               Why?
>
>                               1) It is well known that TCP does not 
> acknowledge ACKs and Standards track TCP has
>             not
>                               been specified to acknowledge ACKs. This means 
> that a reader/implementer of this
>             doc
>                               cannot correctly understand the rule for 
> "Increment-Triggered ACKs" unless there
>             is a
>                               normative reference to a spec that specifies 
> ACKs of ACKs (or tells that it is
>             even
>                               possible).
> 
>
>                         [BB] ACKs of ACKs can indeed be tricky. But there's 
> no need to consider not ACKing ACKs
>             as an
>                         architectural principle. Not Acking ACKs on principle 
> certainly avoids some tricky
>             problems.
>                         However, we have a new situation here where, in 
> limited circumstances, ACKs of ACKs are
>                         necessary.
>                         So the WG has already worked through the tricky 
> problems and they have been addressed in
>             the
>                         draft
>                         (e.g. mistaking ACKs of ACKs for DupACks, infinite 
> ping-pong, etc). We'll discuss below
>             whether
>                         you've found some more trickiness.
> 
>
>                   (MK] I did not mean to refer to any principle but, as I 
> said, that a reader/implementer cannot
>             correctly
>                   understand the rule for "Increment-Triggered ACKs" because 
> it is well-known to her/him that
>             TCP does not Ack
>                   ACKs. This fact is that one can ack ACKs is not specified 
> in this doc
> 
>
>             [BB2] The fact that one /can/ ack ACKs is specified in the AccECN 
> draft. It clearly says:
>                 "Although TCP normally only ACKs newly delivered data, in 
> this case it is mandatory for A to
>             emit ACKs of ACKs
>             because they feed back new congestion state (useful in case B 
> starts sending)."
>
>             We'll also make it clearer that "in this case" means "in the case 
> of ECN-capable ACKs". And we'll
>             make it clear that it's
>             conditional on carefully following all the conditions in any 
> draft that specifies ECN-capable ACKs.
> 
>
>       [MK2] Right, but this (acking Acks) is experimental behaviour, needed 
> only for those participating the
>       experiment. However, this doc specifies the behaviour in the 
> Increment-Triggered ACKs rule: MUST emit an ACK".
>       If one implements this draft but is not interested in ECN++ nor any 
> other similar experiment that allows
>       ECN-capable ACKs, one must still implement these Acks of Acks but there 
> is no specification nor normative ref
>       how to do it (correctly, including all the conditions). If then this 
> implementation is deployed in the wild, it
>       automatically becomes part of the experiment once the other endpoint 
> enables ECN-capable ACKs but without the
>       necessary safety procedures. This doc cannot depend on an experimental 
> spec which explains how to do it and what
>       conditions to follow.
> 
> 
> [BB3] See
> #1) Generic should not be confused with experimental.
> #2) Completeness
> #3) Does not specify the necessary details?
>
>                   nor does this doc give a (normative) reference where it is 
> specified,
> 
>
>             [BB2] As already explained, there is no normative ref where 
> acking ACKs is specified in general,
>             because they are not
>             needed in general. They are specified whenever they are needed, 
> including in the examples Yoshi
>             pointed to (keepalive,
>             etc).
> 
>
>       [MK2] An implementation of this draft send Acks of Acks "randomly" and 
> possibly multiple of them in a row but
>       this draft does not specify the necessary details unlike you claim 
> above.
> 
> 
> [BB3] See #4) Does not specify the necessary details
> 
>
>       [MK2] Pls see also my reply to Yoshi. The specific examples Yoshi 
> mentions are sent in very limited cases and
>       result in sending only a single Ack of an Ack which is not causing any 
> harm to others unlike sending multiple
>       "random" Acks of Acks.
>
>                   including the details on which TSecr value to add or which 
> SACK info if any to include when
>             acking a pure
>                   Ack.
> 
>
>             [BB2] OK.  But I've had to read all the through SACK and TS specs 
> to guess what complication you
>             might have seen. If you
>             know what made you ask this question, it would be much easier if 
> you gave us a clue, or even just
>             told us explicitly.
> 
>
>       [MK2] Sorry for not being clear/detailed enough. Maybe this was too 
> obvious to me. See details below for SACK
>       (and my reply to Richard for Timestamps for which I believe you already 
> figured out the problem).
> 
> 
> [BB3] See #3) Does not specify the necessary details?
> 
>
>             * For TS, I'm pretty sure the normal rules (for which TSecr value 
> to include) apply.
>             * Similarly, for SACK, I can't see that anything new needs to be 
> said.
>
>                   It is easy to misinterpret the "Increment-Triggered ACKs", 
> if one doesn't realize that pure
>             Acks may be
>                   acked.
> 
>
>             [BB2] We'll make it clearer in the rule that 'packet' 
> intentionally includes packets without data.
> 
>
>       [MK2] But as I said earlier, it would specify experimental behavior in 
> a PS doc.
> 
> 
> [BB3]  See: #1) Generic should not be confused with experimental.
> 
>
>                         What is the new situation?
>                          *  Until ECN was introduced, TCP ACKs only 
> acknowledged data. So there was no need to
>                         acknowledge
>                             pure ACKs, which contain no data.
>                          *  When ECN was introduced in RFC3168, TCP ACKs also 
> acknowledged ECN markings.
>             However, because
>                             RFC3168 precluded pure ACKs from being 
> ECN-capable, there was still no need to
>             acknowledge
>                         pure
>                             ACKs.
>                          *  RFC5690, and now the ECN++ draft introduce the 
> possibility of ECN-capable pure ACKs.
>             So, in
>                         the
>                             limited circumstances described in the AccECN 
> draft, ECN-capable pure ACKs now need
>             to be
>                             acknowledged, because they contain new 
> information - their ECN field.
>                         Similarly, even though the final ACK of TCP's 3WHS is 
> an ACK of an ACK , it is sent
>             because it is
>                         needed (to prove that the SYN wasn't from a spoofed 
> address).
> 
>
>                   [MK] All otherwise clear, but I disagree that the final ACK 
> of TCP's 3WHS is an ACK of an ACK.
>             It is required
>                   because SYNACK contains control data that eats one sequence 
> number, i.e., it advances RCV.NXT
>             at the client
>                   end and when the ACK arrives at the server it is needed to 
> advance SND.UNA. Very different
>             from Acks of Acks
>                   in this draft.
> 
>
>             [BB2] Yes, not the same, but similar - ECN information is also 
> control data. But agreed that it
>             doesn't consume a
>             sequence no. Neither do keepalive probes or their ACKs.
> 
>
>       [MK2] Right, but the actual purpose of Acks is to acknowledge sequence 
> numbers (that have arrived) and the logic
>       of many crucial TCP algos depend on how packets are acked.
> 
> 
> [BB3] Good protocol specs never define or limit the uses of the protocol (it 
> would be pointless to try anyway).
> 
> If you only wanted ACKs to be used to acknowledge sequence numbers, you 
> should have objected to RFC3168, which used ACKs to
> feed back ECN as well. Or you could go further back and object to SACK 
> because it acknowledges gaps, not sequence numbers
> that have arrived.

[MK3] With both RFC 3168 and SACK the ACKS ack the sequence numbers in the 
first place. These specs just piggyback additional information on ACKS that 
would be sent anyway. They do not introduce extra pkts to be sent like this 
spec does. I think this makes a major difference.

[snipped the rest of msg as the msg was too long for the wg list]