Re: [tcpm] Seeking WG opinions on ACKing ACKs with good cause (was: Possible error in accurate-ecn)

Yoshifumi Nishida <> Wed, 17 March 2021 07:23 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 2A7D43A03C9 for <>; Wed, 17 Mar 2021 00:23:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id UYsi1FV0zVuJ for <>; Wed, 17 Mar 2021 00:23:10 -0700 (PDT)
Received: from ( [IPv6:2607:f8b0:4864:20::82c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id DACD03A033F for <>; Wed, 17 Mar 2021 00:23:09 -0700 (PDT)
Received: by with SMTP id u7so701478qtq.12 for <>; Wed, 17 Mar 2021 00:23:09 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=jY5OoBnESXjpDo2+KS45LdKKbPmANNNnn2WKpwc1ZO0=; b=cm1p0XQqGEYa8pgfVXu2d6b+/ITH0k4EqeYBz+Q0+e2dYkO6MN+E508lK8xLQ7VME/ 2KLvFBPyaKm4w1T704UGRkvmwuEhKXREbt1riXW/93h230XqE2K9gf3aapWwzya4jm68 kpIh35PiJzh83K+ge3WEknIQWDhc1p8Cg4tPrBWbBETK7N+4bBnxkxZGnMoTQDMLfw31 umiJZkso2ky1wODwQR+uyadH+ydgaRjzqyYUKVKWakMAr8rUAlW76Qn6WxJsnppnFg7N yAFEFrZGNJaM6RDq26T+DE60SVrP23YGUykEeXtlvxPwi0E6SSu4cmzlRIDsyrgBLssK BF7Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=jY5OoBnESXjpDo2+KS45LdKKbPmANNNnn2WKpwc1ZO0=; b=kes7o1adiJZVLnoNNhiUfIMhUkjuKQfXfr+oMLgOzcO01d13qAmVtO8iTnA6mdfuEG VLwRvi0+1H8phKwQIpuqndaOYsrINfvByvYcGtpikzgsuVLnNEzMUThEO9n07Qru4lKr yiS/p1q6YDbXTqucfHRPeLx0WVfunJYv0a7Xnv//H57y+y9CgU/ueGdPQvbIQa7+bJ55 JrGNzDJt4SiNSV0a3iu2DhgEiJUsJfe0x900612jE8E0/9TZXiYF/ybaB4iJyStYTn0b jNXy7ftb9vdG6/x5pkRov/Pvy9YL5WM9XC7NrXTeeJMYzAtBHvL773XeSuTBJF4y7yFH hQ6w==
X-Gm-Message-State: AOAM532Ndh1t+ys7rzV6i3AuxDhrT/nEba115JptXaoB08/6jcfzNkwj RnB6zwKoxElJrJW05eL9+elGBHbgFK0j7s57lZQ=
X-Google-Smtp-Source: ABdhPJyCphQ7yGkDezAX0WSUODb5uuyhidzctV5wquTfy2bPyQmVRS18M6tOhkl7RLjQzEoNxmtWWbedBkfvmRdRUHw=
X-Received: by 2002:ac8:6690:: with SMTP id d16mr2746612qtp.312.1615965787891; Wed, 17 Mar 2021 00:23:07 -0700 (PDT)
MIME-Version: 1.0
References: <> <> <> <> <> <> <> <> <>
In-Reply-To: <>
From: Yoshifumi Nishida <>
Date: Wed, 17 Mar 2021 00:22:56 -0700
Message-ID: <>
To: Bob Briscoe <>
Cc: tcpm IETF list <>, "Scheffenegger, Richard" <>, Mirja Kuehlewind <>, Yoshifumi Nishada <>
Content-Type: multipart/alternative; boundary="000000000000c1ac1905bdb65aad"
Archived-At: <>
Subject: Re: [tcpm] Seeking WG opinions on ACKing ACKs with good cause (was: Possible error in accurate-ecn)
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 17 Mar 2021 07:23:12 -0000

Hi Bob,

On Fri, Mar 12, 2021 at 2:54 AM Bob Briscoe <> wrote:

> Yoshi, tcpm list,
> As promised we set up a small design team on the single issue of
> occasionally ACKing pure ACKs if they carry new info (ECN marking at the IP
> layer). The design team has two solutions and everyone would be prepared to
> accept either, but preferences differ. So we're seeking wider opinions
> (more opinions obviously won't narrow the choices, but at least the WG can
> then make a decision informed by those who care).
> To understand the question, you can either read the email below. Or the 3
> slides for the tcpm meeting are briefer, and include pictures:
> Here's the current draft text in question (see here for context
> ):
> Data Receiver Safety Procedures   An AccECN Data Receiver:
>    o  SHOULD immediately send an ACK whenever a data packet marked CE
>       arrives after the previous packet was not CE.
>    o  MUST immediately send an ACK once 'n' CE marks have arrived since
>       the previous ACK, where 'n' SHOULD be 2 and MUST be in the range 2
>       to 6 inclusive.
> Only the second bullet is in question. Here is the proposed diff for each
> alternative, then we explain:
> Alternative A
>    o  MUST immediately send an ACK once 'n' CE marks have arrived since
> -     the previous ACK,
> +     the previous ACK and there is outstanding data to acknowledge,
>       where 'n' SHOULD be 2 and MUST be in the range 2 to 6 inclusive.
> Alternative B
>    o  MUST immediately send an ACK once 'n' CE marks have arrived since
> -     the previous ACK, where 'n' SHOULD be 2 and MUST be in the range 2
> +     the previous ACK, where 'n' SHOULD be 3 and MUST be in the range 3
>       to 6 inclusive.
> Extra guidance text would be required in each case too (see the end).
> Background:
> AccECN is a change to the TCP wire protocol that requires the packet count
> of congestion feedback to include any congestion experienced (CE) arriving
> on Pure ACKs (amongst other things). AccECN doesn't require Pure ACKs to be
> ECN-capable, but allows for them to be. Similarly, AccECN doesn't require
> any congestion response to CE on pure ACKs, but having the feedback
> information there allows a response to be added with a one-ended update, if
> desired/necessary. Basically the data receiver is a 'dumb reflector'.
> The above two bullets were designed to ensure that an ACK is triggered a)
> on the first sign of congestion, and b) frequently enough for the count of
> CE markings to be fed back using the 3-bit ACE field before it wraps, even
> if an occasional pure ACK is lost.
> We then realized that the wording could require an ACK to be triggered in
> response to a CE-marked pure ACK. The circumstance when this could occur
> would be when peer X sends a volley of data to Y then stops, and the path
> back from Y to X is congested (probably by other flow(s) so that many of
> the ACKs are CE-marked. The second bullet above under alternative (B) would
> require X to ACK every 'n-th' CE-marked pure ACK. However, if Y immediately
> started sending a volley of data to X, Y could misinterpret those ACKs (of
> ACKs) from X as DupACKs.
> There are two ways to deal with this:
> A) Some of us prefer to completely prevent ACKs on pure ACKs, on the basis
> that they do not want to risk sometimes generating more ACKs today
> B) Others want to ensure that these rules will cause pure ACKs to be ACKed
> when the amount of CE on the ACKs merits it. But sparingly and strongly
> damping any ACK ping-pong.

Hmm. This is not easy..
My personal preference is if SACK is negotiated and it 's utilized to
distinguish dupacks, then (B) is fine for me. Otherwise, I would prefer

BTW, I am wondering if this could leave it to implementations since both
methods have pros and cons.
I think It's hard to tell one is much better than the order.


> There are complexity arguments on both sides.
> B more complex:
>     extra (non-mandatory) 'if' condition for lack of SACK options on a
> pure ACK (to decide it's not a DupACK).
> B less complex:
>     consistent handling of CE marking whether on pure ACKs or data (which
> would probably remove an 'if' condition).
> A more complex:
>     CE markings on a string of pure ACKs can build up without feeding them
> back, until released by a data packet (if ever).
>     More code at the other end to deal with the resulting risk of many
> wraps of the ACE field (or ignore?).
> A less complex:
>     less different from current TCP.
> Extra guidance text would be necessary in either case.
> * Alt A) would need text on handling the risk of many ACE wraps
>     (to be written).
> * Alt B) would need something like the following changes:
>     For the avoidance of doubt, the above change-triggered ACK mechanism
>     is deliberately worded to solely apply to data packets, and to ignore
>     the arrival of a control packet with no payload, because it is
> -  important that TCP does not acknowledge pure ACKs.  The change-
> +  important that TCP does not acknowledge pure ACKs which convey no new
> +  state information to the sender. The change-
>     triggered ACK approach can lead to some additional ACKs but it feeds
>     back the timing and the order in which ECN marks are received with
>     minimal additional complexity.  If only CE marks are infrequent, or
>     there are multiple marks in a row, the additional load will be low.
>     Other marking patterns could increase the load significantly.
> +
> +  Providing feedback on the congestion state of the return channel
> +  after a sender has ceased transmitting more data helps inform the
> +  clients TCP congestion controller about the state of the return path.
> +  Should the role of data sender and receiver subsequently change, the
> +  new sender has more up to date knowledge of the network state,
> +  preventing transmissions of inappropriate size at that moment.
>     Even though the first bullet is stated as a "SHOULD", it is important
>     for a transition to immediately trigger an ACK if at all possible, so
>     that the Data Sender can rely on change-triggered ACKs to detect
>     queue growth as soon as possible, e.g. at the start of a flow.  This
>     requirement can only be relaxed if certain offload hardware needed
>     for high performance cannot support change-triggered ACKs (although
>     high performance protocols such as DCTCP already successfully use
>     change-triggered ACKs).  One possible compromise would be for the
>     receiver to heuristically detect whether the sender is in slow-start,
>     then to implement change-triggered ACKs while the sender is in slow-
>     start, and offload otherwise.
> +   The second bullet creates a possible case where an AccECN
> implementation
> +   could sometimes ACK pure ACKs, which in turn might be mistaken for
> +   duplicate ACKs (in scenarios where TCP peers take turns to send
> +   sets of data packets). To prevent spurious transmissions in such
> +   circumstances, if SACK has been negotiated, an implementation could
> +   optionally assume that an ACK is not a Duplicate ACK if it has no SACK
> option,
> +   which would indicate it was an ACK of an  ACK. Alternatively it could
> use
> +   timestamp options to rule out DupACKs.
> Bob
> --
> ________________________________________________________________
> Bob Briscoe