Re: [tcpm] Possible error in accurate-ecn

"Scheffenegger, Richard" <> Wed, 24 February 2021 18:33 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 7F30B3A183A for <>; Wed, 24 Feb 2021 10:33:56 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FROM=0.001, NICE_REPLY_A=-0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 2HJtpZV4bi69 for <>; Wed, 24 Feb 2021 10:33:54 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id B34B33A197D for <>; Wed, 24 Feb 2021 10:33:53 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple;; s=badeba3b8450; t=1614191576; bh=7Lr0CeQBWKj8J9W3MmDMxPobMoS/z178qtSM4QnSljk=; h=X-UI-Sender-Class:Subject:To:Cc:References:From:Date:In-Reply-To; b=j4sFwe/JBESuWSY/4wFi2hB4qFa0dwwcLM3HKUOe0TE+T/TaQbarwsvK8KyGPzmiS Ttnb4pk+XLyH03eNv1RXhPFe4mbxmzpQozIIXTZn8k4RpZjsvLy3za+h/I32CXU+W9 5m1MhvfosJgJPGOhmbxAcWdoS+1wxuUABksvthg0=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from [] ([]) by (mrgmx005 []) with ESMTPSA (Nemesis) id 1McY8d-1lpPa10cvd-00d01R; Wed, 24 Feb 2021 19:32:56 +0100
To: Yoshifumi Nishida <>, Bob Briscoe <>
Cc: Mirja Kuehlewind <>, Yoshifumi Nishada <>, tcpm IETF list <>, Richard Scheffenegger <>
References: <> <> <> <> <> <>
From: "Scheffenegger, Richard" <>
Message-ID: <>
Date: Wed, 24 Feb 2021 19:32:52 +0100
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Transfer-Encoding: quoted-printable
X-Provags-ID: V03:K1:YpKADUI1EmowLDeRr8kLqcpgz8Tk4NMYIRbAcpmKN5zbsr4dsST M4EIQw6bh9rnVB7nhgBlJ1hYGu0TlQVPoCMTCn2G+za/77CQLRjAVkitAWdHlPtCHujaghr N8rKWAId0rBpO+G61XHlMfgtPzhkf26Kl9jupiUPUBVnG4QMho834yq4PnyoNgpahIrTda/ GnsnUcSZ+S5kko5UWyLaQ==
X-UI-Out-Filterresults: notjunk:1;V03:K0:jb4PM/Wy5k0=:FJP2iO8RE71MZH+wpyinZA 8ryJlsNPGY2lCnyBh9frGo+xgxuh3vjsj2BdEa5cgLb6IZCPNjlPc5yprU2y0LsLiv+HnYZCO axbrD1dIrrGcrQiGcEwrqd2G4ibT1hyTLp/HqDseVWgvk2o8KOf2cWiDEl+PUdDgO4Fr0MyUQ rxdf6QH01Zn/0lQwVT3vvEGAWHPy3g685049xCJeEHUVnVfq0aeHFMNU0oXMXIlh3YMXiV/FO lUvl+R0BMMJlKMh6ZaZug33UaFS6kaSwg9Rmcjr9jVc69MBdrzOWmoSPMM2ikgiBG6mwv/yAv FEOA39fzI6f65UQWH8r33w3QVRnDd1TSPVtGV6aPH6eZltsp19/He2rCkAaq74uD71BXnrekM pEf69rN4dSBP9OqIxOPtf4Tca09TsThUpqbB664mi5icHkn0eDZq33SDAthPUVcIeeksZWZpA dRGx/07EbB9II/MjtYkfxW8mWJ+KLK6twMtnuX6BQWFL2okCPzSJLL9yemW8r9W9njkXClrsO DhJKH8yzNvFNUpi1aJtBy2KpOfOh6KR0150uprI0Civ/Vw6e5zTkvP1Vdum/XXacKLQKb1isQ 1rnr7jLF+CgD89tyszx0DQqnuNey5hZ+EGfGGCwUD23mUItq2R6EsPKi+7eAKhDfTZv4q+ORA D4zB2Yj1HkG3eJKw8WfgKKsfD0oNv5YLUaKzpAvfGze2d0rSXQp30F8HkiJN3jQdb6fc3zQhs OMSJ/jKcM5pEBetV7cSiXhPLffnzuuB83clNtKxQd8+AehrQLPnqv5qbSjCFLJaNp/Rr4fci9 Nw9sd3RLBsjwemIxEL4v3TwM7KAFRTNPcL5sIF9P5O26HBaIomzV0LuUO0uLpWI5i5OZYvUOq o4Awy6d7sH9cWLRqRayw==
Archived-At: <>
Subject: Re: [tcpm] Possible error in accurate-ecn
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 24 Feb 2021 18:33:57 -0000

Hi Yoshi,

A valid concern. Two ways to remediate these at least to some extent:

a) used a "n" higher than 2 (e.g. 6)
b) refrain from sending the ACK for n CE immediately, but hold off for
either the delayed ack timeout, or the n+1'th receipt of CE, or a valid
data segment by that time.

While this would not fully rule out any possibilty of an inadvertend
DupAck, it significantly reduces the time window when this may happen.

Also, if an implementation uses other clues (e.g. TS opt, SACK
negotiated but not present in the ACKs incrementing the CE.P ),
ambiguity can be further reduced.

However, I feel that these optimizations may overburden the definition
of the mechanism.

Best regards,

Am 24.02.2021 um 11:03 schrieb Yoshifumi Nishida:
> Hi Bob,
> On Mon, Feb 22, 2021 at 4:12 PM Bob Briscoe <
> <>> wrote:
>     Mirja, Richard, Yoshi,
>     I just posted a new rev of AccECN for a number of other reasons, and
>     realized we hadn't converged on an answer to the tricky problem in
>     this old thread from Nov'20. For now, the text says the following
>     (which I've justified below, but I don't think it's the solution
>     yet), and I've asked Richard & Mirja if we (as co-authors) can act
>     as a little design team to thrash out this problem in the next few days.
>  Data Receiver Safety Procedures
>         An AccECN Data Receiver:
>         o  SHOULD immediately send an ACK whenever a data packet marked CE
>            arrives after the previous packet was not CE.
>         o  MUST immediately send an ACK once 'n' CE marks have arrived since
>            the previous ACK, where 'n' SHOULD be 2 and MUST be in the range 2
>            to 6 inclusive.
>     Justification for first bullet: The aim is to trigger an ACK on a
>     transition from non-CE to CE, but only when the latest packet is a
>     data packet. Cases where the previous packet was a CE-marked pure
>     ACK and therefore not ACK'd ought to be picked up by the 2nd bullet,
>     not the first. Similarly, if there's CE-marked packets (data or pure
>     ACKs) that haven't been ACKd yet, and the next data packet is not
>     CE-marked, I think the first bullet is still correct, and the second
>     rule will pick up cases with enough CE marks.
>     The second bullet is more tricky. I've added Richard's suggesting of
>     flooring 'n' at 2. But made no other changes. That's deliberate,
>     'cos I think it's sufficient to allow ACK CC to be added, but not to
>     preclude it. And I think the danger of being interpreted as DupACKs
>     shouldn't apply if there's no outstanding data. I know I'm wrong in
>     those odd cases I mentioned when new data might cross the ACKs in
>     the other direction. But I don't think that's going to cause too
>     much harm.
> This might be too pathological, but I'm thinking about a certain extreme
> example..
> In my understanding, let's say a node sends 50 ACKs for some reasons and
> all ACKs have CE marks, then its peer sends back 25 ACKs.
> If these 25 ACKs have CE marks as well, it can generate another 12 ACKs,
> and so on.
> If this understanding is correct, I'm not sure if this is a good behavior.
> I am thinking we had better do something here such as don't send an ACK
> if the correspond ACKs carries accecn signals.
> Also, I think it's hard to tell there's no outstanding data from the
> receiver side.
> Let's say a sender send some data packets after sending 10 acks, and the
> receiver sends back 5 acks as the 10 ACKs have CE marks.
> In this case, they look like dup acks from the sender's point of view.
> I know this is a rare case, though.
> --
> Yoshi
>     On 17/11/2020 08:48, Scheffenegger, Richard wrote:
>>     See my suggestion; The critical requirement for the 2nd bullet
>>     point is,
>>     to have n > 1 for pure, CE-marked ACKs. The exponential decay,
>>     after how
>>     many ACKs triggered by CE-marked ACKs these ACK-for-ACK stop,
>>     strongly
>>     depends on "n", so the recommendation should be to use a higher
>>     value of
>>     n in the case where you only observe a stream of pure ACKs (with or
>>     without marks; only those marked would account against "n" though).
>>     In the case of data packets mixed in, you don't want to delay
>>     excessively, thus "n" may be choosen lower...
>>     Richard
>>     Am 10.11.2020 um 17:38 schrieb Bob Briscoe:
>>>     Moving on to the second bullet...
>>>>>     The second bullet doesn't consider the possibility that the
>>>>>     'n'th CE
>>>>>     mark might arrive on a pure ACK. Then, the wording as it stands
>>>>>     says
>>>>>     the Data Receiver MUST immediately ACK a pure ACK. I know TCP
>>>>>     never
>>>>>     ACKs a pure ACK, but I'm not actually sure it does any harm to
>>>>>     do so
>>>>>     in this case (it cannot cause an infinite loop of ACKs). However,
>>>>>     given it would be unorthodox, we maybe ought to rule it out by
>>>>>     rewording anyway?
>>>>     [MK] I think we also can leave this as it is because if that ’n’
>>>>     packet is a pure ACK that still means that you have unacked data
>>>>     packets with CE marks and you should trigger an ACK for those
>>>>     packet
>>>>     now (rather than waiting for the delayed ACK timer to expire).
>>>>     However
>>>>     this case is less important. Also we should probably make sure that
>>>>     this doesn’t apply if there are only pure ACKs with CE marks,
>>>>     maybe by
>>>>     add “if unacknowledged data are outstanding” or something.
>>>     [BB] By the end of your last para you start to realize that, if
>>>     the n'th
>>>     CE mark arrives on pure ACK, it doesn't say anything about
>>>     whether the
>>>     previous (n-1) CE marks were on data or pure ACKs.
>>>     Consider another common scenario; a server delivering chunks of
>>>     video
>>>     over a high BDP path (e.g. 500 packets per RTT), and going idle
>>>     between
>>>     times (perhaps in response to an "exceeded high watermark" data
>>>     packet
>>>     from the server). Now consider the possibility that the reverse path
>>>     bottleneck builds a queue during each chunk, and starts CE marking
>>>     towards the end of each chunk. Then, after the server stops
>>>     sending a
>>>     chunk, it continues to see a couple of hundred CE-marked pure ACKs
>>>     arriving.
>>>     However, by your proposed rule, it doesn't send any feedback,
>>>     'cos it
>>>     has no data to acknowledge. So the client cannot regulate its ACK
>>>     stream. That's just tough, 'cos the client will stop sending more
>>>     pure
>>>     ACKs after it has received the last data, which is before any later
>>>     feedback from the server can reach it.
>>>     However, if the client later sends a data packet that the server can
>>>     acknowledge (e.g. "a low watermark" message), the server's ACE
>>>     counter
>>>     will have wrapped dozens of times, and the client won't be able
>>>     to work
>>>     out how many. That doesn't matter if a few round trips have
>>>     elapsed -
>>>     the congestion info will have become stale. But it does matter if
>>>     it's
>>>     fairly soon.
>>>     Instead, if we keep the text of the second bullet... if the
>>>     server's 'n'
>>>     was 2, it would send a pure ACK for every other pure ACK it
>>>     received.
>>>     But, what if these pure ACKs from the server were also CE-marked?
>>>     Then
>>>     the client would feed back pure ACKs using its 'n'. This regression
>>>     would eventually die out, but I don't like it.
>>>     This conversation about the second bullet presumes AccECN is
>>>     being used
>>>     for ACK congestion control. I'm not comfortable with writing
>>>     anything
>>>     specific about ACK congestion control into a draft on the standards
>>>     track. But I don't want to preclude AccECN being used for ACK CC.
>>>     I believe the second bullet needs something changed to limit the
>>>     ACKing
>>>     of pure ACKs. However, before we put effort into wordsmithing,
>>>     I'd like
>>>     to hear whether you agree with the general idea of not defining
>>>     what to
>>>     do for ACK CC, but not precluding it.
>>>     Cheers
>>>     Bob
>>>>     Mirja
>>>>>     Thoughts anyone?
>>>>>     Bob
>>>>>     --
>>>>>     ________________________________________________________________
>>>>>     Bob Briscoe
>>>>> <>
>>>>     _______________________________________________
>>>>     tcpm mailing list
>>>> <>
>>>>     <>
>>     _______________________________________________
>>     tcpm mailing list
>> <>
>>     <>
>     --
>     ________________________________________________________________
>     Bob Briscoe  <>
>     _______________________________________________
>     tcpm mailing list
> <>
>     <>