Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text

Christian Huitema <> Mon, 02 November 2020 06:50 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 904E63A0E0D for <>; Sun, 1 Nov 2020 22:50:47 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -2.135
X-Spam-Status: No, score=-2.135 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, NICE_REPLY_A=-0.247, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id xKtkvTFVtNCG for <>; Sun, 1 Nov 2020 22:50:44 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id B82AB3A0E02 for <>; Sun, 1 Nov 2020 22:50:44 -0800 (PST)
Received: from ([] by with esmtp (Exim 4.92) (envelope-from <>) id 1kZTft-0015HR-UN for; Mon, 02 Nov 2020 07:50:43 +0100
Received: from (unknown []) by (Postfix) with ESMTPS id 4CPk805pgsz8lm for <>; Sun, 1 Nov 2020 22:50:40 -0800 (PST)
Received: from [] ( by with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.92) (envelope-from <>) id 1kZTfs-0000O3-Ln for; Sun, 01 Nov 2020 22:50:40 -0800
Received: (qmail 17649 invoked from network); 2 Nov 2020 06:50:40 -0000
Received: from unknown (HELO []) ([]) (envelope-sender <>) by (qmail-ldap-1.03) with ESMTPA for <>; 2 Nov 2020 06:50:40 -0000
To: Greg White <>, "De Schepper, Koen (Nokia - BE/Antwerp)" <>, Jonathan Morton <>
Cc: iccrg IRTF list <>, tsvwg IETF list <>, TCP Prague List <>
References: <> <> <> <> <> <> <> <>
From: Christian Huitema <>
Autocrypt:; prefer-encrypt=mutual; keydata= mDMEXtavGxYJKwYBBAHaRw8BAQdA1ou9A5MHTP9N3jfsWzlDZ+jPnQkusmc7sfLmWVz1Rmu0 J0NocmlzdGlhbiBIdWl0ZW1hIDxodWl0ZW1hQGh1aXRlbWEubmV0PoiWBBMWCAA+FiEEw3G4 Nwi4QEpAAXUUELAmqKBYtJQFAl7WrxsCGwMFCQlmAYAFCwkIBwIGFQoJCAsCBBYCAwECHgEC F4AACgkQELAmqKBYtJQbMwD/ebj/qnSbthC/5kD5DxZ/Ip0CGJw5QBz/+fJp3R8iAlsBAMjK r2tmyWyJz0CUkVG24WaR5EAJDvgwDv8h22U6QVkAuDgEXtavGxIKKwYBBAGXVQEFAQEHQJoM 6MUAIqpoqdCIiACiEynZf7nlJg2Eu0pXIhbUGONdAwEIB4h+BBgWCAAmFiEEw3G4Nwi4QEpA AXUUELAmqKBYtJQFAl7WrxsCGwwFCQlmAYAACgkQELAmqKBYtJRm2wD7BzeK5gEXSmBcBf0j BYdSaJcXNzx4yPLbP4GnUMAyl2cBAJzcsR4RkwO4dCRqM9CHpVJCwHtbUDJaa55//E0kp+gH
Message-ID: <>
Date: Sun, 01 Nov 2020 22:50:39 -0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.12.1
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: multipart/alternative; boundary="------------86283D5CADDA09379546A1CF"
Content-Language: en-US
Authentication-Results:; auth=pass smtp.auth=
X-Spampanel-Outgoing-Class: unsure
X-Spampanel-Outgoing-Evidence: Combined (0.15)
X-Recommended-Action: accept
X-Filter-ID: Mvzo4OR0dZXEDF/gcnlw0XvADx2zSFwG+3csxFBPBHmpSDasLI4SayDByyq9LIhVUZbR67CQ7/vm /hHDJU4RXkTNWdUk1Ol2OGx3IfrIJKywOmJyM1qr8uRnWBrbSAGD42+pgDSJdnVPeXOEK1m6uDbl kDZNUkuHiojIVh7uAUtMmQhgAlZ6hsyieRVSg8TDUZq99/v+OWrp+SY7iSuea8EuNfnjQsRqWXYP fdewBkdPR9o014ICOs9pIWX8OtLJdwmMW2Igo3slaTKxvBsN69bNhTNyx2YOvjKTJ3Ps/x2i3GSv wir0OshyEkOwfCYvxs5/T0oXYyKdvWabEYxQFC5Ano+acIK7aoZBBKN3kq8lTiPCf+PwTb/RKAdw /TswIg1fYmNb1kgzViAoNrSrXN1jhnM/Mbva2XLV/LIEzaJm4kYWBU//ZZ8pZ1xZfmaG1NRsUdzW awx6dX0NJ8Bzt99fxN2oReTDHAyOynaY0ClENaQq/2aUAxcG3yLqjApZmdySlZou9qHIGOZDEEo7 O58ZQzrOqjAERHu4pt/Ia6wELzcGxDgkPe7eR6qspNNQGjLhGMBSrFdf8dBbPvtqJwEiRQv+PVjj wa+Z5RFCOMQa1Vus00n+QjkVg8Y9yuTVR7/dvDWlsWWmYvtoo/dCwrK6HflbskIiJsH3WXieD/iI 9kVgrP8d/S99xR9aHf3T0ryXeQDqqkgHDboVrOy1pkXw8BMIsfBCWTvqwtHGsGbsxBoFbdIyTZwD 0tTyhMSsHScFlGkjbz0OvhDlTjXQPYaPmF/7MAKyW1Kb4FKGpjSTHLN3gEp08lb+iT+JYfiyGO8m 5xifdQ6v1Zv869jbIA2MtuoXMRfSG2WhEZRm53tcFvYT7FcUL8JfnIu/hBVs647lNwN4qOsSZg+f YhVZG3Fpf9P2WwOdPRT3EWlMd754CMBw7+ErSKQN7lw8ITGaNmGHCZF9ZXtfSxIkZLsbHZnckpWa LvahyBjmQxBKOzsmHW0N+NPiHPgn+dyxhvT1Z5EPOIlpyGKEA1x50oTQDUtseoetsROuM82YMzIj 8EU=
Archived-At: <>
Subject: Re: [tcpPrague] [tsvwg] ecn-l4s-id: Proposed Changed to Normative Classic ECN detection Text
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "To coordinate implementation and standardisation of TCP Prague across platforms. TCP Prague will be an evolution of DCTCP designed to live alongside other TCP variants and derivatives." <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 02 Nov 2020 06:50:48 -0000

On 11/1/2020 9:58 PM, Greg White wrote:
> On 11/1/2020 3:55 PM, Christian Huitema wrote:
>     On 11/1/2020 1:53 PM, De Schepper, Koen (Nokia - BE/Antwerp) wrote:
>         >>By the way, I do think that L4S makes too many hypotheses
>         about the CC. I wish the AQM was based on intrinsic
>         properties, such as "this flow is using more than its fair
>         share of bandwidth"
>         This last quote is exactly what the L4S marking is intended to
>         do: the smoothed marking probability over a longer time
>         interval (about 10 RTTs) indicates the fair target rate.
>         Assuming we agree on 15ms being the reference RTT, then the
>         average rate can be r=2/(p*0.015)=133/p packets per second. So
>         you can evaluate yourself if you use more or less than the
>         fair share. How you respond to this knowledge is up to your CC.
>     I think that part of the complexity in L4S AQM comes from trying
>     to achieve fair sharing of resource without actually implementing
>     fair queuing. You end up expecting that implementations apply a
>     formula and derive their packet sending rate from a smoothed
>     average of the packet marking rate. It may work well in controlled
>     environments, but I am afraid that very few implementations are
>     ever going to do that in practice.
>     Congestion Control algorithms are much more likely to treat packet
>     losses, markings, or delay increases as signals that they are
>     sending too fast, while considering their absence or lower
>     frequency as signal that they might be able to send faster.
>     Implementations also are likely to take into account the recent
>     history, for example monitoring the min RTT (LEDBAT, BBR), or
>     monitoring the recent available bandwidth (BBR). They also
>     typically filter signals, e.g., consider only one packet loss or
>     ECN mark per RTT (New Reno, Cubic, etc.). This is expected,
>     because we do see packet losses arriving in batches, and can
>     reasonably expect ECN marks to have the same behavior. But
>     implementations are inherently selfish, do not really trust the
>     signals from the network, and are very unlikely to end up applying
>     the formula that you suggest.
>     In any case, you have a scaling issue. Let's consider a 1.5Gbps
>     link, with 15 ms delay and 1500 bytes packets. The nominal sending
>     rate is 125,000 packets per second. The marking rate with your
>     formula shall be p = 2/(r*0.015), which is about 0.0008%. Over the
>     last 10 RTT, the connection will on average see 0.14 marks -- that
>     is, no mark over the last 10 RTT 86% of the time. This falls well
>     short of the requirement to provide frequent feedback!
> Sorry, bug in my spreadsheet. The marking rate is actually 0.1%, about
> 2 packets per RTT. Still not frequent enough for my taste, but much
> better than 0.0008%. Nevertheless, the inverse relation between
> marking rate and data rate is not great.
> [GW] But, this always works out to 2 marks per RTT or 2 marks per 15ms
> (whichever is slower), so in that sense it doesn’t depend on data rate
> at all.  The sender slows down if it sees more frequent marks than
> this, accelerates if it sees fewer marks than this, and stays put if
> it sees marking at this rate.
Yes, there is that. But even if the rate is large enough, the CC
implementations can only program such a constant if they have assurance
that every bottleneck that they encounter will use the same time
constant. I don't think we are ready to make such assumptions outside of
controlled experiments. I would expect a mix of L4S-AQM, CODEL and PIE
variants, classic ECN, etc. The host implementation has to be ready to
handle all that, and trying to use a time constant there is very

>     If you give up the formula, you could end up with a significantly
>     simpler event based congestion control, in which applications slow
>     down if they see too many CE marks, accelerate if they only see a
>     very low marking rate, and stay put if they see an infrequent
>     marking rate. I can see applications doing that, resulting in a
>     fairly stable network and low delays. Of course, you will have to
>     do some form of fair share enforcement to catch violators, but I
>     am sure you can come up with something.
I still believe that this kind of "natural signaling" is easier to
handle than the formulaic behavior expected with L4S. It can easily
deliver the "low queing delay" goal of L4S. It cannot guarantee "fair
sharing of bandwidth", but my thesis is that such fair sharing requires
per-flow enforcement.

-- Christian Huitema