Re: [tsvwg] "Pacing" requirement is lost in L4S

"Bless, Roland (TM)" <roland.bless@kit.edu> Fri, 19 April 2024 10:56 UTC

Return-Path: <roland.bless@kit.edu>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 37B15C14F6F2 for <tsvwg@ietfa.amsl.com>; Fri, 19 Apr 2024 03:56:48 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.397
X-Spam-Level:
X-Spam-Status: No, score=-4.397 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=kit.edu
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XeQts39shkfu for <tsvwg@ietfa.amsl.com>; Fri, 19 Apr 2024 03:56:43 -0700 (PDT)
Received: from scc-mailout-kit-01.scc.kit.edu (scc-mailout-kit-01.scc.kit.edu [IPv6:2a00:1398:9:f712::810d:e751]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2B7B0C14F5F4 for <tsvwg@ietf.org>; Fri, 19 Apr 2024 03:56:41 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=kit.edu; s=kit2; h=Content-Transfer-Encoding:Content-Type:In-Reply-To:From:References: To:Subject:MIME-Version:Date:Message-ID:Sender:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=RZIQP2+ar1kdvU6SflGFm6PtiIExmhOw0lHTRZgpXbI=; b=IpseBEN3KT05bspXoeSOPndEvP xkhXwjJPxdfvLe8rfe+HlNrvYoJclXzaGNJ7rv60k8+6rHqnYgQ/rCuN/UAr+ebBNT0irjD2D41Wa cu+GqCC3qKjJaf4W7FSpeIW6+MfHrtOX0PaeeiAyVFbM2ifGr/wS7Ef+xL7Gr96NaMpptzWqnrR/u +uieFUiGT5FrIAWN9h67F+DNKvsj5yGe/tWLuTSS+uLZcu6rkok/y8com3f/zs2yqxUSLYfgDUhtf LLq1t76IFDBm79cA53p+5KGL3/S7vcueWtgm9jA7K6Lfx5sPdEUBriBHImG7TM8BInkUWHP/pGJm3 56xA9Ivw==;
Received: from iramx1.informatik.kit.edu ([2a00:1398:2::10:80]) by scc-mailout-kit-01.scc.kit.edu with esmtps (TLS1.2:ECDHE_SECP256R1__RSA_SHA512__AES_256_GCM:256) (envelope-from <roland.bless@kit.edu>) id 1rxlv3-004xNi-2m; Fri, 19 Apr 2024 12:56:38 +0200
Received: from i72vorta.tm.kit.edu ([141.3.71.26]) by iramx1.informatik.kit.edu with esmtpsa port 25 iface 141.3.10.8 id 1rxlv0-000000004lw-0UwJ; Fri, 19 Apr 2024 12:56:34 +0200
Received: from [IPV6:::1] (ip6-localhost [IPv6:::1]) by i72vorta.tm.kit.edu (Postfix) with ESMTPS id E6818D000CE; Fri, 19 Apr 2024 12:56:33 +0200 (CEST)
Message-ID: <e45b1d63-62e5-446c-abcf-3a22e911de1b@kit.edu>
Date: Fri, 19 Apr 2024 12:56:33 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
To: Vasilenko Eduard <vasilenko.eduard@huawei.com>, "tsvwg@ietf.org" <tsvwg@ietf.org>
References: <a19c38376c7541b89a3d52841141fa0c@huawei.com> <cff2147d-e203-4106-b7d6-65a8573e2c22@kit.edu> <12c7c1300b004691a59ac950e66c3e2b@huawei.com>
From: "Bless, Roland (TM)" <roland.bless@kit.edu>
Content-Language: de-DE, en-US
Autocrypt: addr=roland.bless@kit.edu; keydata= xsFNBFi0OxABEACy2VohJ7VhSu/xPCt4/6qCrw4Pw2nSklWPfAYEk1QgrbiwgvLAP9WEhAIU w45cojBaDxytIGg8eaYeIKSmsXjHGbV/ZTfo8r11LX8yPYR0WHiMWZpl0SHUd/CZIkv2pChO 88vF/2FKN95HDcp24pwONF4VhxJoSFk6c0mDNf8Em/Glt9BcWX2AAvizTmpQDshaPje18WH3 4++KwPZDd/sJ/hHSXiPg1Gdhs/OG/C0CJguOAlqbgSVAe3qKOr1M4K5M+wVpsk373pXRfxd7 ZAmZ05iBTn+LfgVcz+AfaKKcsWri5CdTT+7JDL6QNQpox+b5FXZFSHnEIST+/qzfG7G2LqqY mml6TYY8XbaNyXZP0QKncfSpRx8uTRWReHUa1YbSuOxXYh6bXpcugD25mlC/Lu0g7tz4ijiK iIwq9+P2H1KfAAfYyYZh6nOoE6ET0TjOjUSa+mA8cqjPWX99kEEgf1Xo+P9fx9QLCLWIY7zc mSM+vjQKgdUFpMSCKcYEKOuwlPuOz8bVECafxaEtJJHjCOK8zowe2eC9OM+G+bmtAO3qYcYZ hQ/PV3sztt/PjgdtnFAYPFLc9189rHRxKsWSOb4xPkRw/YQAI9l15OlUEpsyOehxmAmTsesn tSViCz++PCdeXrQc1BCgl8nDytrxW+n5w1aaE8aL3hn8M0tonQARAQABzShSb2xhbmQgQmxl c3MgKFRNKSA8cm9sYW5kLmJsZXNzQGtpdC5lZHU+wsGABBMBCAAqAhsDBQkSzAMABQsJCAcC BhUICQoLAgQWAgMBAh4BAheABQJYtYdHAhkBAAoJEKON2tlkOJXuzWkP+wfjUnDNzRm4r34a AMWepcQziTgqf4I1crcL6VD44767HhyFsjcKH31E5G5gTDxbpsM4pmkghKeLrpPo30YK3qb7 E9ifIkpJTvMu0StSUmcXq0zPyHZ+HxHeMWkosljG3g/4YekCqgWwrB62T7NMYq0ATQe1MGCZ TAPwSPGCUZT3ioq50800FMI8okkGTXS3h2U922em7k8rv7E349uydv19YEcS7tI78pggMdap ASoP3QWB03tzPKwjqQqSevy64uKDEa0UgvAM3PRbJxOYZlX1c3q/CdWwpwgUiAhMtPWvavWW Tcw6Kkk6e0gw4oFlDQ+hZooLv5rlYR3egdV4DPZ1ugL51u0wQCQG9qKIMXslAdmKbRDkEcWG Oi2bWAdYyIHhhQF5LSuaaxC2P2vOYRHnE5yv5KTV3V7piFgPFjKDW+giCRd7VGfod6DY2b2y zwidCMve1Qsm8+NErH6U+hMpMLeCJDMu1OOvXYbFnTkqjeg5sKipUoSdgXsIo4kl+oArZlpK qComSTPhij7rMyeu/1iOwbNCjtiqgb55ZE7Ekd84mr9sbq4Jm/4QGnVI30q4U2vdGSeNbVjo d1nqjf3UNzP2ZC+H9xjsCFuKYbCX6Yy4SSuEcubtdmdBqm13pxua4ZqPSI0DQST2CHC7nxL1 AaRGRYYh5zo2vRg3ipkEzsFNBFi0OxABEAC2CJNp0/Ivkv4KOiXxitsMXZeK9fI0NU2JU1rW 04dMLF63JF8AFiJ6qeSL2mPHoMiL+fG5jlxy050xMdpMKxnhDVdMxwPtMiGxbByfvrXu18/M B7h+E1DHYVRdFFPaL2jiw+Bvn6wTT31MiuG9Wh0WAhoW8jY8IXxKQrUn7QUOKsWhzNlvVpOo SjMiW4WXksUA0EQVbmlskS/MnFOgCr8q/FqwC81KPy+VLHPB9K/B65uQdpaw78fjAgQVQqpx H7gUF1EYpdZWyojN+V8HtLJx+9yWAZjSFO593OF3/r0nDHEycuOjhefCrqr0DDgTYUNthOdU KO2CzT7MtweRtAf0n27zbwoYvkTviIbR+1lV1vNkxaUtZ6e1rtOxvonRM1O3ddFIzRp/Qufu HfPe0YqhEsrBIGW1aE/pZW8khNQlB6qt20snL9cFDrnB6+8kDG3e//OjK1ICQj9Y/yyrJVaX KfPbdHhLpsgh8TMDPoH+XXQlDJljMD0++/o7ckO3Sfa8Zsyh1WabyKQDYXDmDgi9lCoaQ7Lf uLUpoMvJV+EWo0jE4RW/wBGQbLJp5usy5i0fhBKuDwsKdLG3qOCf4depIcNuja6ZmZHRT+3R FFjvZ/dAhrCWpRTxZANlWlLZz6htToJulAZQJD6lcpVr7EVgDX/y4cNwKF79egWXPDPOvQAR AQABwsFlBBgBCAAPBQJYtDsQAhsMBQkSzAMAAAoJEKON2tlkOJXukMoP/jNeiglj8fenH2We 7SJuyBp8+5L3n8eNwfwY5C5G+etD0E6/lkt/Jj9UddTazxeB154rVFXRzmcN3+hGCOZgGAyV 1N7d8xM6dBqRtHmRMPu5fUxfSqrM9pmqAw2gmzAe0eztVvaM+x5x5xID2WZOiOq8dx9KOKrp Zorekjs3GEA3V1wlZ7Nksx/o8KZ04hLeKcR1r06zEDLN/yA+Fz8IPa0KqpuhrL010bQDgAhe 9o5TA0/cMJpxpLqHhX2As+5cQAhKDDsWJu3oBzZRkN7Hh/HTpWurmTQRRniLGSeiL0zdtilX fowyxGXH6QWi3MZYmpOq+etr7o4EGGbm2inxpVbM+NYmaJs+MAi/z5bsO/rABwdM5ysm8hwb CGt+1oEMORyMcUk/uRjclgTZM1NhGoXm1Un67+Rehu04i7DA6b8dd1H8AFgZSO2H4IKi+5yA Ldmo+ftCJS83Nf6Wi6hJnKG9aWQjKL+qmZqBEct/D2uRJGWAERU5+D0RwNV/i9lQFCYNjG9X Tew0BPYYnBtHFlz9rJTqGhDu4ubulSkbxAK3TIk8XzKdMvef3tV/7mJCmcaVbJ2YoNUtkdKJ goOigJTMBXMRu4Ibyq1Ei+d90lxhojKKlf9yguzpxk5KYFGUizp0dtvdNuXRBtYrwzykS6vB zTlLqHZ0pvGjNfTSvuuN
Organization: Institute of Telematics, Karlsruhe Institute of Technology (KIT)
In-Reply-To: <12c7c1300b004691a59ac950e66c3e2b@huawei.com>
Content-Type: text/plain; charset="UTF-8"; format="flowed"
Content-Transfer-Encoding: 7bit
X-ATIS-AV: ClamAV (iramx1.informatik.kit.edu)
X-ATIS-Checksum: v3zoCAcc32ckk
X-ATIS-Timestamp: iramx1.informatik.kit.edu esmtpsa 1713524194.170727079
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/XKNHW_cSjsg4KDt6thx6z010p5I>
Subject: Re: [tsvwg] "Pacing" requirement is lost in L4S
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 19 Apr 2024 10:56:48 -0000

Hi Vasilenko,

On 19.04.24 at 12:01 Vasilenko Eduard wrote:
> 1.
> CC may measure RTT, understand buffer growth, and increase the interval between packets.

Delay-based CC is a different aspect and orthogonal to pacing, but
typically pacing is also beneficial for them.

> Reaction by W and T looks the same effective against loaded buffer.
> Actially W is easy to re-calculate to T. I do not see a problem.

As I said, the difference is that a sheer existence of a congestion 
window has a self-stabilizing effect on buffer occupation, whereas
two senders that simply send with perfect average rate shares
towards a bottleneck (but slightly bursty sending rate, e.g.,
packet sending times from an exponential distribution) may
cause unlimited queue growth.

> 2.
> It looks like I use the wrong terminology because "pacing" for me is exactly the situation when information is sent on some timer (that may change slowly).
> But you use "pacing" in combination with "BBR rate-based sending" which looks to me like "pacing for pacing".
> It looks like "pacing" is something different for you.
> Probably it is my fault. But I have not understood.

Personally, I actually like to distinguish between rate-based sending 
and pacing:
BBR calculates a target sending rate and uses paced sending to avoid
burstiness even on smaller time scales. You could effectively use
the same target sending rate over a larger interval, e.g., RTT_min
and send it in a much more bursty manner.
That would still be rate-based, but without pacing.

Regards,
  Roland

> 
> Eduard
> -----Original Message-----
> From: Bless, Roland (TM) <roland.bless@kit.edu>
> Sent: Friday, April 19, 2024 12:17
> To: Vasilenko Eduard <vasilenko.eduard@huawei.com>; tsvwg@ietf.org
> Subject: Re: [tsvwg] "Pacing" requirement is lost in L4S
> 
> Hi Vasilenko,
> 
> [not answering w.r.t. the Scalable requirement, but like to discuss a different point]
> 
> I strongly disagree with your conclusion that we should forget about Window-based CC. Window-based CC has the big advantage of being self-stabilizing and thus limiting the amount of queuing delay.
> Rate-based CCs are harder to control in this respect: the amount of queued data at the bottleneck buffer may grow over time in case you overestimated the available bandwidth.
> 
> However, you are right that window-based CC may cause micro-bursts due to various causes (application rate limits or distorted ACK clocking). While I agree that the ACK clock isn't that reliable anymore these days (see also
> https://dl.acm.org/doi/10.1145/3371934.3371956 "Deprecating the TCP macroscopic model"), I disagree with abandoning window-based CCs due to that.
> The counter-measure against micro-bursts and underutilization that may be caused by distorted ACK clocks is *pacing*!
> This avoids micro-bursts and lets the sender keep sending for a limited time period even if ACKs are not on time.
> 
> So my conclusion is that you should actually combine both:
> *window-based CC + pacing* or similar to BBR rate-based sending+pacing + control of the inflight data amount (which is similar to window-based CC).
> 
> Regards,
>    Roland
> 
> On 19.04.24 at 10:39 Vasilenko Eduard wrote:
>> I see L4S as the "Congestion Control Next Generation from IETF" (that is actually in competition with "Congestion Control Next Generation from Google").
>> Then I see the important requirement that is missing in L4S.
>>
>> The primary requirement for CC is that it "should not grow the buffer on the bottleneck link".
>> It is very disputable: is "the Scalable" requirement about it or not? Let's pretend that it is about it.
>>
>> Then the next critical requirement is "pacing" which is missed in L4S completely.
>> Kudos to Google because I understood its importance after one local (but big) company tested and investigated BBRv1 (then implemented it).
>> After tests, they concluded that pacing is even more important than
>> low latency. (I doubt, probably latency is more important) Imagine that the server would increase the window sharply. The Server may have a 100GE interface. It could generate 10us of traffic as a burst (or even more).
>> Intermediate links could be 100GE or even bigger (highly probable), and the burst would travel as it is (without any spreading).
>> Then this burst could arrive at 10Mbps subscriber (good performance for shared public WiFi). 0.01ms burst would become 100ms burst.
>> It would create many negative consequences for the bottleneck link:
>> - tail drop if buffers are not enough
>> - guaranteed huge latency
>> Hence, we should completely forget about W (window) while discussing CC, we have to use T (time between packets).
>> CC next generation "should avoid bursts regulating inter-packet time, not the information permitted in transit".
>