Re: [Webtransport] Flow Control for high-throughput Streams

Marten Seemann <martenseemann@gmail.com> Wed, 27 March 2024 04:55 UTC

Return-Path: <martenseemann@gmail.com>
X-Original-To: webtransport@ietfa.amsl.com
Delivered-To: webtransport@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9B639C15153C for <webtransport@ietfa.amsl.com>; Tue, 26 Mar 2024 21:55:12 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -7.105
X-Spam-Level:
X-Spam-Status: No, score=-7.105 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UApPNg_dh__g for <webtransport@ietfa.amsl.com>; Tue, 26 Mar 2024 21:55:11 -0700 (PDT)
Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [IPv6:2a00:1450:4864:20::631]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9F7D0C151527 for <webtransport@ietf.org>; Tue, 26 Mar 2024 21:55:11 -0700 (PDT)
Received: by mail-ej1-x631.google.com with SMTP id a640c23a62f3a-a44f2d894b7so766766066b.1 for <webtransport@ietf.org>; Tue, 26 Mar 2024 21:55:11 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1711515309; x=1712120109; darn=ietf.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=LoUDQwcaq+bcucI32nlQJ+e08p23Y2TO4Vrm0dqOeW8=; b=muiXLsfJRz+X9rqj2gVqLZL3t44As4xTAYScCWBiM2wOXdudTYY4Qz2EItqtd8o3ki KRcY5Lz8LUlj6XNj+ldFnVDu41TQNCweFAPcc5ZZoaUMM+z0SLrXY6kRoDFNQmuwUgyp Z1fSrcpzLm8YLvmr/6063faP4334hUIOf6JPEs7tlRjuw4nyOf6p3pL31ib1WcXPwJPe 3bWntEYoORNZwliVnnXNvvs1I5z5H3zYIskK3rAMB+UQ4dfSVbd3koDhifesaRQIzZjo dIugGvL1AkCDfqjxg099lDwyuJgeY95/ygOMVUu1eVfYh6LKiw/4FTQeNuuQHXoBXMu2 FIMA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711515309; x=1712120109; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=LoUDQwcaq+bcucI32nlQJ+e08p23Y2TO4Vrm0dqOeW8=; b=iVD2mDziECvFgt9EQGGCQYqyHlJ6tf6hcs0TDp6s4O3r5Zr7i/v5ElyTBCnAjLB/7z iwnttW5r/dUu38mgBWBZyHSIrl2iuiTjMFcvozVbA6cWLrZlEoQTiYe0xmJQ7H/ZhbZn gcGY/ie9K81lx7LHL+wpI5QHxEbBoEenz0gAU9w5cFP9S39TkTqYBSzZPNJ0MzPRfP6Z 66Jf1IDABOcUpvxidiK/G9By+Eh8Ej8z2hzzv7kzpWAThyH0wMF5k8nRCNqLwMzCGpXo riYZgrGWhdzS8YvETezqOZjngomXzmQfU9nV72gFnPJ5EkgBpKPwY8nioAsm6rFNa7n5 t3SQ==
X-Gm-Message-State: AOJu0YwZfQzPaqWtY9PV5jR0eOpb69gyiunnHuSbDey2UNSkX/pjsILR zQFah9LoKhWoPqTBQS1AfKcFmRcFVSC8eTHMUfYiGfjdo8AZab85+7Ujq+3HxOKt46M+3gjhCeD f0DzuW7wFs9yLEuIKQlwEQDOis+455cP9C6jR5cSv
X-Google-Smtp-Source: AGHT+IF6lc7fTdyr9jKgioAKK16YbIYWGklKxlkgjKF+0HGeYVdnBotRYsh/ACLuW89zViqLTYVJGHFD0oKXk+gKz5o=
X-Received: by 2002:a17:907:7f22:b0:a48:56b2:d988 with SMTP id qf34-20020a1709077f2200b00a4856b2d988mr4175773ejc.59.1711515308806; Tue, 26 Mar 2024 21:55:08 -0700 (PDT)
MIME-Version: 1.0
References: <CAOYVs2qshgTP6SaB0KtwvMixQ_pHKYUajfHzDqf1eG04-hAk7w@mail.gmail.com> <CAAZdMacsXzueOFvnTeJ3zhWmdPC5f_aoPN59QMutmWsW+vLTaw@mail.gmail.com>
In-Reply-To: <CAAZdMacsXzueOFvnTeJ3zhWmdPC5f_aoPN59QMutmWsW+vLTaw@mail.gmail.com>
From: Marten Seemann <martenseemann@gmail.com>
Date: Wed, 27 Mar 2024 17:54:56 +1300
Message-ID: <CAOYVs2rkDyg_bBsF-ejZQapELef_KhVvEHTsw4hPR=u7Cb-U_A@mail.gmail.com>
To: Victor Vasiliev <vasilvv@google.com>
Cc: WebTransport <webtransport@ietf.org>
Content-Type: multipart/alternative; boundary="000000000000025c5c06149d36fa"
Archived-At: <https://mailarchive.ietf.org/arch/msg/webtransport/ijE1GXQSXl-HMYE_-UstwUpOLh0>
Subject: Re: [Webtransport] Flow Control for high-throughput Streams
X-BeenThere: webtransport@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: WebTransport WG <webtransport.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/webtransport>, <mailto:webtransport-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/webtransport/>
List-Post: <mailto:webtransport@ietf.org>
List-Help: <mailto:webtransport-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/webtransport>, <mailto:webtransport-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 27 Mar 2024 04:55:12 -0000

> I think the key property in this scenario is that it's very sensitive to
loss of window updates, not necessarily the HOLB on the control stream.

Isn't that a general property of flow control?
In my QUIC stack, I solve this by sending ~5 window updates per RTT. Doing
it frequently is basically for free, as long as you bundle it with ACKs
that you would send anyway.
On WebTransport this doesn't work, since the 5 frames are not delivered
independently from each other.

On Wed, 27 Mar 2024 at 00:17, Victor Vasiliev <vasilvv@google.com> wrote:

> I think the key property in this scenario is that it's very sensitive to
> loss of window updates, not necessarily the HOLB on the control stream.
>
> One approach here is that whenever you send a new capsule on the control
> stream, you could retransmit all of the previous ones that aren't
> acknowledged; since those capsules are small, the resulting stream frame
> will normally fit into a single packet.  Now that I think about it, this
> approach can be applied to any latency-sensitive stream with a lot of small
> writes: you could have a flag that says "if this stream has less than an
> MTU worth of unacked data, send all of it every time".
>
>   -- Victor.
>
> On Tue, Mar 19, 2024 at 5:47 AM Marten Seemann <martenseemann@gmail.com>
> wrote:
>
>> After the discussion we had in yesterday’s session, I’ve spent some more
>> time thinking about how and in which situations the HoL blocking introduced
>> by putting flow control capsules on a control stream might hurt
>> performance. Unfortunately, this is not an edge case, but a fairly common
>> scenario.
>>
>> Consider an application transferring a large amount of data on a single
>> stream. In QUIC, the receiver would select or auto-tune the stream receive
>> window to slightly exceed the connection's BDP, assuming it sends
>> MAX_STREAM_DATA frames multiple times per RTT. This ensures that
>> flow-control doesn’t block the transfer. If MAX_STREAM_DATA is sent less
>> frequently, a window slightly larger than the BDP is needed to be able to
>> fully utilize the available bandwidth.
>>
>> For connections not terminating at a CDN's edge, BDPs can be quite large
>> due to the longer RTTs. For instance, a 200ms RTT connection with a
>> bandwidth of 1 Gbps has a BDP of 25 MB.
>>
>> When packet loss occurs on the WebTransport control stream (in upstream
>> direction), no capsules can be read until this packet loss is repaired,
>> which takes at least (since loss detection might take some time as well) 1
>> RTT. This applies even if the control stream is given the highest priority
>> when sending. For a BDP-sized flow control window, this means that the
>> high-throughput stream (in downstream direction) is now blocked on flow
>> control until the retransmission arrives (i.e. for at least 1 RTT).
>>
>> The solution suggested in yesterday's session, "just use higher flow
>> control limits," would mean increasing the stream flow control window to
>> (at least!) 50 MB from 25 MB for our example. This approach seems
>> unappealing from a resource allocation standpoint.
>> --
>> Webtransport mailing list
>> Webtransport@ietf.org
>> https://www.ietf.org/mailman/listinfo/webtransport
>>
>