Re: [Webtransport] Flow Control for high-throughput Streams

Bernard Aboba <bernard.aboba@gmail.com> Wed, 17 April 2024 01:00 UTC

Return-Path: <bernard.aboba@gmail.com>
X-Original-To: webtransport@ietfa.amsl.com
Delivered-To: webtransport@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C5DD0C151062 for <webtransport@ietfa.amsl.com>; Tue, 16 Apr 2024 18:00:44 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.094
X-Spam-Level:
X-Spam-Status: No, score=-2.094 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dfoIfI5T2dU4 for <webtransport@ietfa.amsl.com>; Tue, 16 Apr 2024 18:00:40 -0700 (PDT)
Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E3E7FC14CE5D for <webtransport@ietf.org>; Tue, 16 Apr 2024 18:00:40 -0700 (PDT)
Received: by mail-ej1-x62e.google.com with SMTP id a640c23a62f3a-a524ecaf215so454231266b.2 for <webtransport@ietf.org>; Tue, 16 Apr 2024 18:00:40 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1713315639; x=1713920439; darn=ietf.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=gcO9I9Be3XKWaUsBCm/i9tMiudCKjbTlJC/NnNe4gVg=; b=mCLM5a/T1vy3Bi8Yyrjx0+PJAUDf6iqAnIG/gy10+lTHlvciDj/PfDGzo5GFXdaQP2 pj+iN73TFHDF8PpiUVVFFsYkzCyy9dGNzfb5ByBIDas19o+ycSPvt+BINU/Sbhe9mh8h xzUSSG0ouStzRQQTwqCuflLuk0LgSkfzojD+G7a6c8tgIYgmAhsDi8elJB4dWx4C2UCL WdviSZD74yYYG+TukQvRplSQMqbE2DWzhnIi3KOYJsg6ZKxP+E9zOu9TOQ1DBcChmbLE o3112rVXUC2dgsHv6fZJtYuy/D0BVBv0/qTSKzjUtS7dZha/eqGdcq5qLF5LEYnzzZiW SpUA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1713315639; x=1713920439; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=gcO9I9Be3XKWaUsBCm/i9tMiudCKjbTlJC/NnNe4gVg=; b=FXmMsm+Nd9QxU+yd4vtsjW+aKi5bgQsbaiO2Pst06NBM54SCPZFHTi9MOs8MczETZP jAIozSiOGpEmQ/LXEJ15Ua7arfyt0IRIFCNM8G8N8/XSHWp7McPJSP/L451fl7T/zB59 v5OiCmVbgyMFeKodi0sTGBqBxodB8Ii7fGP3/h0VUGnXACvk53FvcdrN+3VoDs6KuQL1 RfcRFhAJNikSOd3daUsoXPPD0c/Wam8u17TaNe1lk2rkSMDuU0GZ3/qWpOPsGNLoJYQa HRHWOxNklOVa3/XfR7EmfLSXFBYYD6GkFA6xRiH1/Fm+Q1fxY58yDzzshtcGIu306moR 25mw==
X-Forwarded-Encrypted: i=1; AJvYcCVM2xoZlcOC+1WET9bWtTXdGdydyIulH6zbS/U2HbzpWxzx0Uh+mvDzR1jV9QzeQskLRwN8016qr+yaourQTy9+x9ly/js=
X-Gm-Message-State: AOJu0Yz4BhHmXYyAKWtwlClYhd9EbfnjusmWeqo4mB91BYNqCJoEGKTN GW/Y1DfxqTxR7NX9Ak0dLMG+ogdQ47Y/dcIOqAuF9fyrnUuxonJHBrPcag/Dq7MQpvV09iC9yeU O54ceElWYe7ci0y+N5fVTbC73e7A=
X-Google-Smtp-Source: AGHT+IG5/cPgf8Z/62m52Pw99iwJpXB97ULSIpPKE2MleJLCsrDicGGSIA9jI/TQP9EXrEAmqZntKVWHzPWr/Q7jHV0=
X-Received: by 2002:a17:906:fd8e:b0:a52:54be:de23 with SMTP id xa14-20020a170906fd8e00b00a5254bede23mr6681389ejb.15.1713315638753; Tue, 16 Apr 2024 18:00:38 -0700 (PDT)
MIME-Version: 1.0
References: <CAOYVs2qshgTP6SaB0KtwvMixQ_pHKYUajfHzDqf1eG04-hAk7w@mail.gmail.com> <CAAZdMacsXzueOFvnTeJ3zhWmdPC5f_aoPN59QMutmWsW+vLTaw@mail.gmail.com>
In-Reply-To: <CAAZdMacsXzueOFvnTeJ3zhWmdPC5f_aoPN59QMutmWsW+vLTaw@mail.gmail.com>
From: Bernard Aboba <bernard.aboba@gmail.com>
Date: Tue, 16 Apr 2024 18:00:25 -0700
Message-ID: <CAOW+2dtQLY7ngqN5-A=GR3Txvu1HsDSZstqeQqxisosXd-4gjQ@mail.gmail.com>
To: Victor Vasiliev <vasilvv=40google.com@dmarc.ietf.org>
Cc: Marten Seemann <martenseemann@gmail.com>, WebTransport <webtransport@ietf.org>
Content-Type: multipart/alternative; boundary="0000000000000945fc06164062c7"
Archived-At: <https://mailarchive.ietf.org/arch/msg/webtransport/x2hJnCl9G_e26vP2-bs7daCGe00>
Subject: Re: [Webtransport] Flow Control for high-throughput Streams
X-BeenThere: webtransport@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: WebTransport WG <webtransport.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/webtransport>, <mailto:webtransport-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/webtransport/>
List-Post: <mailto:webtransport@ietf.org>
List-Help: <mailto:webtransport-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/webtransport>, <mailto:webtransport-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 17 Apr 2024 01:00:44 -0000

Victor said:

"the key property in this scenario is that it's very sensitive to loss of
window updates, "

[BA] I think that is an important point, WebTransport API provides close
coupling between application consumption of incoming data and window
updates.

This was an issue with the WebSockets and WebRTC data channel APIs, where
incoming data is considered "received" when it is removed from the receive
buffer and placed onto the event queue.  Unfortunately, data moved to the
event queue has only been "received" by the browser, not by the
application. If the application's omessage event handler cannot keep up,
the event queue grows but the receive window does not constrict.  For
low-end devices (such as inexpensive mobile devices or game consoles), this
can result in event queue bloat, stuttering audio/video and high observed
e2e latency.

This shouldn't happen with WebTransport, as long as the feedback loop
remains operational.  Throughput may be constricted, but as the application
consumes data and window updates arrive at the sender, data will continue
to be sent.  It doesn't seem like this is susceptible to "deadlock", only
degraded performance.  Or am I missing something?



On Tue, Mar 26, 2024 at 4:18 AM Victor Vasiliev <vasilvv=
40google.com@dmarc.ietf.org> wrote:

> I think the key property in this scenario is that it's very sensitive to
> loss of window updates, not necessarily the HOLB on the control stream.
>
> One approach here is that whenever you send a new capsule on the control
> stream, you could retransmit all of the previous ones that aren't
> acknowledged; since those capsules are small, the resulting stream frame
> will normally fit into a single packet.  Now that I think about it, this
> approach can be applied to any latency-sensitive stream with a lot of small
> writes: you could have a flag that says "if this stream has less than an
> MTU worth of unacked data, send all of it every time".
>
>   -- Victor.
>
> On Tue, Mar 19, 2024 at 5:47 AM Marten Seemann <martenseemann@gmail.com>
> wrote:
>
>> After the discussion we had in yesterday’s session, I’ve spent some more
>> time thinking about how and in which situations the HoL blocking introduced
>> by putting flow control capsules on a control stream might hurt
>> performance. Unfortunately, this is not an edge case, but a fairly common
>> scenario.
>>
>> Consider an application transferring a large amount of data on a single
>> stream. In QUIC, the receiver would select or auto-tune the stream receive
>> window to slightly exceed the connection's BDP, assuming it sends
>> MAX_STREAM_DATA frames multiple times per RTT. This ensures that
>> flow-control doesn’t block the transfer. If MAX_STREAM_DATA is sent less
>> frequently, a window slightly larger than the BDP is needed to be able to
>> fully utilize the available bandwidth.
>>
>> For connections not terminating at a CDN's edge, BDPs can be quite large
>> due to the longer RTTs. For instance, a 200ms RTT connection with a
>> bandwidth of 1 Gbps has a BDP of 25 MB.
>>
>> When packet loss occurs on the WebTransport control stream (in upstream
>> direction), no capsules can be read until this packet loss is repaired,
>> which takes at least (since loss detection might take some time as well) 1
>> RTT. This applies even if the control stream is given the highest priority
>> when sending. For a BDP-sized flow control window, this means that the
>> high-throughput stream (in downstream direction) is now blocked on flow
>> control until the retransmission arrives (i.e. for at least 1 RTT).
>>
>> The solution suggested in yesterday's session, "just use higher flow
>> control limits," would mean increasing the stream flow control window to
>> (at least!) 50 MB from 25 MB for our example. This approach seems
>> unappealing from a resource allocation standpoint.
>> --
>> Webtransport mailing list
>> Webtransport@ietf.org
>> https://www.ietf.org/mailman/listinfo/webtransport
>>
> --
> Webtransport mailing list
> Webtransport@ietf.org
> https://www.ietf.org/mailman/listinfo/webtransport
>