Re: [Webtransport] Flow Control for high-throughput Streams

Martin Thomson <mt@lowentropy.net> Tue, 16 April 2024 23:16 UTC

Return-Path: <mt@lowentropy.net>
X-Original-To: webtransport@ietfa.amsl.com
Delivered-To: webtransport@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D0312C14F69A for <webtransport@ietfa.amsl.com>; Tue, 16 Apr 2024 16:16:45 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.796
X-Spam-Level:
X-Spam-Status: No, score=-2.796 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=lowentropy.net header.b="wOVGJRPK"; dkim=pass (2048-bit key) header.d=messagingengine.com header.b="fLrSmuGr"
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1RJKFXE3rBVB for <webtransport@ietfa.amsl.com>; Tue, 16 Apr 2024 16:16:41 -0700 (PDT)
Received: from wfout1-smtp.messagingengine.com (wfout1-smtp.messagingengine.com [64.147.123.144]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 15630C14CE52 for <webtransport@ietf.org>; Tue, 16 Apr 2024 16:16:40 -0700 (PDT)
Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailfout.west.internal (Postfix) with ESMTP id EEAE21C000D3 for <webtransport@ietf.org>; Tue, 16 Apr 2024 19:16:37 -0400 (EDT)
Received: from imap41 ([10.202.2.91]) by compute6.internal (MEProxy); Tue, 16 Apr 2024 19:16:38 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lowentropy.net; h=cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm3; t=1713309397; x=1713395797; bh=ivx9nmGLrAXosiDFHULmyfUXfMcl1pFyDy91IBs0GJ8=; b= wOVGJRPK2OEaJI2n3TTlFZD9rKlDSghjeBv9nhrWBOjeoBMROC9wB0LlWmSTrfQj VO3iMu9GFRRnEs8icNZxrAqUhw13i2YfkFV0wEm7YKFfUcVelvDfjrECBSxb1RzQ RfKkNr8yB31Z0HppPX3INOFtNfiuc0A7rYPUINFohfT63q9DSpRGAC7dcYPRsOgW QGunCCAWIX5nukLkj4Hd8F4UYdc7aw3p/1auvRbyC/TF9n/mmo3eNicEVIvI5zm1 r++rsAfDtOUlq6wBEDG10uz8LlE4MK3Vi3ssKo1oazhwzRFK6yCA12Vj2I/Ld58L 8k06zysGLhAvotDahzhpQA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1713309397; x= 1713395797; bh=ivx9nmGLrAXosiDFHULmyfUXfMcl1pFyDy91IBs0GJ8=; b=f LrSmuGrpqACGr/+Sw6svqjFMA91hLErvWgMHlgZTOS2G98LpJ2faGrC6PFyVy48c tETyGjJ+C6e1NQkvUDDofZFXld/YB0IS1CNoDVe2t5fJbw81GZbt4r4ei/M0vH2W 6rKoeLeiUmDqXNgY6jOCbC4IGrU+0Xl0hTK3HSzQoA1ovp1GByeHoZ7mdU3hCh6z yBxJYPOb7mMtAXBRlKAiXYC8t4Q1p8JN1yxMQFfNFXge4/75lPqsCxZA8QHs6FnU gXVEkHQaKu4Ekf4qLIsvbDIS8t9pmKtPL+jpUNrhbB30+kkh3iABFz0PcCNlh3B9 IsaOAL/p6JcmZSVkgNtRw==
X-ME-Sender: <xms:1QYfZh_7ZgLRRpNrHF4ej00E2XiW4M3DLbti4gbqTf1O2c4Og90qug> <xme:1QYfZlviADbT0TvAy56jyZSPTOwF44NCIY-RHUj4ACmzYKYAw38JlC_RXRJItkNWU wWjYSU3iM_1bGJPaAs>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudejjedgudejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtgfesth hqredtreerjeenucfhrhhomhepfdforghrthhinhcuvfhhohhmshhonhdfuceomhhtsehl ohifvghnthhrohhphidrnhgvtheqnecuggftrfgrthhtvghrnhepffetgfevvdfhtdffhe ejudefhffhveehudefhffgleelteeifeegfefggfelhefgnecuffhomhgrihhnpehivght fhdrohhrghenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhroh hmpehmtheslhhofigvnhhtrhhophihrdhnvght
X-ME-Proxy: <xmx:1QYfZvB6AledfCLxyxq32llQu-IXpH2176WzdFcJ6o7Bn0SqrafuHA> <xmx:1QYfZldkpQDQ-aPtyIO1xsDsxMwmhlDccR4NfZ3XnSK6usR5g4Rlrg> <xmx:1QYfZmOnCQ2vmExs9cYVWuIVkVcjhwRV9U7RTVeBqRgbGGhLxr197Q> <xmx:1QYfZnnTqeMYToYYrH4otHVmcZ4KBBROhx6kCHGqFXKFKfqKfBfOQw> <xmx:1QYfZgUGqr6r8S_vjS64iQ3IeYdmB3gFH49qveklbc10aaQhmd3MJ5It>
Feedback-ID: ic129442d:Fastmail
Received: by mailuser.nyi.internal (Postfix, from userid 501) id 418692340080; Tue, 16 Apr 2024 19:16:37 -0400 (EDT)
X-Mailer: MessagingEngine.com Webmail Interface
User-Agent: Cyrus-JMAP/3.11.0-alpha0-379-gabd37849b7-fm-20240408.001-gabd37849
MIME-Version: 1.0
Message-Id: <e21e5b8b-6b26-4bda-bd03-764c41836873@betaapp.fastmail.com>
In-Reply-To: <CAOYVs2qshgTP6SaB0KtwvMixQ_pHKYUajfHzDqf1eG04-hAk7w@mail.gmail.com>
References: <CAOYVs2qshgTP6SaB0KtwvMixQ_pHKYUajfHzDqf1eG04-hAk7w@mail.gmail.com>
Date: Wed, 17 Apr 2024 09:16:15 +1000
From: Martin Thomson <mt@lowentropy.net>
To: webtransport@ietf.org
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://mailarchive.ietf.org/arch/msg/webtransport/JtfqsBVoAd4zwbWzNA2kuJwvdjI>
Subject: Re: [Webtransport] Flow Control for high-throughput Streams
X-BeenThere: webtransport@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: WebTransport WG <webtransport.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/webtransport>, <mailto:webtransport-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/webtransport/>
List-Post: <mailto:webtransport@ietf.org>
List-Help: <mailto:webtransport-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/webtransport>, <mailto:webtransport-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 16 Apr 2024 23:16:45 -0000

I want to acknowledge that this is a real problem.  At least in theory.

The reason that I'm not in favour of a fix here is largely down to an engineering trade-off.

Though a large BDP flow can stall, there are plenty of strategies available.  Whether that be a larger commitment, more aggressive transmission of the stream, or simply accepting the risk of stall or lower throughput.

That needs to be balanced against the cost to us of building a better option.  The solutions on the table are pretty awkward and would require fairly extensive analysis and discussion to make right.

Two things push me toward doing nothing about this (other than writing some text about the risk):

1. I'd like to ship WebTransport.
2. We can fix this in a nicer way later.

So I'm not going to ask for more evidence that this is a real problem.  That would delay shipping further.  It has already taken too long.  I'd prefer to ship something with this modest flaw than continue to delay that ship date.

Cheers,
Martin

On Tue, Mar 19, 2024, at 20:47, Marten Seemann wrote:
> After the discussion we had in yesterday’s session, I’ve spent some 
> more time thinking about how and in which situations the HoL blocking 
> introduced by putting flow control capsules on a control stream might 
> hurt performance. Unfortunately, this is not an edge case, but a fairly 
> common scenario.
>
> Consider an application transferring a large amount of data on a single 
> stream. In QUIC, the receiver would select or auto-tune the stream 
> receive window to slightly exceed the connection's BDP, assuming it 
> sends MAX_STREAM_DATA frames multiple times per RTT. This ensures that 
> flow-control doesn’t block the transfer. If MAX_STREAM_DATA is sent 
> less frequently, a window slightly larger than the BDP is needed to be 
> able to fully utilize the available bandwidth.
>
> For connections not terminating at a CDN's edge, BDPs can be quite 
> large due to the longer RTTs. For instance, a 200ms RTT connection with 
> a bandwidth of 1 Gbps has a BDP of 25 MB.
>
> When packet loss occurs on the WebTransport control stream (in upstream 
> direction), no capsules can be read until this packet loss is repaired, 
> which takes at least (since loss detection might take some time as 
> well) 1 RTT. This applies even if the control stream is given the 
> highest priority when sending. For a BDP-sized flow control window, 
> this means that the high-throughput stream (in downstream direction) is 
> now blocked on flow control until the retransmission arrives (i.e. for 
> at least 1 RTT).
>
> The solution suggested in yesterday's session, "just use higher flow 
> control limits," would mean increasing the stream flow control window 
> to (at least!) 50 MB from 25 MB for our example. This approach seems 
> unappealing from a resource allocation standpoint.
> -- 
> Webtransport mailing list
> Webtransport@ietf.org
> https://www.ietf.org/mailman/listinfo/webtransport