Re: Priority implementation complexity (was: Re: Extensible Priorities and Reprioritization)
Lucas Pardue <lucaspardue.24.7@gmail.com> Fri, 19 June 2020 16:26 UTC
Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AE9063A09BD for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Fri, 19 Jun 2020 09:26:17 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.748
X-Spam-Level:
X-Spam-Status: No, score=-2.748 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, MAILING_LIST_MULTI=-1, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id r73uDVrpZgBm for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Fri, 19 Jun 2020 09:26:15 -0700 (PDT)
Received: from lyra.w3.org (lyra.w3.org [128.30.52.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 84E543A09C3 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Fri, 19 Jun 2020 09:26:15 -0700 (PDT)
Received: from lists by lyra.w3.org with local (Exim 4.92) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1jmJnD-0000iT-86 for ietf-http-wg-dist@listhub.w3.org; Fri, 19 Jun 2020 16:23:03 +0000
Resent-Date: Fri, 19 Jun 2020 16:23:03 +0000
Resent-Message-Id: <E1jmJnD-0000iT-86@lyra.w3.org>
Received: from titan.w3.org ([128.30.52.76]) by lyra.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <lucaspardue.24.7@gmail.com>) id 1jmJn6-0000hZ-II for ietf-http-wg@listhub.w3.org; Fri, 19 Jun 2020 16:22:56 +0000
Received: from mail-wm1-x32a.google.com ([2a00:1450:4864:20::32a]) by titan.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from <lucaspardue.24.7@gmail.com>) id 1jmJn4-00008d-6H for ietf-http-wg@w3.org; Fri, 19 Jun 2020 16:22:56 +0000
Received: by mail-wm1-x32a.google.com with SMTP id g21so5182458wmg.0 for <ietf-http-wg@w3.org>; Fri, 19 Jun 2020 09:22:53 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=9v2QZXp91sgHw1J+VGHJDzr56qQBRPwLC48NQEQHmQc=; b=P5cpl/ez7GaoM/wJVPEcgZLI8s9uZaHQRQidJj+yLTSbB+kImhqGKMEgsWWDHOOV3k YMmnux5RzKl4QOrgCDGX497iS2GWQqHYd3KGtTsPCGnRdX7RiR2K4ncccK64ZYPBl/Ox LLn6mf8WrYwfIw47kHQAClK6crhQqhKVB5gqW/BEXUNnz0nXO6KoWmqcvelrQ9wKVGX6 JaWp5B+M6HfCLGhcfhC8XLFRSXcQGBMp0ei/Bo5kOOkjVK2wuTUTjnJziEzrNIF5MLjb d8fRTB+9YZ7Mv9MBpI5zwUvDa+dRt7DQaP71+SBNmAndgGXNsFO/0dlddMuNRtmZUmOa vwKQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=9v2QZXp91sgHw1J+VGHJDzr56qQBRPwLC48NQEQHmQc=; b=pcoutR3ugFN6LlVTPq9xeaDtRZj2AMBEds0QrDim8WQ973uk83nX3O98DzJrMaPMZP g91fGn1suj5/SNwJ99P/saBmAY73tpeaBEGexMf7BeP2OgD6FuG79tfnzcXMBOO2SLeP IQ7mm7GZto7rXH5aM4wUIk62htPwAi7BwF7HXi87sOhtiWiPThG6NU0ms+qdZ1AYKO0R PvW2PEIZ+OnsmTHkZdFF8P7Q2KMGFFVOax5P0dQ1/rT3sdb9lKWc9u6id9YapmsiFFCJ afuojTXByHdq5BOS3yONY24h6XtzKBa++e9fj+ppgJYPbExq8BjSadptLIJnfxP/uNVR dgBg==
X-Gm-Message-State: AOAM531Yuk36zcnW9f5fJpc0kF7lLG5CndV+cdRdbGVlJZPw2KUcxGb6 Gqp8shVdiijukcaUgfPgbyJLKX59kG0vRqbU0yHRGGOf
X-Google-Smtp-Source: ABdhPJywx0RH7ytXBbjw0OIrzviVSKUkXkcPlPIsJWcaxkjKS3aZ9A4WAH8DruE2EHhozkMzBej6Ikmj0bwMqbJ5UvU=
X-Received: by 2002:a1c:6788:: with SMTP id b130mr3903497wmc.100.1592583762524; Fri, 19 Jun 2020 09:22:42 -0700 (PDT)
MIME-Version: 1.0
References: <CALGR9obRjBSADN1KtKF6jvFVzNS1+JzaS0D0kCVKHKkd4sn+MQ@mail.gmail.com> <459C86F8-A989-4EF4-84DC-3568FF594F36@apple.com> <CANatvzwSpSHd7kZD-4tyMGkBJDdCBi6r_pLBvnaT8rrQy6SBHQ@mail.gmail.com> <CACMu3treK0m2mbpw9FebOjOcEed0bW-DbLbryHJH1DWAHoz+9g@mail.gmail.com> <CALGR9oZgE7ZfXdoYdUh9LUYC1fi8fMUyyTpvmV3GF7Z6Oxgg1g@mail.gmail.com> <20200609144428.GC22180@lubuntu> <CAJV+MGyuhxx=P6kZKktuREeq5pipZjxmwWP4jE_Sxhj_+krU2Q@mail.gmail.com> <CANatvzx_eg84V7UefOtSF+NHGHnTg7h-9n5bsRZRXxBqsaOkfQ@mail.gmail.com> <CACj=BEip6+7AunFsD=6qM5rsgrTfg6bRctOMu1gOe-KVjAW7Dw@mail.gmail.com> <CANatvzyv03VH9=+J=M2yY0EwCXp7HMWsXYaXOE=WYGDKBHdaVA@mail.gmail.com> <2C53D8AF-EFA8-42A3-9666-955A054468DB@greenbytes.de> <CACj=BEic2qzMXEfcsKS9CYnowChc-kMRjH66d3uKs+pqTz9Fug@mail.gmail.com> <4E0E8032-A903-46A2-A131-F1F4DE3CC037@greenbytes.de> <CACj=BEjOC-8S38U36Jw+Yb7yH_BZjxBkeLE6dLWH=8VMyBW80Q@mail.gmail.com> <ECF2C350-5D53-4E3B-9AAC-2F7E3FD4B528@greenbytes.de> <CACj=BEh53ZWj1UV6tNDaWPiuHbmbVmkYimu_rdYcYm06dZJAAQ@mail.gmail.com> <CAJV+MGzesbFRZFGK5SUM70HJrx3fdJ8AAGGqmwqDQhrL3eFmUw@mail.gmail.com> <CACj=BEiVPfOPGPuSu-tx5AovWwvTEwjCTqEooMq2muEteHZLAw@mail.gmail.com> <CAJV+MGyrKyhamN3WY+HE1i_0hKWQo6kuLe-6hO53YMrHPbto=w@mail.gmail.com> <CAM37g06T_U+Th83D7sH3S_LB-b_9TcaX5b9nDSFE8Z-x5U+c9A@mail.gmail.com> <CANatvzwEr97ec5h=YOZopFt2ou+vabthD+YMRwh0aESKy2jkkA@mail.gmail.com> <CAM37g05x7hZ=AfAicFHmPBxtz4VJhH14cCF2MKcRpfvq4_y1Cw@mail.gmail.com>
In-Reply-To: <CAM37g05x7hZ=AfAicFHmPBxtz4VJhH14cCF2MKcRpfvq4_y1Cw@mail.gmail.com>
From: Lucas Pardue <lucaspardue.24.7@gmail.com>
Date: Fri, 19 Jun 2020 17:22:30 +0100
Message-ID: <CALGR9oZdQnmdVo5hhA68r6CtRtDZUCRpBfbZjqQy=ggv9xmfeQ@mail.gmail.com>
To: HTTP Working Group <ietf-http-wg@w3.org>
Content-Type: multipart/alternative; boundary="00000000000070bdfd05a8724d02"
Received-SPF: pass client-ip=2a00:1450:4864:20::32a; envelope-from=lucaspardue.24.7@gmail.com; helo=mail-wm1-x32a.google.com
X-W3C-Hub-Spam-Status: No, score=-7.8
X-W3C-Hub-Spam-Report: BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, W3C_AA=-1, W3C_IRA=-1, W3C_IRR=-3, W3C_WL=-1
X-W3C-Scan-Sig: titan.w3.org 1jmJn4-00008d-6H 3d2b1b61a7e38b2afadfbb36a3f3065b
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Priority implementation complexity (was: Re: Extensible Priorities and Reprioritization)
Archived-At: <https://www.w3.org/mid/CALGR9oZdQnmdVo5hhA68r6CtRtDZUCRpBfbZjqQy=ggv9xmfeQ@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/37800
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <https://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>
Thanks for the different perspectives on this. Quoting isn't going to work so great so I'll pick out some points from this and the parent thread: Google have added a flag [1] to Chrome that allows toggling of H2 reprioritization, some experimental work is happening with this. Thanks! We've talked a bit about how priorities might affect both scheduling server work, and selecting the bytes to emit from the server's response send queue. I agree with Kazuho that we don't want to specify much about the internals of server applications. However, there are some DoS considerations (depending on the outcome of the repriority discussion), so looking ahead we might find it is useful to capture anything already not already covered in the spec. The server's role in optimizing the use of available bandwidth is an interesting perspective to take, especially considering the client's responsibility for providing flow control updates. In the basic HTTP/3 priority implementation of the quiche library, the application processes the priority header and provides that information when sending response data. Internally the library uses an implementation-specific API method to set the priority of the transport stream; this does account for the properties Kazuho mentioned i) it returns an error if the stream ID exceeds current maximum ii) it no-ops if the stream ID refers to a closed stream. HTTP response payload data is written to each stream until the QUIC layer tells me it cannot take any more because the send window is full. In a separate loop, the QUIC layer selects stream data to transmit based on the priority. The client's expedience in providing flow control updates affects phase 1 (local buffering) but not phase 2 (emission). A client reprioritization would affect phase 2 not phase 1. In my case, quiche's transport priority method does account for the properties Kazuho mentioned i) it returns an error if the stream ID exceeds current maximum ii) it no-ops if the stream ID refers to a closed stream. The funnies will happen with trying to accommodate reprioritization signals: - Exposing the reception of a reprioritization signal (PRIORITY_UPDATE frame) to the application might be useful, or useless if we consider some of Stefan's points. - Reordering can cause the reprioritization to arrive before the initial priority. Exposing an event to the application just made things harder. - Reordering isn't the only concern. In quiche, when an application asks us to read from transport, we internally always read from the control stream and QPACK streams before request streams. So we'd always pull out the PRIORITY_UPDATE first. - Exposing this scenario of reprioritization event to the application is mostly useless because the application has no idea of what is being reprioritized. If the priority is used for deciding server work, one of the layers above transport needs to first validate and then remember the details. This means that the library needs to expose a broader API surface than it already does (e.g. exposing Kazuho's properties) - If the transport layer API simply actions the last invoked priority, naively calling it when the signals were received in the "wrong" order means that reprioritization might be ignored. - If a reprioritization event is simply hair pinned back into the quiche library, there is an argument for not exposing it. - I could simply accommodate things by modifying the transport priority method to take a bool, is_initial. This would prevent an initial priority from being applied after a reprioritization. In conjunction, defining in the spec that initial priority is *always the header* would remove some of the complexity of buffering data above the transport layer. All of this is additional consideration and speculation specific to my implementation, applicability to others can vary. I can see how things would be harder for implementers that attempt to manage more of the priority scheme in the HTTP/3 layer than the QUIC one. We also haven't mentioned reprioritization of server push. The client cannot control the initial priority of a pushed response and there is an open issue about the default priority of a push [2]. In that thread we are leaning towards defining no default priority and letting a server pick based on information *it* has. However, Mike Bishop's point about reprioritizing pushes is interesting [3]. To paraphrase, if you consider the RTT of the connection, there are three conditions: a) the push priority was low: so no data was sent by the time a reprioritization was received at the server. It is possible to apply the reprioritization but importantly, the push was pointless and we may as well have waited for the client to make the request. b) the push priority was high, response size "small": so all data was sent by the time a reprioritization was received at the server. The reprioritization was useless. c) the push priority was high, response size "large": some data sent at initial priority but at the time a reprioritization is received at the server, the remaining data can be sent appropriately. However, anecdotally we know that pushing large objects is not a good idea. If we agree to those conditions, it makes for a poor argument to keep reprioritization of server push. But maybe there is data that disagrees. Cheers Lucas [1] - https://chromium-review.googlesource.com/c/chromium/src/+/2232923 [2] - https://github.com/httpwg/http-extensions/issues/1056 [3] - https://github.com/httpwg/http-extensions/issues/1056#issuecomment-593496441
- Extensible Priorities and Reprioritization Lucas Pardue
- Re: Extensible Priorities and Reprioritization Yoav Weiss
- Re: Extensible Priorities and Reprioritization Lucas Pardue
- Re: Extensible Priorities and Reprioritization Patrick Meenan
- Re: Extensible Priorities and Reprioritization Lucas Pardue
- Re: Extensible Priorities and Reprioritization Yoav Weiss
- Re: Extensible Priorities and Reprioritization Eric Kinnear
- Re: Extensible Priorities and Reprioritization Kazuho Oku
- Re: Extensible Priorities and Reprioritization Martin Thomson
- Re: Extensible Priorities and Reprioritization Bence Béky
- Priority implementation complexity (was: Re: Exte… Lucas Pardue
- Re: Priority implementation complexity (was: Re: … Dmitri Tikhonov
- Re: Priority implementation complexity (was: Re: … Patrick Meenan
- Re: Extensible Priorities and Reprioritization Patrick Meenan
- Re: Extensible Priorities and Reprioritization Lucas Pardue
- Re: Priority implementation complexity (was: Re: … Lucas Pardue
- Re: Priority implementation complexity (was: Re: … Dmitri Tikhonov
- Re: Priority implementation complexity (was: Re: … Roy T. Fielding
- Re: Extensible Priorities and Reprioritization Kinuko Yasuda
- Re: Extensible Priorities and Reprioritization Kinuko Yasuda
- Re: Priority implementation complexity (was: Re: … Kazuho Oku
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Martin Thomson
- Re: Priority implementation complexity (was: Re: … Kazuho Oku
- Re: Priority implementation complexity (was: Re: … Stefan Eissing
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Stefan Eissing
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Stefan Eissing
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Patrick Meenan
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Patrick Meenan
- Re: Priority implementation complexity (was: Re: … Kazuho Oku
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Barry Pollard
- Re: Priority implementation complexity (was: Re: … Barry Pollard
- Re: Extensible Priorities and Reprioritization Kazuho Oku
- Re: Extensible Priorities and Reprioritization Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Lucas Pardue
- Re: Priority implementation complexity (was: Re: … Tom Bergan
- Re: Priority implementation complexity (was: Re: … Lucas Pardue
- Re: Priority implementation complexity (was: Re: … Tom Bergan
- Re: Priority implementation complexity (was: Re: … Lucas Pardue
- Re: Extensible Priorities and Reprioritization Patrick Meenan
- Re: Extensible Priorities and Reprioritization Patrick Meenan
- Re: Extensible Priorities and Reprioritization Lucas Pardue
- Reprioritization - implementation intent Mark Nottingham
- Re: Reprioritization - implementation intent Eric Kinnear
- Nice to have guidance (was: Re: Reprioritization … Lucas Pardue
- Re: Nice to have guidance (was: Re: Reprioritizat… Eric Kinnear
- Re: Reprioritization - implementation intent Yoav Weiss
- Re: Reprioritization - implementation intent Yoav Weiss
- Re: Reprioritization - implementation intent Yoav Weiss