Re: Priority implementation complexity (was: Re: Extensible Priorities and Reprioritization)
Lucas Pardue <lucaspardue.24.7@gmail.com> Fri, 19 June 2020 21:47 UTC
Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 70B1B3A0ED2 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Fri, 19 Jun 2020 14:47:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.748
X-Spam-Level:
X-Spam-Status: No, score=-2.748 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, MAILING_LIST_MULTI=-1, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YtXiYnDzmZvt for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Fri, 19 Jun 2020 14:47:43 -0700 (PDT)
Received: from lyra.w3.org (lyra.w3.org [128.30.52.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 49E4F3A0ED1 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Fri, 19 Jun 2020 14:47:42 -0700 (PDT)
Received: from lists by lyra.w3.org with local (Exim 4.92) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1jmOoL-00086l-7t for ietf-http-wg-dist@listhub.w3.org; Fri, 19 Jun 2020 21:44:33 +0000
Resent-Date: Fri, 19 Jun 2020 21:44:33 +0000
Resent-Message-Id: <E1jmOoL-00086l-7t@lyra.w3.org>
Received: from titan.w3.org ([128.30.52.76]) by lyra.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <lucaspardue.24.7@gmail.com>) id 1jmOoI-00085v-RR for ietf-http-wg@listhub.w3.org; Fri, 19 Jun 2020 21:44:30 +0000
Received: from mail-wm1-x32a.google.com ([2a00:1450:4864:20::32a]) by titan.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from <lucaspardue.24.7@gmail.com>) id 1jmOoG-0008T8-ME for ietf-http-wg@w3.org; Fri, 19 Jun 2020 21:44:30 +0000
Received: by mail-wm1-x32a.google.com with SMTP id t194so10351833wmt.4 for <ietf-http-wg@w3.org>; Fri, 19 Jun 2020 14:44:28 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=rRbxPSLb634MXjpMbYpmbp7ZHpVAGctmTs4EIIR7Gfs=; b=guQ94SskdeFn6q696QttTksM5pFqyeLasPDKDn86lQoY2afph4HARuzV1s7C0KbjNB eEbgyXL8P6OYN2TX9ZCQuOz4qsbxKO7Wa7fElQ2HYtFFAN+p151gciESaH9kmdNcuQnP LglJQ/yiW9al9MFSJtc+is5c98n/Rf2rbYLzow6AaTQTQyoPwkv7SCIxHM1YvrRzoaES fnGZmTdqn6cmmweAIVZCQCrU1mQO7p4QT0vCOySkN9zulMsVxmu+1Pw0+T/yRge0C3M8 sDJu2EmnmHOlJXc8dNpITNXrgWl08cxmwQEgBp+55ewGOC/dnKLfA6zRmWZIiT3FXjC7 qomg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=rRbxPSLb634MXjpMbYpmbp7ZHpVAGctmTs4EIIR7Gfs=; b=HN2fRsFxSpCps0ex7+x7LapMbGIlssuagI11X0NkTyIgokWqJHYD0XSoy5KqLXujep /MT0DuYh6TxL5YpqQCRb0YvID39ovdo95jTEDDIt1SBYCSdj1bexWqqGk6RwgKEjzhvW oFNg3agCirS48lYBz/gMUTX9biyACDOeaMsw3Do2QfxNclQW5F7kVtH6HPEawpK8mRza WI59QLouB1BDScumTCd5vh3R0PgrjTjQPUrs5c/5H0XMsafaxB+NcQZt7yBD0/zchBwh 6bMTvTeBED8dfXoa6YJ9Qtm231NBA+vXcOmn2/D1wH+TZtCZH3/iCwxdWKm4FBno15Ng gWYg==
X-Gm-Message-State: AOAM531pzBSbsst9bUCf0Qh2ISISOYIRbLIj26BU3cLt0jSc/Bj8OA7b fFbfAHPdM4SATFVG05OTUYxYSLyf2EtyWopuSPI=
X-Google-Smtp-Source: ABdhPJwPLrOmozXXD0SMvkigK7Xo1KsY0lvTYL7RsjV+ogW/pPOqHObSa4xfM14Q4zPclRSZmPhhgTHUNHLyu6Sa4B0=
X-Received: by 2002:a1c:2b01:: with SMTP id r1mr5966758wmr.26.1592603057218; Fri, 19 Jun 2020 14:44:17 -0700 (PDT)
MIME-Version: 1.0
References: <CALGR9obRjBSADN1KtKF6jvFVzNS1+JzaS0D0kCVKHKkd4sn+MQ@mail.gmail.com> <459C86F8-A989-4EF4-84DC-3568FF594F36@apple.com> <CANatvzwSpSHd7kZD-4tyMGkBJDdCBi6r_pLBvnaT8rrQy6SBHQ@mail.gmail.com> <CACMu3treK0m2mbpw9FebOjOcEed0bW-DbLbryHJH1DWAHoz+9g@mail.gmail.com> <CALGR9oZgE7ZfXdoYdUh9LUYC1fi8fMUyyTpvmV3GF7Z6Oxgg1g@mail.gmail.com> <20200609144428.GC22180@lubuntu> <CAJV+MGyuhxx=P6kZKktuREeq5pipZjxmwWP4jE_Sxhj_+krU2Q@mail.gmail.com> <CANatvzx_eg84V7UefOtSF+NHGHnTg7h-9n5bsRZRXxBqsaOkfQ@mail.gmail.com> <CACj=BEip6+7AunFsD=6qM5rsgrTfg6bRctOMu1gOe-KVjAW7Dw@mail.gmail.com> <CANatvzyv03VH9=+J=M2yY0EwCXp7HMWsXYaXOE=WYGDKBHdaVA@mail.gmail.com> <2C53D8AF-EFA8-42A3-9666-955A054468DB@greenbytes.de> <CACj=BEic2qzMXEfcsKS9CYnowChc-kMRjH66d3uKs+pqTz9Fug@mail.gmail.com> <4E0E8032-A903-46A2-A131-F1F4DE3CC037@greenbytes.de> <CACj=BEjOC-8S38U36Jw+Yb7yH_BZjxBkeLE6dLWH=8VMyBW80Q@mail.gmail.com> <ECF2C350-5D53-4E3B-9AAC-2F7E3FD4B528@greenbytes.de> <CACj=BEh53ZWj1UV6tNDaWPiuHbmbVmkYimu_rdYcYm06dZJAAQ@mail.gmail.com> <CAJV+MGzesbFRZFGK5SUM70HJrx3fdJ8AAGGqmwqDQhrL3eFmUw@mail.gmail.com> <CACj=BEiVPfOPGPuSu-tx5AovWwvTEwjCTqEooMq2muEteHZLAw@mail.gmail.com> <CAJV+MGyrKyhamN3WY+HE1i_0hKWQo6kuLe-6hO53YMrHPbto=w@mail.gmail.com> <CAM37g06T_U+Th83D7sH3S_LB-b_9TcaX5b9nDSFE8Z-x5U+c9A@mail.gmail.com> <CANatvzwEr97ec5h=YOZopFt2ou+vabthD+YMRwh0aESKy2jkkA@mail.gmail.com> <CAM37g05x7hZ=AfAicFHmPBxtz4VJhH14cCF2MKcRpfvq4_y1Cw@mail.gmail.com> <CALGR9oZdQnmdVo5hhA68r6CtRtDZUCRpBfbZjqQy=ggv9xmfeQ@mail.gmail.com> <CA+3+x5HRMOD_XRUpGRqFY=pttj=izswzSLdSDKuKXhAPCx6wfQ@mail.gmail.com>
In-Reply-To: <CA+3+x5HRMOD_XRUpGRqFY=pttj=izswzSLdSDKuKXhAPCx6wfQ@mail.gmail.com>
From: Lucas Pardue <lucaspardue.24.7@gmail.com>
Date: Fri, 19 Jun 2020 22:44:05 +0100
Message-ID: <CALGR9obh6GmU2nToNwq1XWMJ9JG6VpyE-sg7RTjeL3fETJk-VQ@mail.gmail.com>
To: Tom Bergan <tombergan@chromium.org>
Cc: HTTP Working Group <ietf-http-wg@w3.org>
Content-Type: multipart/alternative; boundary="0000000000007e690005a876cb7f"
Received-SPF: pass client-ip=2a00:1450:4864:20::32a; envelope-from=lucaspardue.24.7@gmail.com; helo=mail-wm1-x32a.google.com
X-W3C-Hub-Spam-Status: No, score=-7.8
X-W3C-Hub-Spam-Report: BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, W3C_AA=-1, W3C_IRA=-1, W3C_IRR=-3, W3C_WL=-1
X-W3C-Scan-Sig: titan.w3.org 1jmOoG-0008T8-ME c2fb5f4e8d711b9d8012660db0d50539
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Priority implementation complexity (was: Re: Extensible Priorities and Reprioritization)
Archived-At: <https://www.w3.org/mid/CALGR9obh6GmU2nToNwq1XWMJ9JG6VpyE-sg7RTjeL3fETJk-VQ@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/37802
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <https://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>
On Fri, Jun 19, 2020 at 7:10 PM Tom Bergan <tombergan@chromium.org> wrote: > I didn't follow those details. I think it would be helpful to summarize > the API you're referring to. > > This might be naive: While there are potentially tricky implementation > issues, as discussed by Kazuho and Patrick earlier, and potentially tricky > scheduling decisions, as discussed by Stefan, I'm not seeing how those > translate into API problems. Generally speaking, at the HTTP application > level, a request doesn't really exist until the HEADERS arrive (example > <https://golang.org/pkg/net/http/#Handler>; all other HTTP libraries I'm > familiar with work in basically the same way). At that point, the request > has an initial priority, defined either by the HEADERS, or by the > PRIORITY_UPDATE, if one arrived before HEADERS and there's no Priority > field. Further PRIORITY_UPDATEs can be delivered with whatever event > mechanism is most convenient (callbacks, channels, etc). > The quiche HTTP/3 library layer provides a poll() method that an application calls. This queries readable transport streams and tries to read requests (a complete headers list). Today, communicating the initial priority is easy, I just pass the Priority header that I received. The application chooses the priority to send responses by providing a desired priority in the send_request() method - there is no coupling between the request and response priority. send_request() more or less passes everything through to the QUIC library which manages the packetization of STREAM frames once data is queued up. Minimal state is required in the HTTP/3 library layer, once a response is started the application just tries to fill the transport stream send buffer (via the H3 layer) as quickly as the client drains it. When the application is ready to complete the response, it send the last piece of data with a FIN. The application can then forget about it. At this stage only the transport maintains stream state, because it is not complete until the client reads the remaining data. If we deem reprioritization as useful, then it needs to be supported through the full lifetime of the stream. Adding PRIORITY_UPDATE requires some more work in my HTTP/3 library layer. One question that comes to mind is if the application cares about the full sequence of PRIORITY_UPDATES, or if it is fine to skip/collapse them. Before the request has been poll()ed out it seems sensible to buffer PRIORITY_UPDATE, and then only present the most recent one. To call this the initial priority is a slight fib, "most recent priority at the time you discovered the request existed" is more apt but this is splitting hairs. The one concern I would have is where the priority header and most-recent value disagree. By passing both priorities out to the application I'm making it responsible for picking. Witholding a reprioritization event until after the request has been poll()ed helps a bit. But I think there is not a clean way to deal with reprioritization events after the application is done with the stream; if the application is nominally done with processing the request all it can do is tell the transport to behave differently. What's the point in that? Attempting to explain the oddities caused by QUIC's behavior is part of the API problem IMO. > We also haven't mentioned reprioritization of server push. The client >> cannot control the initial priority of a pushed response and there is an >> open issue about the default priority of a push [2]. In that thread we are >> leaning towards defining no default priority and letting a server pick >> based on information *it* has. However, Mike Bishop's point about >> reprioritizing pushes is interesting [3]. To paraphrase, if you consider >> the RTT of the connection, there are three conditions: >> >> a) the push priority was low: so no data was sent by the time a >> reprioritization was received at the server. It is possible to apply the >> reprioritization but importantly, the push was pointless and we may as well >> have waited for the client to make the request. >> b) the push priority was high, response size "small": so all data was >> sent by the time a reprioritization was received at the server. The >> reprioritization was useless. >> c) the push priority was high, response size "large": some data sent at >> initial priority but at the time a reprioritization is received at the >> server, the remaining data can be sent appropriately. However, anecdotally >> we know that pushing large objects is not a good idea. >> >> If we agree to those conditions, it makes for a poor argument to keep >> reprioritization of server push. But maybe there is data that disagrees. >> > > FWIW, I have the opposite interpretation. We can't ignore case (a) by > simply saying that "the push was pointless and we may as well have waited > for the client". That assumes the server should have known the push would > be pointless, but in practice that conclusion depends on a number of > factors that can be difficult to predict (size of other responses, > congestion control state, network BDP). Sometimes push is useful, sometimes > it's not, and when it's not, we should gracefully fallback to behavior that > is equivalent to not using push at all. From that perspective, case (a) is > WAI. > > This lack of a graceful fallback is a big reason why push can be such a > footgun. Frankly, if pushes cannot be reprioritized in this way, then IMO > push is essentially dead as a feature (and it's already on rocky ground, as > it's so hard to find cases where it works well in the first place). > That's a fair opinion too. Have you any thoughts about server push reprioritization being a motivating factor for maintaining the feature? Unfortunately I don't have a server push API that I can use to speculate about reprioritization, although I suspect that I'd have similar problems determining when a pushed response was "done", with the added complication that something would need to maintain state to map push IDs to stream IDs. Cheers Lucas
- Extensible Priorities and Reprioritization Lucas Pardue
- Re: Extensible Priorities and Reprioritization Yoav Weiss
- Re: Extensible Priorities and Reprioritization Lucas Pardue
- Re: Extensible Priorities and Reprioritization Patrick Meenan
- Re: Extensible Priorities and Reprioritization Lucas Pardue
- Re: Extensible Priorities and Reprioritization Yoav Weiss
- Re: Extensible Priorities and Reprioritization Eric Kinnear
- Re: Extensible Priorities and Reprioritization Kazuho Oku
- Re: Extensible Priorities and Reprioritization Martin Thomson
- Re: Extensible Priorities and Reprioritization Bence Béky
- Priority implementation complexity (was: Re: Exte… Lucas Pardue
- Re: Priority implementation complexity (was: Re: … Dmitri Tikhonov
- Re: Priority implementation complexity (was: Re: … Patrick Meenan
- Re: Extensible Priorities and Reprioritization Patrick Meenan
- Re: Extensible Priorities and Reprioritization Lucas Pardue
- Re: Priority implementation complexity (was: Re: … Lucas Pardue
- Re: Priority implementation complexity (was: Re: … Dmitri Tikhonov
- Re: Priority implementation complexity (was: Re: … Roy T. Fielding
- Re: Extensible Priorities and Reprioritization Kinuko Yasuda
- Re: Extensible Priorities and Reprioritization Kinuko Yasuda
- Re: Priority implementation complexity (was: Re: … Kazuho Oku
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Martin Thomson
- Re: Priority implementation complexity (was: Re: … Kazuho Oku
- Re: Priority implementation complexity (was: Re: … Stefan Eissing
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Stefan Eissing
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Stefan Eissing
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Patrick Meenan
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Patrick Meenan
- Re: Priority implementation complexity (was: Re: … Kazuho Oku
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Barry Pollard
- Re: Priority implementation complexity (was: Re: … Barry Pollard
- Re: Extensible Priorities and Reprioritization Kazuho Oku
- Re: Extensible Priorities and Reprioritization Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Lucas Pardue
- Re: Priority implementation complexity (was: Re: … Tom Bergan
- Re: Priority implementation complexity (was: Re: … Lucas Pardue
- Re: Priority implementation complexity (was: Re: … Tom Bergan
- Re: Priority implementation complexity (was: Re: … Lucas Pardue
- Re: Extensible Priorities and Reprioritization Patrick Meenan
- Re: Extensible Priorities and Reprioritization Patrick Meenan
- Re: Extensible Priorities and Reprioritization Lucas Pardue
- Reprioritization - implementation intent Mark Nottingham
- Re: Reprioritization - implementation intent Eric Kinnear
- Nice to have guidance (was: Re: Reprioritization … Lucas Pardue
- Re: Nice to have guidance (was: Re: Reprioritizat… Eric Kinnear
- Re: Reprioritization - implementation intent Yoav Weiss
- Re: Reprioritization - implementation intent Yoav Weiss
- Re: Reprioritization - implementation intent Yoav Weiss