Re: Priority implementation complexity (was: Re: Extensible Priorities and Reprioritization)
Yoav Weiss <yoav@yoav.ws> Thu, 11 June 2020 07:55 UTC
Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B25053A175A for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 11 Jun 2020 00:55:04 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.648
X-Spam-Level:
X-Spam-Status: No, score=-2.648 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, MAILING_LIST_MULTI=-1, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=yoav-ws.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mvxAL1FFW_xI for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 11 Jun 2020 00:55:02 -0700 (PDT)
Received: from lyra.w3.org (lyra.w3.org [128.30.52.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 663A13A0F59 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Thu, 11 Jun 2020 00:55:01 -0700 (PDT)
Received: from lists by lyra.w3.org with local (Exim 4.92) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1jjI0S-0001E2-25 for ietf-http-wg-dist@listhub.w3.org; Thu, 11 Jun 2020 07:52:12 +0000
Resent-Date: Thu, 11 Jun 2020 07:52:12 +0000
Resent-Message-Id: <E1jjI0S-0001E2-25@lyra.w3.org>
Received: from titan.w3.org ([128.30.52.76]) by lyra.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <yoav@yoav.ws>) id 1jjI0P-0001DC-PU for ietf-http-wg@listhub.w3.org; Thu, 11 Jun 2020 07:52:10 +0000
Received: from mail-lf1-x134.google.com ([2a00:1450:4864:20::134]) by titan.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from <yoav@yoav.ws>) id 1jjI0N-00088H-Dd for ietf-http-wg@w3.org; Thu, 11 Jun 2020 07:52:09 +0000
Received: by mail-lf1-x134.google.com with SMTP id u16so2969682lfl.8 for <ietf-http-wg@w3.org>; Thu, 11 Jun 2020 00:52:06 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yoav-ws.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=hwLTZncp/+GuhQqstVSZblY5hcCAilV+TYMlD4eEPok=; b=oi2DdIUILuli+Ezxav8JrQHQf5ZzOw/Z/msdXmVHLOtr6q/VD8MkSURxmTNaPQp3xw bJtzFAmrzqhrxGhjI/aaJIz5Q+b9d5sz3aiAFkkiWEm7XxQj2naJhujWrG7oCtM4cseb 1ocgeqR0pb+51fADlb6y9ilfPM1L4KdEOFni8+SVyLXtIVZHATM5nIZJ7GFNFwgPHSv2 ixkFWWeK1UhPbQ/dooMRsqzgIiKE/LXhsjAjqX7zyS5UDynZ2AJnbxs6gkMwuN+AcZYR inNNB7LI6YaV0EmHv9rWo15UwuKTNdhMbZhYZwwPi1JjIE6TjSAhTvExCJnHUJZdsxIr zzCg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=hwLTZncp/+GuhQqstVSZblY5hcCAilV+TYMlD4eEPok=; b=oh4q2fmnGJT1cfe2UVJjTIu2v4KRSc3oGnrx6CR8gaZsdvps1P8nHvqVUssxH19oY3 gIpPdOsLcZMmE4JrWzd2bcvUEO4NrqJb8KQdtPQYi1ZUpRMOxWehS+IY3/kT+uU2+t+Q m5wCcfiKRj7Uo++Rq3sFMe9iiWcrCi4/uc1hctKCXnCnU+yHd+cycFuMY+1iv780TG5T Ke3cFtetR8wj2AFM2jEkSnnJqHhM0uF+IC55QTS5hfK5W52L6jODdwEMni2ggb7RROfE FF2W2vhomSqFJDs8C5FyfZVPqipPQSnjazFPBGrhFULF9DufraerMczDRJh+z+19HfoX RyUQ==
X-Gm-Message-State: AOAM533kWNUJGh1F9Ce4OLV30KtNed24c8A3eJ/6u6Uldvo0s/yAuByj xRTv7cExmxY3OqDjwdvBA1dNv18jITzIF9iJRgYAaQ==
X-Google-Smtp-Source: ABdhPJwchkroCg+4E1ufNbsIJPAnSPmSTGRwxW3QRoKh7BE0hWOJAuoxuznJjikWdx+1CiBX12uzoo8EHoMnAIJzWY0=
X-Received: by 2002:a19:8447:: with SMTP id g68mr3713978lfd.132.1591861915453; Thu, 11 Jun 2020 00:51:55 -0700 (PDT)
MIME-Version: 1.0
References: <CALGR9obRjBSADN1KtKF6jvFVzNS1+JzaS0D0kCVKHKkd4sn+MQ@mail.gmail.com> <459C86F8-A989-4EF4-84DC-3568FF594F36@apple.com> <CANatvzwSpSHd7kZD-4tyMGkBJDdCBi6r_pLBvnaT8rrQy6SBHQ@mail.gmail.com> <CACMu3treK0m2mbpw9FebOjOcEed0bW-DbLbryHJH1DWAHoz+9g@mail.gmail.com> <CALGR9oZgE7ZfXdoYdUh9LUYC1fi8fMUyyTpvmV3GF7Z6Oxgg1g@mail.gmail.com> <20200609144428.GC22180@lubuntu> <CAJV+MGyuhxx=P6kZKktuREeq5pipZjxmwWP4jE_Sxhj_+krU2Q@mail.gmail.com> <CANatvzx_eg84V7UefOtSF+NHGHnTg7h-9n5bsRZRXxBqsaOkfQ@mail.gmail.com>
In-Reply-To: <CANatvzx_eg84V7UefOtSF+NHGHnTg7h-9n5bsRZRXxBqsaOkfQ@mail.gmail.com>
From: Yoav Weiss <yoav@yoav.ws>
Date: Thu, 11 Jun 2020 09:51:39 +0200
Message-ID: <CACj=BEip6+7AunFsD=6qM5rsgrTfg6bRctOMu1gOe-KVjAW7Dw@mail.gmail.com>
To: Kazuho Oku <kazuhooku@gmail.com>
Cc: Patrick Meenan <patmeenan@gmail.com>, Lucas Pardue <lucaspardue.24.7@gmail.com>, HTTP Working Group <ietf-http-wg@w3.org>, Bence Béky <bnc@chromium.org>
Content-Type: multipart/alternative; boundary="00000000000000993205a7ca3c56"
Received-SPF: pass client-ip=2a00:1450:4864:20::134; envelope-from=yoav@yoav.ws; helo=mail-lf1-x134.google.com
X-W3C-Hub-Spam-Status: No, score=-8.9
X-W3C-Hub-Spam-Report: BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, W3C_AA=-1, W3C_DB=-1, W3C_IRA=-1, W3C_IRR=-3, W3C_WL=-1
X-W3C-Scan-Sig: titan.w3.org 1jjI0N-00088H-Dd 7445f77a93851805385ae8d0deb9f04b
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Priority implementation complexity (was: Re: Extensible Priorities and Reprioritization)
Archived-At: <https://www.w3.org/mid/CACj=BEip6+7AunFsD=6qM5rsgrTfg6bRctOMu1gOe-KVjAW7Dw@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/37748
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <https://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>
On Thu, Jun 11, 2020 at 1:57 AM Kazuho Oku <kazuhooku@gmail.com> wrote: > > > 2020年6月10日(水) 0:20 Patrick Meenan <patmeenan@gmail.com>: > >> Maybe I'm missing something but the priority updates don't need to >> coordinate across multiple data streams, just between the one stream that >> is being reprioritized and the control stream. >> >> Would something like this not work? >> - Control stream gets priority update for stream X >> - If stream X is known and the request side is complete/closed then >> update the priority as requested >> > > The problem is that when a H3 server receives a reprioritization frame and > fails to find the state of the stream being designated by that frame, it > has to decide either to queue or drop the frame. As you correctly point > out, the size of the queue has to be bounded. > > To determine if it should queue or drop, a server needs to have access to > both of the following states maintained by the QUIC stack: > * i) current maximum stream ID permitted to the peer > * ii) list of stream IDs that have not been closed yet > > Without having access to (i), a server cannot reject reprioritization > frames specifying an unreasonable large stream ID. Without having access to > (ii), a server might start remembering information for streams that have > already been closed. > > The question is if we think it is okay to require all QUIC stacks to > provide access to this information (or to provide an API that allows an > application to query if a given stream ID meets the two criteria). > > I would also point out that the size of the queue should not be restricted > any further. This is because when reprioritization is considered as an > indispensable part of Extensible Priorities, a client might use the > reprioritization frame for sending initial priorities too, instead of using > the header field for indicating the initial priority. > > That is what Chrome does today. If an HTML contains 100 images, and if > Chrome receives them at once, it sends 100 PRIORITY_UPDATE frames, then > sends the requests for all those images, assuming that 100 is the maximum > stream concurrency permitted by the server. > If what Chrome is doing today is sub-optimal, it would be great if you could file an issue on that front. > > If some servers fail to implement reprioritization correctly, and if > clients rely overly on reprioritization, the negative impact on performance > could be far greater than when *not* having reprioritization. I think that > the concern that some of us have, and the reason why they (I) think > defining reprioritization as an optional feature would be a safer approach. > If that's indeed the case, we shouldn't define reprioritization *at all*. Making it optional is a cop-out, which would result in inconsistencies, meaning clients can't rely on it and need to work-around its absence. We should decide if reprioritization is good or bad, based on as much data as we can pull, and make sure it's implemented only if we see benefits for it in some cases, and then make sure it's only used in those cases. > > >> - If stream X is either not known or still in the process of receiving >> request details, store the priority update for stream X in a fixed >> queue/map (size can be small but a safe size would be the max number of >> streams supported) >> - If there is already a pending priority update for stream X, discard it >> and replace it with the current priority update >> - If the pending priority update queue is full, drop the oldest and >> insert the new update >> - When a new request stream closes, check the pending priority update >> queue to see if there is an update waiting for the stream. If so, remove it >> from the queue and apply the new priority >> >> There should be no DOS concerns since the queue is fixed and small. The >> performance overhead would be trivial if we assume that out-of-order >> reprioritizations are rare (i.e. the list will almost always be empty). >> >> On Tue, Jun 9, 2020 at 10:48 AM Dmitri Tikhonov < >> dtikhonov@litespeedtech.com> wrote: >> >>> On Tue, Jun 09, 2020 at 03:15:44PM +0100, Lucas Pardue wrote: >>> > I can hypothesize that an implementation with QPACK dynamic support has >>> > already crossed the threshold of complexity that means implementing >>> > reprioritization is not burdensome. I'd like to hear from other >>> > implementers if they agree or disagree with this. >>> >>> I don't think we can judge either way. If Alice implements QPACK and >>> Bob implement reprioritization, results will vary based on their level >>> of competence. The degree of burden will also vary for each >>> particular implementation. Speaking for lsquic, reprioritization >>> had to [1] touch more code and was much more tightly coupled than >>> QPACK; on the other had, QPACK encoder logic was a lot more code. >>> >>> At a higher level, I don't understand the concern with complexity. >>> If you look up "complexity" in the dictionary, you will see >>> >>> complexity (n), see QUIC. >>> >>> - Dmitri. >>> >>> 1. Before it was ripped out of the spec, that is, thanks a lot... >>> >>> > > -- > Kazuho Oku >
- Extensible Priorities and Reprioritization Lucas Pardue
- Re: Extensible Priorities and Reprioritization Yoav Weiss
- Re: Extensible Priorities and Reprioritization Lucas Pardue
- Re: Extensible Priorities and Reprioritization Patrick Meenan
- Re: Extensible Priorities and Reprioritization Lucas Pardue
- Re: Extensible Priorities and Reprioritization Yoav Weiss
- Re: Extensible Priorities and Reprioritization Eric Kinnear
- Re: Extensible Priorities and Reprioritization Kazuho Oku
- Re: Extensible Priorities and Reprioritization Martin Thomson
- Re: Extensible Priorities and Reprioritization Bence Béky
- Priority implementation complexity (was: Re: Exte… Lucas Pardue
- Re: Priority implementation complexity (was: Re: … Dmitri Tikhonov
- Re: Priority implementation complexity (was: Re: … Patrick Meenan
- Re: Extensible Priorities and Reprioritization Patrick Meenan
- Re: Extensible Priorities and Reprioritization Lucas Pardue
- Re: Priority implementation complexity (was: Re: … Lucas Pardue
- Re: Priority implementation complexity (was: Re: … Dmitri Tikhonov
- Re: Priority implementation complexity (was: Re: … Roy T. Fielding
- Re: Extensible Priorities and Reprioritization Kinuko Yasuda
- Re: Extensible Priorities and Reprioritization Kinuko Yasuda
- Re: Priority implementation complexity (was: Re: … Kazuho Oku
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Martin Thomson
- Re: Priority implementation complexity (was: Re: … Kazuho Oku
- Re: Priority implementation complexity (was: Re: … Stefan Eissing
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Stefan Eissing
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Stefan Eissing
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Patrick Meenan
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Patrick Meenan
- Re: Priority implementation complexity (was: Re: … Kazuho Oku
- Re: Priority implementation complexity (was: Re: … Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Barry Pollard
- Re: Priority implementation complexity (was: Re: … Barry Pollard
- Re: Extensible Priorities and Reprioritization Kazuho Oku
- Re: Extensible Priorities and Reprioritization Yoav Weiss
- Re: Priority implementation complexity (was: Re: … Lucas Pardue
- Re: Priority implementation complexity (was: Re: … Tom Bergan
- Re: Priority implementation complexity (was: Re: … Lucas Pardue
- Re: Priority implementation complexity (was: Re: … Tom Bergan
- Re: Priority implementation complexity (was: Re: … Lucas Pardue
- Re: Extensible Priorities and Reprioritization Patrick Meenan
- Re: Extensible Priorities and Reprioritization Patrick Meenan
- Re: Extensible Priorities and Reprioritization Lucas Pardue
- Reprioritization - implementation intent Mark Nottingham
- Re: Reprioritization - implementation intent Eric Kinnear
- Nice to have guidance (was: Re: Reprioritization … Lucas Pardue
- Re: Nice to have guidance (was: Re: Reprioritizat… Eric Kinnear
- Re: Reprioritization - implementation intent Yoav Weiss
- Re: Reprioritization - implementation intent Yoav Weiss
- Re: Reprioritization - implementation intent Yoav Weiss