Re: Extensible Priorities and Reprioritization

Lucas Pardue <lucaspardue.24.7@gmail.com> Mon, 06 July 2020 21:54 UTC

Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3BCA13A0B5B for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Mon, 6 Jul 2020 14:54:16 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.748
X-Spam-Level:
X-Spam-Status: No, score=-2.748 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, MAILING_LIST_MULTI=-1, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cqA8cbScMsiv for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Mon, 6 Jul 2020 14:54:11 -0700 (PDT)
Received: from lyra.w3.org (lyra.w3.org [128.30.52.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8E1E53A0B5C for <httpbisa-archive-bis2Juki@lists.ietf.org>; Mon, 6 Jul 2020 14:54:11 -0700 (PDT)
Received: from lists by lyra.w3.org with local (Exim 4.92) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1jsZ1J-0001CV-5g for ietf-http-wg-dist@listhub.w3.org; Mon, 06 Jul 2020 21:51:25 +0000
Resent-Date: Mon, 06 Jul 2020 21:51:25 +0000
Resent-Message-Id: <E1jsZ1J-0001CV-5g@lyra.w3.org>
Received: from titan.w3.org ([128.30.52.76]) by lyra.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from <lucaspardue.24.7@gmail.com>) id 1jsZ1H-0001Bk-Dj for ietf-http-wg@listhub.w3.org; Mon, 06 Jul 2020 21:51:23 +0000
Received: from mail-wm1-x329.google.com ([2a00:1450:4864:20::329]) by titan.w3.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from <lucaspardue.24.7@gmail.com>) id 1jsZ19-0002y9-IO for ietf-http-wg@w3.org; Mon, 06 Jul 2020 21:51:22 +0000
Received: by mail-wm1-x329.google.com with SMTP id o8so40777971wmh.4 for <ietf-http-wg@w3.org>; Mon, 06 Jul 2020 14:51:14 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=MjwQkgUfFNM5B9iIuxA9J6NpZ2oyYPoRJUgkY0+8H1E=; b=E8+arX+JnNXfox4r+6Ux5UzUH7TtqvzEGmppD4pCts/zB7AKjgzlQgj7N5NMVI0MN8 EosXkto3ugzwFVgdSTEAKSIFsBPCJ6MyyVVe24zS9G85qriLSL/q/DBjGRcYTdgFKPGu ZgtpeRa92E0/m8Ux3w+I/129jQ7VVeIbeLWPNSi3PGSCh3+gLtfQAThcu3kiwC9uNyoY 2SzqPOv/YPk9h9JFOrqC+kCrnXMIZwY2xJw9ztIsZp44HNx3Uy0XF8yubz4fLY/88JnN yhxTqNnPBY2KGZ3NjK/3VhoUlfZJjLoImopgrFj0n/1vWQSOzdnB0N5iWYNfpETCo3TG l/5Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=MjwQkgUfFNM5B9iIuxA9J6NpZ2oyYPoRJUgkY0+8H1E=; b=dOzfDBCTX7xvrguVpLMzaKLFzDfHY2ABV/hpKQEkLbHfki/5TPIBFW+DVTVn5UAw88 ay6Pgv0uxmJ6dS5vKRCr/rlPqF9CB3o7B2ol2NmL1jNCHoDMXfEd56I+qgL8TKZDUvg6 Aa5XeyGzp9FDuu7sWvmsR/DCv+KMEbmxXiK0OP+gQao8U8IYeBFySKhMY+yeAq2nqKhR 4wHhZOTDhrlI+qv6aHg6GCBHtgAB4SJYDlvXVO5g1xe3pRItl+6YuL41subVYTw8CEmI zJDraCVW5HwutfllftyMFPIu0bwx7/eBN+RVn8QdQ/ghcpxl33NY72pL47Rl38hDacUo zOZA==
X-Gm-Message-State: AOAM531yzVcrj4sCDVUZQRMeUOejAy33b51PH0YBx7d+TB8dQThXSr+N Jvmv46Fj8PmG2LGbGEgfwJZNhxAuWnUaLFy8Byw=
X-Google-Smtp-Source: ABdhPJz7pF8J92WZl9lG2D9FYcNboF8MoA+S8aPoj8zngtTYbEDkttJ/ArGIDFmGdn+q/YxPrsAmzZGMIiFA6X58z+w=
X-Received: by 2002:a1c:6788:: with SMTP id b130mr1126049wmc.100.1594072263214; Mon, 06 Jul 2020 14:51:03 -0700 (PDT)
MIME-Version: 1.0
References: <CALGR9obRjBSADN1KtKF6jvFVzNS1+JzaS0D0kCVKHKkd4sn+MQ@mail.gmail.com> <459C86F8-A989-4EF4-84DC-3568FF594F36@apple.com> <CANatvzwSpSHd7kZD-4tyMGkBJDdCBi6r_pLBvnaT8rrQy6SBHQ@mail.gmail.com> <CACMu3treK0m2mbpw9FebOjOcEed0bW-DbLbryHJH1DWAHoz+9g@mail.gmail.com> <CAJV+MGy2CytgPVEwEO3nDfpZ6h9+CCL-bODk=65cXexvS3N7Lw@mail.gmail.com> <CALGR9oYDApddLFzXv180TEXpmTaOpDCDNY41PxmbMJK7N4F4zQ@mail.gmail.com> <CAMWgRNaMZxph3zQv+O-SW7=PKBtDuGZNQ4+3X2geyXU545Vx9w@mail.gmail.com> <CAMWgRNaBAodWewpbi4cqFiMLWVd0SDnau7B4x0tjk+i=sMURpQ@mail.gmail.com> <CANatvzyQiNXY6xOYju8afe7-T6ZNMtQTPQE-AkfFK=2_yTzB1Q@mail.gmail.com> <CACj=BEhh+K=uMS613OsDFmvH18miNvm9m11M7QsL02Lc+JxUhg@mail.gmail.com> <CAJV+MGx3-cvPER2q1SPsgTbVP0TwAgPzNCQk_40dDPSr3JfkNg@mail.gmail.com> <CAJV+MGwXLoVe3RWPMCw9iJQ1Qr0TrJOezWq1VWOqrWYnBneQ4Q@mail.gmail.com>
In-Reply-To: <CAJV+MGwXLoVe3RWPMCw9iJQ1Qr0TrJOezWq1VWOqrWYnBneQ4Q@mail.gmail.com>
From: Lucas Pardue <lucaspardue.24.7@gmail.com>
Date: Mon, 6 Jul 2020 22:50:56 +0100
Message-ID: <CALGR9ob5E9ufzut7gZ37HwFvzeUb8mZYcyy=M3xKhS3hCCfyyw@mail.gmail.com>
To: Patrick Meenan <patmeenan@gmail.com>
Cc: Yoav Weiss <yoav@yoav.ws>, Kazuho Oku <kazuhooku@gmail.com>, Kinuko Yasuda <kinuko@chromium.org>, =?UTF-8?Q?Bence_B=C3=A9ky?= <bnc@chromium.org>, Eric Kinnear <ekinnear@apple.com>, Patrick Meenan <pmeenan@webpagetest.org>, HTTP Working Group <ietf-http-wg@w3.org>
Content-Type: multipart/alternative; boundary="000000000000fecbad05a9ccde35"
Received-SPF: pass client-ip=2a00:1450:4864:20::329; envelope-from=lucaspardue.24.7@gmail.com; helo=mail-wm1-x329.google.com
X-W3C-Hub-Spam-Status: No, score=-7.8
X-W3C-Hub-Spam-Report: BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, W3C_AA=-1, W3C_IRA=-1, W3C_IRR=-3, W3C_WL=-1
X-W3C-Scan-Sig: titan.w3.org 1jsZ19-0002y9-IO cd4af8b8174925ed4feb42ed00f64079
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Extensible Priorities and Reprioritization
Archived-At: <https://www.w3.org/mid/CALGR9ob5E9ufzut7gZ37HwFvzeUb8mZYcyy=M3xKhS3hCCfyyw@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/37843
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <https://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

Hi Patrick,

Thanks for running this experiment and presenting the data back to the
group.

Also thanks to the Chrome folk for enabling the disabling flag.

Cheers
Lucas


On Mon, 6 Jul 2020, 21:19 Patrick Meenan, <patmeenan@gmail.com> wrote:

> Sorry about the delay, just gathered the results.  The full raw results
> are here
> <https://docs.google.com/spreadsheets/d/14iyeO--I-K-l7er1kGuW-iTKogsgXFz4Z3M-aowSUdI/edit?usp=sharing>.
> It looks like the impact dropped quite a bit across the full 25k URLs but
> looking at individual tests the impact is quite dramatic when it does
> impact (and it does exactly what we'd expect it to do for those outlier
> cases).
>
> The 95th percentile numbers tend to be the more interesting ones and in
> the data set, reprioritization enabled is the control and disabled is the
> experiment so positive changes means disabling reprioritization is that
> much slower.
>
> Largest Contentful Paint: 4% slower without reprioritization
> Speed Index: 2.75% slower without reprioritization
> Dom Content Loaded: 1.3% faster without reprioritization
>
> This is pretty much (directionally) what we'd expect since
> reprioritization boosts the priority of visible images (LPC/Speed Index)
> above late-body scripts (DCL). It's particularly dramatic for pages that
> use background images for any part of the page because they are discovered
> after all other resources and would normally be scheduled after all other
> scripts and inline images but if they are visible in the viewport the
> reprioritization helps them load much sooner.
>
> Looking at a few examples of the extreme cases:
>
> https://www.thehelm.co/ - (Filmstrip
> <https://www.webpagetest.org/video/compare.php?tests=200625_MT_32af039543326a2bdb5d87e2adb95fe7-r%3A3-l%3AStock%2C200701_HY_6bb4d26adff32186e991c5b96ffaecea-r%3A1-l%3ADisabled%2C&thumbSize=200&ival=5000&end=full>) -
> The main background image in the interstitial loads at < 10s vs 90s without
> reprioritization
> https://blog.nerdfactory.ai/ - (Filmstrip
> <https://www.webpagetest.org/video/compare.php?tests=200616_KZ_496553703935231e5725c252844918db-r%3A1-l%3AStock%2C200616_BJ_798e2417374c03dfa3995586b01444a3-r%3A3-l%3ADisabled%2C&thumbSize=200&ival=5000&end=full>)
> - The background image for the main content loads at <5s vs 70s without
> reprioritization. No cost to DCL, just prioritized ahead of not-visible
> images.
> https://events.nuix.com/ - (Filmstrip
> <https://www.webpagetest.org/video/compare.php?tests=200628_1A_0e61aa9f59e08f3bbb8e5d9690fe64fb-r%3A3-l%3AStock%2C200628_Q2_1793a50022566cedd6ab48dd871d5c7e-r%3A3-l%3ADisabled%2C&thumbSize=200&ival=5000&end=full>)
> - Another hero background image (detecting a theme?) loads at 10s vs 60s
>
> Looking at a few of the bigger DCL regressions:
>
> https://oaklandcitychurch.org/ - (Filmstrip
> <https://www.webpagetest.org/video/compare.php?tests=200627_XR_fae3bd7aa57238591cb122c9fe634cb7-r%3A2-l%3AStock%2C200627_R9_26ad61b65965e7bf89a8aa27a7d78ff1-r%3A2-l%3ADisabled%2C&thumbSize=200&ival=5000&end=full>)
> - DCL got much slower (11s -> 33s) as a direct result of the background
> image moving from 30s to 10s (the pop-up interstitial was delayed along
> with the scripts that control it).
>
> For the specific case that most of these tests exposed (background image
> discovered late by CSS) it is theoretically possible for Chrome to detect
> the position before making the initial request (since it is only discovered
> at layout anyway) but that wouldn't help any of the more dynamic cases like
> when a user scrolls a page or a carousel rotates and what is on screen
> changes dynamically.
>
> I'm still of the pretty strong opinion that we need reprioritization but
> the web won't necessarily break without it and sites (and browsers) may be
> able to minimize the impact of not being able to reprioritize (though that
> might involve holding back requests and prioritizing locally like Chrome
> does for slow HTTP/2 connections).
>
>
> On Sat, Jun 20, 2020 at 10:17 AM Patrick Meenan <patmeenan@gmail.com>
> wrote:
>
>> An early read on Yoav's Canary test is that most metrics are neutral but
>> Largest Contentful Paint degrades ~6.8% on average and 12% at the 95th
>> percentile without reprioritization and Speed Index degrades 2.6% on
>> average and 5.4% at the 95th percentile. This is not entirely unexpected
>> because the main use case for reprioritization in Chrome right now is
>> boosting the priority of visible images after layout is done.
>>
>> We'll see if it holds after the full test is complete. The early read is
>> from 3,000 of the 25,000 URLs that we are testing (all https hosted on
>> Fastly for simplicity since we know it handles HTTP/2 reprioritization
>> correctly).  The tests are all run at "3G Fast" speeds with desktop pages
>> to maximize the liklihood that there will be time for reprioritization to
>> happen.  I'll provide the full raw data as well as summary results when the
>> test is complete (at least another week, maybe 2).
>>
>> On Wed, Jun 17, 2020 at 5:43 AM Yoav Weiss <yoav@yoav.ws> wrote:
>>
>>>
>>>
>>> On Wed, Jun 17, 2020 at 9:55 AM Kazuho Oku <kazuhooku@gmail.com> wrote:
>>>
>>>>
>>>>
>>>> 2020年6月11日(木) 6:46 Kinuko Yasuda <kinuko@chromium.org>rg>:
>>>>
>>>>> (Sorry, sent it too soon...)
>>>>>
>>>>> On Thu, Jun 11, 2020 at 6:12 AM Kinuko Yasuda <kinuko@chromium.org>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> Reg: reprioritization benefit I can share some recent data for
>>>>>> Chrome.  For the two cases that are currently discussed I'm actually not
>>>>>> fully sure about its benefit.
>>>>>>
>>>>>> For the renderer-triggered image reprioritization cases: this is a
>>>>>> bit interesting one, we recently found two things:
>>>>>> - Delaying to start low-prio requests could often work better (partly
>>>>>> because of server-side handling) than re-prioritizing while inflight
>>>>>> - In-lab measurements (tested with top 10k real sites, both on Mobile
>>>>>> and Desktop) showed that removing in-flight re-prioritization doesn't
>>>>>> impact page load performance a lot
>>>>>>
>>>>>
>>>>> Let me stress though that testing this with servers that can properly
>>>>> handle reprioritization could change the landscape, and again this isn't
>>>>> really capturing how it affects long-lived request cases, or cases where
>>>>> tabs go foreground & background while loading, so for now I'm not very
>>>>> motivated to remove the reprioritization feature either.
>>>>>
>>>>
>>>> Hi Kinuko,
>>>>
>>>> Thank you for sharing your data. I feel a bit sad that reprioritization
>>>> isn't showing much benefit at the moment. I tend to agree that we are
>>>> likely to see different results between server implementations and HTTP
>>>> versions being used. The effectiveness of reprioritization depends on the
>>>> depth of the send buffer (after prioritization decision is made), at least
>>>> to certain extent.
>>>>
>>>
>>> FWIW, I added a flag
>>> <https://chromium-review.googlesource.com/c/chromium/src/+/2232923> to
>>> turn off Chromium's H2 request prioritization. I believe +Pat Meenan
>>> <patmeenan@gmail.com> is currently running tests with and without this
>>> flag a list of servers we estimate is likely to handle them well.
>>>
>>>
>>>>
>>>>
>>>>>
>>>>>
>>>>>> I suspect this is maybe because server-side handling is not always
>>>>>> perfect and most of requests on the web are short-lived, and this may not
>>>>>> be true for the cases where long-running requests matter.  I don't have
>>>>>> data for whether may impact background / foreground cases (e.g. Chrome
>>>>>> tries to lower priorities when tabs become background)
>>>>>>
>>>>>> For download cases, Chrome always starts a new download with a low
>>>>>> priority (even if it has started as a navigation), so reprioritization
>>>>>> doesn't happen.
>>>>>>
>>>>>> Kinuko
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 10, 2020 at 1:21 AM Lucas Pardue <
>>>>>> lucaspardue.24.7@gmail.com> wrote:
>>>>>>
>>>>>>> On Tue, Jun 9, 2020 at 4:27 PM Patrick Meenan <patmeenan@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Eric's download example is a great one for exposing the risks that
>>>>>>>> would come for an implementation that supported prioritization but not
>>>>>>>> reprioritization.
>>>>>>>>
>>>>>>>> Take the trivial example of an anchor link that links to a download
>>>>>>>> (say, a 200MB installer of some kind):
>>>>>>>> - When the user clicks on the link, the browser assumes it is doing
>>>>>>>> a navigation and issues the request with the "HTML" priority (relatively
>>>>>>>> high, possibly non-incremental
>>>>>>>> - When the response starts coming back, it has the
>>>>>>>> content-disposition to download to a file.
>>>>>>>> - At this point, the 200MB download will block every other
>>>>>>>> lower-priority request on the same connection (or possibly navigation if it
>>>>>>>> is non-incremental)
>>>>>>>> - The user clicks on another page on the same site and gets nothing
>>>>>>>> or a broken experience until the 200MB download completes
>>>>>>>>
>>>>>>>> Without reprioritization the browser will effectively have to burn
>>>>>>>> the existing QUIC connection and issue any requests on a new connection
>>>>>>>> (and repeat for each new download).
>>>>>>>>
>>>>>>>> Implementing prioritization without reprioritization in this case
>>>>>>>> is worse than having no prioritization support at all.
>>>>>>>>
>>>>>>>
>>>>>>> Thanks Eric for presenting this case, and Patrick for breaking it
>>>>>>> down. That does seem like a pretty bad outcome.
>>>>>>>
>>>>>>> Is this a good candidate for a test case? IIUC correctly the problem
>>>>>>> might occur today with HTTP/2 depending on how exclusive priorities are
>>>>>>> used. I'm curious if browsers can share any more information about what
>>>>>>> they do already. How does Firefox manage such a resource with it's priority
>>>>>>> groups?
>>>>>>>
>>>>>>> Cheers
>>>>>>> Lucas
>>>>>>>
>>>>>>>
>>>>
>>>> --
>>>> Kazuho Oku
>>>>
>>>