Re: Stream State and PRIORITY Frames
Scott Mitchell <scott.k.mitch1@gmail.com> Thu, 19 January 2017 22:50 UTC
Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D16DB129533 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 19 Jan 2017 14:50:37 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.719
X-Spam-Level:
X-Spam-Status: No, score=-9.719 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, RP_MATCHES_RCVD=-3.199, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gUt6YC41IfxC for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 19 Jan 2017 14:50:35 -0800 (PST)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id F3FF0129530 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Thu, 19 Jan 2017 14:50:34 -0800 (PST)
Received: from lists by frink.w3.org with local (Exim 4.80) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1cULUp-0007Qd-Nc for ietf-http-wg-dist@listhub.w3.org; Thu, 19 Jan 2017 22:47:55 +0000
Resent-Date: Thu, 19 Jan 2017 22:47:55 +0000
Resent-Message-Id: <E1cULUp-0007Qd-Nc@frink.w3.org>
Received: from mimas.w3.org ([128.30.52.79]) by frink.w3.org with esmtps (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from <scott.k.mitch1@gmail.com>) id 1cULUk-0007Pr-QB for ietf-http-wg@listhub.w3.org; Thu, 19 Jan 2017 22:47:50 +0000
Received: from mail-lf0-f52.google.com ([209.85.215.52]) by mimas.w3.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.84_2) (envelope-from <scott.k.mitch1@gmail.com>) id 1cULUc-0003N7-Kb for ietf-http-wg@w3.org; Thu, 19 Jan 2017 22:47:45 +0000
Received: by mail-lf0-f52.google.com with SMTP id k86so46553760lfi.0 for <ietf-http-wg@w3.org>; Thu, 19 Jan 2017 14:47:22 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=/1oNJHClpbmdTvJkiG5eL+9tPzFsfFAdic0XwtTRQWM=; b=a0iMpEfZhY3Jf5jDoRWdGllRujvTANoG5EKkEC8Xoph3kakSvWTccg0JJzzMmZyd19 2HKinjdGGZuL006PsdzyXbnfdtl7D7NIvPnMsPGPuEG8wgzARLHjcPAMrORFZuzsuQTZ q0qjjmaYgTFRqHiX+GVaaFeV8R20w6d+I2s1Q7L1ybUFzzX/M+RG6oWIGEIzxK3WLKsh CE2mO4Cu+PCjAe2M7SI46uECx/8exUEpJj2Mq398xB0DTrsKewdU1aRPInU0NkM8G0m8 3Find8dMgOGkhD/Mj8snx0vkbVrWbsjhDDUG4yeJW7aBF71Bn31t0Ko24ddWuQGse0zq b29w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=/1oNJHClpbmdTvJkiG5eL+9tPzFsfFAdic0XwtTRQWM=; b=OPcTDY3soxqtVL4w2qfAJJnLEHhbiP/XEr867etZ45wWlwwe3giBHwhxTF2uARpD7n zz3OuAKBmKGW7eK7fKxDXuiIh0dhpqd6MK4/+5LyTxUwt4p+Dtan7nT3yWhUhElnS2+S z21oaBJ15m35q4LUH7wf9H29KkQuS/35vpcgk3NUQdNBsk3vJuLuUj7EGNO9i0E1C2d0 c4cnOhGddVI0D0BlqTHgCtRhDglEX0PIwByuMK539uVxZg8B0lFs7A1ZTx3I1YD5VgOo 20as4b8sEWWJWmHqaVH9ve109EZvyPTF5RqgH0DyydFQ3KWJu1cXt/CSCfrXtStxLbLf Ecmw==
X-Gm-Message-State: AIkVDXJcGPmo3hMQJzbaXKsJBVy07ZXZxkL6xNBkZw6ItSTraYJ8FSip80nDWCwqPqQrhdfbnkdQpg+vJQGc5Q==
X-Received: by 10.25.156.144 with SMTP id f138mr4032929lfe.80.1484866035734; Thu, 19 Jan 2017 14:47:15 -0800 (PST)
MIME-Version: 1.0
Received: by 10.25.18.200 with HTTP; Thu, 19 Jan 2017 14:47:14 -0800 (PST)
In-Reply-To: <0FCBA000-E026-431A-802A-C2D39A98EF57@lukasa.co.uk>
References: <CAFn2buAYWHQSWhhoKZ2GKbqXR1A+tScjkAwZmOuQ9gV9jMp2bA@mail.gmail.com> <CAGZNdJWe0Y=M_SWmgYabKbWZwPEuJdw67Km8+UtR4oUtZuXA5A@mail.gmail.com> <2BC01E49-91A1-43A7-AFD0-5A34F2689428@lukasa.co.uk> <CAFn2buDCMwMp=rR0C_yt4-gUwyFhY6ruonz9wT-jVMu+Kir=nA@mail.gmail.com> <C51C3F51-37BE-489F-BA1D-76B101517307@lukasa.co.uk> <CAFn2buAE-EdTEG9x-wLQSy5Osmcq1Hdps9YLW_7j9CTV4_Fuyw@mail.gmail.com> <CANatvzzEV6Qd1huVxO=T2m7UwwX207RCetrXyZ1FMMGrBtzeEg@mail.gmail.com> <e420eda0-57fc-43e8-a82a-0988db75ab32@treenet.co.nz> <0FCBA000-E026-431A-802A-C2D39A98EF57@lukasa.co.uk>
From: Scott Mitchell <scott.k.mitch1@gmail.com>
Date: Thu, 19 Jan 2017 14:47:14 -0800
Message-ID: <CAFn2buD1Y6LBMWi-rKwXt9Ht0tZcuXYMKHGsCivzp58ZoJT27A@mail.gmail.com>
To: Cory Benfield <cory@lukasa.co.uk>
Cc: Amos Jeffries <squid3@treenet.co.nz>, HTTP Working Group <ietf-http-wg@w3.org>
Content-Type: multipart/alternative; boundary="001a11410e7c995f6e05467a5094"
Received-SPF: pass client-ip=209.85.215.52; envelope-from=scott.k.mitch1@gmail.com; helo=mail-lf0-f52.google.com
X-W3C-Hub-Spam-Status: No, score=-2.5
X-W3C-Hub-Spam-Report: AWL=0.732, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_SORBS_SPAM=0.5, SPF_PASS=-0.001, W3C_AA=-1, W3C_WL=-1
X-W3C-Scan-Sig: mimas.w3.org 1cULUc-0003N7-Kb 03900d0c4812c792acbfa084f3fd3228
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Stream State and PRIORITY Frames
Archived-At: <http://www.w3.org/mid/CAFn2buD1Y6LBMWi-rKwXt9Ht0tZcuXYMKHGsCivzp58ZoJT27A@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/33333
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>
On Thu, Jan 19, 2017 at 6:11 AM, Cory Benfield <cory@lukasa.co.uk> wrote: > > > On 19 Jan 2017, at 09:00, Amos Jeffries <squid3@treenet.co.nz> wrote: > > > > IIRC the intention behind the closing behaviour for idle streams was to > > ensure that we could optimize away the need to maintain a list or array > > of 2^31 state entries. > > > > Omitting PRIORITY from the MAX_STREAMS voids that benefit. > > No it doesn’t. > > Agreed. There is no mechanism defined in the specification to communicate a limit for the state retained due to prioritization/dependency information. This is currently limited to implementation specific heuristics. > Firstly, let’s note that the PRIORITY frame is defined not to change > stream state. Receiving a PRIORITY frame on stream N leaves it in the idle > state. A server implementation can thus treat the receipt of a PRIORITY frame on > any stream that has a higher ID than the last stream on which it received a > HEADERS frame as having the *exact* same effect on state retention as > receiving a PING frame. It doesn’t *require* the insertion of any stream > state data. > > Specific implementations may require the insertion of stream state data if > priority information is stored on the same structure as stream state > information, but that is certainly not required. For example, receiving a > bunch of PRIORITY frames does not cause the Python HTTP/2 implementation to > allocate any more state information than receiving a bunch of PING frames. > > > It is causing state to be allocated on the server and on every > > intermediary along the way which accepts it. > > > > Consider the effects of 2^31 PRIORITY frames being sent with different > > IDs before the first HEADERS is used. > > Notice how even this one case is markedly worse than sending just one > > HEADERS with stream ID == 2^31 to waste server sockets. > > This problem is orthogonal to the one we’re discussing. > > So far I haven’t seen a proposal to say that PRIORITY frames should > transition a stream out of the idle state. If we continue to leave RFC 7540 > saying that PRIORITY frames leave streams in the idle state, then > definitionally they are excluded from the restrictions on > MAX_CONCURRENT_STREAMS. So if you are allocating stream state for idle > streams, then you are open to this DoS vector, but none of the proposals > here are trying to address it. > > Additionally, I should note that the unbounded insertion of priority > information into priority trees was discussed as an attack vector in this > paper: https://www.imperva.com/docs/Imperva_HII_HTTP2.pdf. The paper was > published in August of last year, and several implementations took steps to > reduce their vulnerability to it (the Python priority implementation even > filed a CVE, CVE-2016-6580). > > I absolutely support adding an erratum to indicate that implementations > should resist unbounded insertion of PRIORITY information. But unless > you’re willing to say that idle streams should be counted against > MAX_CONCURRENT_STREAMS (which is a tricky thing to state), then the only > way to resolve this problem at the spec level is to force PRIORITY frames > to make state transitions, further complicating the state diagram for > streams. > > > What exactly does it mean for them to set PRIORITY on a non-existent > stream? > > It means exactly what RFC 7540 says it means. Quoting from Section 5.3.4 > of RFC 7540: > > > Similarly, streams that are in the "idle" state can be assigned priority > or become a parent of other streams. This allows for the creation of a > grouping node in the dependency tree, which enables more flexible > expressions of priority. Idle streams begin with a default priority > (Section 5.3.5). > > While we’re here, right below that section is: > > > The retention of priority information for streams that are not counted > toward the limit set by SETTINGS_MAX_CONCURRENT_STREAMS could create a > large state burden for an endpoint. Therefore, the amount of prioritization > state that is retained MAY be limited. > > To this end, the Python priority implementation allows a user-configured > maximum amount of data retention for priority information. This priority > information is held separately from stream state information because they > are entirely orthogonal concerns: in fact, the bits of code that manage > priority information and those that manage stream state information are in > entirely separate installable packages with no dependency relationship to > each other. > > To my eye, there is a clear intent to allow setting of priority > information on streams that are in the idle state, without that affecting > the stream itself. > > Cory > > > > > > > >
- Stream State and PRIORITY Frames Scott Mitchell
- Re: Stream State and PRIORITY Frames Benedikt Christoph Wolters
- Re: Stream State and PRIORITY Frames Tatsuhiro Tsujikawa
- Re: Stream State and PRIORITY Frames Cory Benfield
- Re: Stream State and PRIORITY Frames Scott Mitchell
- Re: Stream State and PRIORITY Frames Cory Benfield
- Re: Stream State and PRIORITY Frames Scott Mitchell
- Re: Stream State and PRIORITY Frames Matthew Kerwin
- Re: Stream State and PRIORITY Frames Scott Mitchell
- Re: Stream State and PRIORITY Frames Scott Mitchell
- Re: Stream State and PRIORITY Frames Scott Mitchell
- Re: Stream State and PRIORITY Frames Martin Thomson
- Re: Stream State and PRIORITY Frames Daurnimator
- Re: Stream State and PRIORITY Frames Tom Bergan
- Re: Stream State and PRIORITY Frames Cory Benfield
- Re: Stream State and PRIORITY Frames Scott Mitchell
- Re: Stream State and PRIORITY Frames Scott Mitchell
- Re: Stream State and PRIORITY Frames Cory Benfield
- Re: Stream State and PRIORITY Frames Scott Mitchell
- Re: Stream State and PRIORITY Frames Scott Mitchell
- Re: Stream State and PRIORITY Frames Kazuho Oku
- Re: Stream State and PRIORITY Frames Amos Jeffries
- Re: Stream State and PRIORITY Frames Cory Benfield
- Re: Stream State and PRIORITY Frames Patrick McManus
- Re: Stream State and PRIORITY Frames Patrick McManus
- Re: Stream State and PRIORITY Frames Patrick McManus
- Re: Stream State and PRIORITY Frames Patrick McManus
- Re: Stream State and PRIORITY Frames Patrick McManus
- Re: Stream State and PRIORITY Frames Patrick McManus
- Re: Stream State and PRIORITY Frames Cory Benfield
- Re: Stream State and PRIORITY Frames Scott Mitchell
- Re: Stream State and PRIORITY Frames Scott Mitchell
- Re: Stream State and PRIORITY Frames laike9m
- Re: Stream State and PRIORITY Frames Scott Mitchell
- Re: Stream State and PRIORITY Frames Tom Bergan
- Re: Stream State and PRIORITY Frames Scott Mitchell