Re: Stream State and PRIORITY Frames

[Cory Benfield <[e-mail]>]

> On 19 Jan 2017, at 09:00, Amos Jeffries <[e-mail]> wrote:
> 
> IIRC the intention behind the closing behaviour for idle streams was to
> ensure that we could optimize away the need to maintain a list or array
> of 2^31 state entries.
> 
> Omitting PRIORITY from the MAX_STREAMS voids that benefit.

No it doesn’t.

Firstly, let’s note that the PRIORITY frame is defined not to change stream state. Receiving a PRIORITY frame on stream N leaves it in the idle state. A server implementation can thus treat the receipt of a PRIORITY frame on any stream that has a higher ID than the last stream on which it received a HEADERS frame as having the *exact* same effect on state retention as receiving a PING frame. It doesn’t *require* the insertion of any stream state data.

Specific implementations may require the insertion of stream state data if priority information is stored on the same structure as stream state information, but that is certainly not required. For example, receiving a bunch of PRIORITY frames does not cause the Python HTTP/2 implementation to allocate any more state information than receiving a bunch of PING frames.

> It is causing state to be allocated on the server and on every
> intermediary along the way which accepts it.
> 
> Consider the effects of 2^31 PRIORITY frames being sent with different
> IDs before the first HEADERS is used.
> Notice how even this one case is markedly worse than sending just one
> HEADERS with stream ID == 2^31 to waste server sockets.

This problem is orthogonal to the one we’re discussing.

So far I haven’t seen a proposal to say that PRIORITY frames should transition a stream out of the idle state. If we continue to leave RFC 7540 saying that PRIORITY frames leave streams in the idle state, then definitionally they are excluded from the restrictions on MAX_CONCURRENT_STREAMS. So if you are allocating stream state for idle streams, then you are open to this DoS vector, but none of the proposals here are trying to address it.

Additionally, I should note that the unbounded insertion of priority information into priority trees was discussed as an attack vector in this paper: https://www.imperva.com/docs/Imperva_HII_HTTP2.pdf. The paper was published in August of last year, and several implementations took steps to reduce their vulnerability to it (the Python priority implementation even filed a CVE, CVE-2016-6580).

I absolutely support adding an erratum to indicate that implementations should resist unbounded insertion of PRIORITY information. But unless you’re willing to say that idle streams should be counted against MAX_CONCURRENT_STREAMS (which is a tricky thing to state), then the only way to resolve this problem at the spec level is to force PRIORITY frames to make state transitions, further complicating the state diagram for streams.

> What exactly does it mean for them to set PRIORITY on a non-existent stream?

It means exactly what RFC 7540 says it means. Quoting from Section 5.3.4 of RFC 7540:

> Similarly, streams that are in the "idle" state can be assigned priority or become a parent of other streams. This allows for the creation of a grouping node in the dependency tree, which enables more flexible expressions of priority. Idle streams begin with a default priority (Section 5.3.5).

While we’re here, right below that section is:

> The retention of priority information for streams that are not counted toward the limit set by SETTINGS_MAX_CONCURRENT_STREAMS could create a large state burden for an endpoint. Therefore, the amount of prioritization state that is retained MAY be limited.

To this end, the Python priority implementation allows a user-configured maximum amount of data retention for priority information. This priority information is held separately from stream state information because they are entirely orthogonal concerns: in fact, the bits of code that manage priority information and those that manage stream state information are in entirely separate installable packages with no dependency relationship to each other.

To my eye, there is a clear intent to allow setting of priority information on streams that are in the idle state, without that affecting the stream itself.

Cory