Re: Proposal: New Frame Size Text (was: Re: Design Issue: Frame Size Items)

[James M Snell <[e-mail]>]

This is now pull request #92: https://github.com/http2/http2-spec/pull/92

On Thu, May 9, 2013 at 2:20 PM, James M Snell <[e-mail]> wrote:
> On Thu, May 9, 2013 at 2:01 PM, William Chan (陈智昌)
> <[e-mail]> wrote:
>> I discussed this with Roberto and Hasan. Here are some thoughts on including
>> header blocks within flow control:
>>
>
> Note that my proposal does not explicitly state that the header blocks
> would be put covered by flow control, only that the current window
> size be used to limit the maximum allowed size of those frames.
> Whether or not headers count towards the flow control limits is a
> separate (but definitely related) question... with that in mind, some
> responses below...
>
>> * In the proposed text, it appears that only the compressed header block is
>> counting towards the window. Is that really what's desirable in order to
>> properly control memory? Processing the header block requires decompressing
>> it, and that's what will be kept in memory buffers until drained by the
>> sink.
>
> Yes, I think it is desirable. The memory management for the
> decompressor state is separate from the notion of how much data the
> receiver is willing to accept over the network. It's likely, for
> instance, that the data in a DATA frame could be compressed as well.
> It would not really make sense to count the decompressed bytes there
> towards the flow control limits.
>
>> * One problem with having headers count toward the per-stream flow control
>> window is that, in non-HTTP semantic layering, the header metadata could be
>> used as a control channel for the stream. Similar to how it would be
>> unfortunate to block connection control frames with the session flow control
>> window, it might also be unfortunate to block stream header/control frames
>> on the per-stream flow control window.
>
> Agreed. That said, however, because header blocks contain user
> provided data, not including these in flow control opens the distinct
> likelihood for abuse. For instance, if an application is not able to
> get it's data passed through in DATA frames because of flow control
> issues, they could simply bypass flow control by packaging the data up
> into headers frames. Whether that's a realistic risk or not, I don't
> know, but I've learned to never underestimate the ingenuity of
> developers who are motivated enough to work around MUST and SHOULD
> level requirements. :-)
>
> - James
>
>> * As previously noted in other threads, it's unfortunate that stream headers
>> aren't counted toward any flow control windows, since that means we may have
>> to use the last-resort option to prevent further memory consumption - stop
>> calling read().
>>
>>
>> On Wed, May 8, 2013 at 9:12 PM, James M Snell <[e-mail]> wrote:
>>>
>>> Suggested replacement text for the current "Frame Size" discussion in
>>> the spec...
>>>
>>> ...
>>>    While the flow control protocol and framing mechanisms defined by
>>> this specification are largely independent of one another, the flow
>>> control WINDOW_SIZE places an upper limit on the total amount of data
>>> an endpoint can send to a peer at any given time. DATA, HEADERS,
>>> HEADERS+PRIORITY and PUSH_PROMISE frame sizes MUST NOT exceed the
>>> current WINDOW_SIZE for the stream or connection and MUST NOT be
>>> greater than 65,535 bytes. The 8 bytes of the frame header are not
>>> counted toward this limit.
>>>
>>>    When a new connection is established, both endpoints are permitted
>>> to begin sending frames prior to the establishment of an initial flow
>>> control WINDOW_SIZE. Accordingly, there is a risk that an endpoint
>>> might initially send frames that are too large for the peer to handle.
>>> To mitigate this risk, it is RECOMMENDED that, until the initial
>>> WINDOW_SIZE is established, the total size of individual
>>> header-bearing frames not exceed the current TCP Maximum Segment Size
>>> (MSS) and that individual DATA frames are no larger than 4096 bytes.
>>> The 8-byte frame header is included in these limits.
>>>
>>> If an endpoint is unable to process a frame due to its size and the
>>> frame specifies any stream identifier field value other than 0x0, the
>>> endpoint MUST respond with a <xref target="StreamErrorHandler">stream
>>> error</xref> using the FRAME_TOO_LARGE error code. If the stream
>>> identifier field value is 0x0, the endpoint MUST send a <xref
>>> target="ConnectionErrorHandler">connection error</xref> using the
>>> FRAME_TOO_LARGE error code.
>>> ...
>>>
>>> - James
>>>
>>>
>>>
>>> On Wed, May 8, 2013 at 1:56 PM, Poul-Henning Kamp <[e-mail]>
>>> wrote:
>>> > In message
>>> > <[e-mail]>
>>> > , James M Snell writes:
>>> >
>>> >>Going back through this, here's a counter proposal:
>>> >>
>>> >>Let's get rid of the 8192 frame size rule and simply say that the
>>> >>maximum size for all DATA, HEADERS, HEADERS+PRIORITY and PUSH_PROMISE
>>> >>frames is either 65,535 or the current flow control WINDOW_SIZE,
>>> >>whichever is less.
>>> >
>>> > Hmm, *now* you're talking...
>>> >
>>> > I like it.
>>> >
>>> > --
>>> > Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
>>> > [e-mail]         | TCP/IP since RFC 956
>>> > FreeBSD committer       | BSD since 4.3-tahoe
>>> > Never attribute to malice what can adequately be explained by
>>> > incompetence.
>>
>>