Re: Proposal: New Frame Size Text (was: Re: Design Issue: Frame Size Items)

James M Snell <jasnell@gmail.com> Fri, 10 May 2013 17:09 UTC

Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 10B1F21F8D7A for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Fri, 10 May 2013 10:09:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.422
X-Spam-Level:
X-Spam-Status: No, score=-10.422 tagged_above=-999 required=5 tests=[AWL=-0.123, BAYES_00=-2.599, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id t6C5uB4Jebh9 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Fri, 10 May 2013 10:09:23 -0700 (PDT)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) by ietfa.amsl.com (Postfix) with ESMTP id 997BC21F8972 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Fri, 10 May 2013 10:09:23 -0700 (PDT)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1UaqoC-00075J-A6 for ietf-http-wg-dist@listhub.w3.org; Fri, 10 May 2013 17:08:40 +0000
Resent-Date: Fri, 10 May 2013 17:08:40 +0000
Resent-Message-Id: <E1UaqoC-00075J-A6@frink.w3.org>
Received: from maggie.w3.org ([128.30.52.39]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <jasnell@gmail.com>) id 1Uaqo1-00074Z-To for ietf-http-wg@listhub.w3.org; Fri, 10 May 2013 17:08:29 +0000
Received: from mail-oa0-f41.google.com ([209.85.219.41]) by maggie.w3.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.72) (envelope-from <jasnell@gmail.com>) id 1Uaqo0-0003o7-O7 for ietf-http-wg@w3.org; Fri, 10 May 2013 17:08:29 +0000
Received: by mail-oa0-f41.google.com with SMTP id n9so926876oag.0 for <ietf-http-wg@w3.org>; Fri, 10 May 2013 10:08:02 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:in-reply-to:references:from:date:message-id :subject:to:cc:content-type:content-transfer-encoding; bh=t5sIqVAP/q27kQj9dq3j7kA5ltx3fIVP0ycPUfWMorU=; b=0EmkCp7najdwyU15SiPf4MohxQnM2SQTFkPKrV3NtZlR/iN5hYmE2WHLHEIvOLpxmF vSAkJgrhL6EDU3TusdKNur3olUAYMzwBjpcKsmIY0sJn/n09KPlOIgwD82QLTRqb859S jEXPXwmW6sSJCDUe8H3SfuQa66RCB4rQ85cIhy/DUB+uMYS1AGOkSAC/rTUIRU38yqIX F3h/8SZXLOCCfSO8bRUiK86vxjuh8Xx13Op+vZnFpq3D9DEO3ma5Vw3yAQFbE//LcBp9 NtTw9NKn608HDx6+U7TYb9WoSldB5zyiB8FqPgR/sF/nbszPWqWs5OJa87zos7oAhHL8 W7CQ==
X-Received: by 10.60.55.231 with SMTP id v7mr7301423oep.135.1368205682824; Fri, 10 May 2013 10:08:02 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.60.3.137 with HTTP; Fri, 10 May 2013 10:07:42 -0700 (PDT)
In-Reply-To: <CABP7RbdqnH0JK-UaMiaR5rLvZo8txywEcXXSUXa_y95hrLC5yA@mail.gmail.com>
References: <CABP7RbcfTjN5QFFuGm-P-rQMpAR3FGSC58WCy3qKn+29YCjn+w@mail.gmail.com> <CAA4WUYiwNSzvrY1LF_Sex_82TSDwMbTvYqo7LyKfBAOu0j4pfQ@mail.gmail.com> <CABP7RbdqnH0JK-UaMiaR5rLvZo8txywEcXXSUXa_y95hrLC5yA@mail.gmail.com>
From: James M Snell <jasnell@gmail.com>
Date: Fri, 10 May 2013 10:07:42 -0700
Message-ID: <CABP7Rbd-VfTFYurZ-JEKjHKOeKvZCKoYLGMXf+0mi-_wbdKYqA@mail.gmail.com>
To: =?UTF-8?B?V2lsbGlhbSBDaGFuICjpmYjmmbrmmIwp?= <willchan@chromium.org>
Cc: Poul-Henning Kamp <phk@phk.freebsd.dk>, Martin Thomson <martin.thomson@gmail.com>, "ietf-http-wg@w3.org" <ietf-http-wg@w3.org>, Roberto Peon <grmocg@gmail.com>, Hasan Khalil <hkhalil@google.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass client-ip=209.85.219.41; envelope-from=jasnell@gmail.com; helo=mail-oa0-f41.google.com
X-W3C-Hub-Spam-Status: No, score=-3.5
X-W3C-Hub-Spam-Report: AWL=-2.702, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001
X-W3C-Scan-Sig: maggie.w3.org 1Uaqo0-0003o7-O7 bd20960242ed33d7006be0c61bf89590
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Proposal: New Frame Size Text (was: Re: Design Issue: Frame Size Items)
Archived-At: <http://www.w3.org/mid/CABP7Rbd-VfTFYurZ-JEKjHKOeKvZCKoYLGMXf+0mi-_wbdKYqA@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/17915
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

This is now pull request #92: https://github.com/http2/http2-spec/pull/92

On Thu, May 9, 2013 at 2:20 PM, James M Snell <jasnell@gmail.com> wrote:
> On Thu, May 9, 2013 at 2:01 PM, William Chan (陈智昌)
> <willchan@chromium.org> wrote:
>> I discussed this with Roberto and Hasan. Here are some thoughts on including
>> header blocks within flow control:
>>
>
> Note that my proposal does not explicitly state that the header blocks
> would be put covered by flow control, only that the current window
> size be used to limit the maximum allowed size of those frames.
> Whether or not headers count towards the flow control limits is a
> separate (but definitely related) question... with that in mind, some
> responses below...
>
>> * In the proposed text, it appears that only the compressed header block is
>> counting towards the window. Is that really what's desirable in order to
>> properly control memory? Processing the header block requires decompressing
>> it, and that's what will be kept in memory buffers until drained by the
>> sink.
>
> Yes, I think it is desirable. The memory management for the
> decompressor state is separate from the notion of how much data the
> receiver is willing to accept over the network. It's likely, for
> instance, that the data in a DATA frame could be compressed as well.
> It would not really make sense to count the decompressed bytes there
> towards the flow control limits.
>
>> * One problem with having headers count toward the per-stream flow control
>> window is that, in non-HTTP semantic layering, the header metadata could be
>> used as a control channel for the stream. Similar to how it would be
>> unfortunate to block connection control frames with the session flow control
>> window, it might also be unfortunate to block stream header/control frames
>> on the per-stream flow control window.
>
> Agreed. That said, however, because header blocks contain user
> provided data, not including these in flow control opens the distinct
> likelihood for abuse. For instance, if an application is not able to
> get it's data passed through in DATA frames because of flow control
> issues, they could simply bypass flow control by packaging the data up
> into headers frames. Whether that's a realistic risk or not, I don't
> know, but I've learned to never underestimate the ingenuity of
> developers who are motivated enough to work around MUST and SHOULD
> level requirements. :-)
>
> - James
>
>> * As previously noted in other threads, it's unfortunate that stream headers
>> aren't counted toward any flow control windows, since that means we may have
>> to use the last-resort option to prevent further memory consumption - stop
>> calling read().
>>
>>
>> On Wed, May 8, 2013 at 9:12 PM, James M Snell <jasnell@gmail.com> wrote:
>>>
>>> Suggested replacement text for the current "Frame Size" discussion in
>>> the spec...
>>>
>>> ...
>>>    While the flow control protocol and framing mechanisms defined by
>>> this specification are largely independent of one another, the flow
>>> control WINDOW_SIZE places an upper limit on the total amount of data
>>> an endpoint can send to a peer at any given time. DATA, HEADERS,
>>> HEADERS+PRIORITY and PUSH_PROMISE frame sizes MUST NOT exceed the
>>> current WINDOW_SIZE for the stream or connection and MUST NOT be
>>> greater than 65,535 bytes. The 8 bytes of the frame header are not
>>> counted toward this limit.
>>>
>>>    When a new connection is established, both endpoints are permitted
>>> to begin sending frames prior to the establishment of an initial flow
>>> control WINDOW_SIZE. Accordingly, there is a risk that an endpoint
>>> might initially send frames that are too large for the peer to handle.
>>> To mitigate this risk, it is RECOMMENDED that, until the initial
>>> WINDOW_SIZE is established, the total size of individual
>>> header-bearing frames not exceed the current TCP Maximum Segment Size
>>> (MSS) and that individual DATA frames are no larger than 4096 bytes.
>>> The 8-byte frame header is included in these limits.
>>>
>>> If an endpoint is unable to process a frame due to its size and the
>>> frame specifies any stream identifier field value other than 0x0, the
>>> endpoint MUST respond with a <xref target="StreamErrorHandler">stream
>>> error</xref> using the FRAME_TOO_LARGE error code. If the stream
>>> identifier field value is 0x0, the endpoint MUST send a <xref
>>> target="ConnectionErrorHandler">connection error</xref> using the
>>> FRAME_TOO_LARGE error code.
>>> ...
>>>
>>> - James
>>>
>>>
>>>
>>> On Wed, May 8, 2013 at 1:56 PM, Poul-Henning Kamp <phk@phk.freebsd.dk>
>>> wrote:
>>> > In message
>>> > <CABP7Rbc8rs4-ktyGKwVxVC4MztcvYtARqBDoyEBYujfcpo4YDw@mail.gmail.com>
>>> > , James M Snell writes:
>>> >
>>> >>Going back through this, here's a counter proposal:
>>> >>
>>> >>Let's get rid of the 8192 frame size rule and simply say that the
>>> >>maximum size for all DATA, HEADERS, HEADERS+PRIORITY and PUSH_PROMISE
>>> >>frames is either 65,535 or the current flow control WINDOW_SIZE,
>>> >>whichever is less.
>>> >
>>> > Hmm, *now* you're talking...
>>> >
>>> > I like it.
>>> >
>>> > --
>>> > Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
>>> > phk@FreeBSD.ORG         | TCP/IP since RFC 956
>>> > FreeBSD committer       | BSD since 4.3-tahoe
>>> > Never attribute to malice what can adequately be explained by
>>> > incompetence.
>>
>>