Re: Proposal: New Frame Size Text (was: Re: Design Issue: Frame Size Items)

William Chan (陈智昌) <willchan@chromium.org> Fri, 10 May 2013 17:41 UTC

Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A1F6621F84B7 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Fri, 10 May 2013 10:41:17 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.676
X-Spam-Level:
X-Spam-Status: No, score=-9.676 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5rn3jVKahewi for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Fri, 10 May 2013 10:41:12 -0700 (PDT)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) by ietfa.amsl.com (Postfix) with ESMTP id 719D821F84AF for <httpbisa-archive-bis2Juki@lists.ietf.org>; Fri, 10 May 2013 10:41:12 -0700 (PDT)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1UarJE-0002uM-JR for ietf-http-wg-dist@listhub.w3.org; Fri, 10 May 2013 17:40:44 +0000
Resent-Date: Fri, 10 May 2013 17:40:44 +0000
Resent-Message-Id: <E1UarJE-0002uM-JR@frink.w3.org>
Received: from maggie.w3.org ([128.30.52.39]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <willchan@google.com>) id 1UarJ3-0002rr-Py for ietf-http-wg@listhub.w3.org; Fri, 10 May 2013 17:40:33 +0000
Received: from mail-qc0-f178.google.com ([209.85.216.178]) by maggie.w3.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.72) (envelope-from <willchan@google.com>) id 1UarIy-0004st-9y for ietf-http-wg@w3.org; Fri, 10 May 2013 17:40:33 +0000
Received: by mail-qc0-f178.google.com with SMTP id l11so2380582qcy.37 for <ietf-http-wg@w3.org>; Fri, 10 May 2013 10:40:02 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=iDJG+DAZhUeiJSN6ankhAxT+rR+mJJZ/2/A7mTdbyB4=; b=Ccx/XxFk602vLsDVN8RushTqnpQn+EJ1c82ZdSG0wAvARjOWW+tP0zn84II+GgLluC IYazdQ2EkD+d9CNb8dZzHyRid9GkJAqQ0+49GGP5WiMumV9ZZZkY2BMIdhNotk2MXNZ0 IwN0/ODvRkZJx5XeqR5lXsjIhvxOxFlg7vFKgDwMZMc7GLQmuaG/9nFExEkV1DBxx0i/ WE7hypbLrnnebugxVqvSqapB05mZUSAmhS/aCshCCx6RKIm4mcakWhk55YHyCAFdVkYQ aHcUXSU9nd0drSztV0i2MT0iGkLY9mrZjiM078tdJCi2Z1NBt7J0kRndQeHl2ByR+pnF dTSQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=iDJG+DAZhUeiJSN6ankhAxT+rR+mJJZ/2/A7mTdbyB4=; b=j5B8aA9TSDhm3YnWBpUWCwp/HHbTFgl4xKdqQHndLkjU8rra8m7Kyvhdpu8xo4d6pV 8Ppc2+UvEA7MizL8ngwIodvv6SiaYlSUcy7xwoTkcsPbgxzPrtpKfnnv4TMa2P3PqwMW F5RuLN5tuMamFxBPiWmi2PT26k8mBoN83IIXk=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :x-gm-message-state; bh=iDJG+DAZhUeiJSN6ankhAxT+rR+mJJZ/2/A7mTdbyB4=; b=fE5U4VP0yObbXSY/AaOkuXrI2ZgY3TK4cxxXGFAnDQTO8R/tFNwEajiBZ4lPwiqIMe 4W9R7YfDFVA/UB4gPauAn9M7pUGLap/6uefxfCNgONBQVKzxHlmI82Ry07y26MUPC7BU 8wtlxD46tv0tbL+/Sdv4FVmGGWGgDVlfmiKZMhSx5y/UQZqt+YAu4TsRv1dVNmdAUSH2 D1ZRiui2e04jnGRopwUlMFdrw92p+nDUZfgOKYNyZ7BQjw4r1cAHQgVHb8qhdhxNKPB+ rbHbnzrEQyYP2g/KUDyReXIWDOiEVQqJeWkPfLACx+ACxnDSz+DZGd45ijOJYFfTcNak G1Sw==
MIME-Version: 1.0
X-Received: by 10.49.128.225 with SMTP id nr1mr14627717qeb.22.1368207602592; Fri, 10 May 2013 10:40:02 -0700 (PDT)
Sender: willchan@google.com
Received: by 10.229.180.4 with HTTP; Fri, 10 May 2013 10:40:01 -0700 (PDT)
In-Reply-To: <CABP7Rbd-VfTFYurZ-JEKjHKOeKvZCKoYLGMXf+0mi-_wbdKYqA@mail.gmail.com>
References: <CABP7RbcfTjN5QFFuGm-P-rQMpAR3FGSC58WCy3qKn+29YCjn+w@mail.gmail.com> <CAA4WUYiwNSzvrY1LF_Sex_82TSDwMbTvYqo7LyKfBAOu0j4pfQ@mail.gmail.com> <CABP7RbdqnH0JK-UaMiaR5rLvZo8txywEcXXSUXa_y95hrLC5yA@mail.gmail.com> <CABP7Rbd-VfTFYurZ-JEKjHKOeKvZCKoYLGMXf+0mi-_wbdKYqA@mail.gmail.com>
Date: Fri, 10 May 2013 14:40:01 -0300
X-Google-Sender-Auth: yTmi7B0aMUuNp6MH8zwyG2HTedc
Message-ID: <CAA4WUYgfu=rcji-bdxNPsE9KCE4T67+vN9b0iojnvycx5R-StA@mail.gmail.com>
From: =?UTF-8?B?V2lsbGlhbSBDaGFuICjpmYjmmbrmmIwp?= <willchan@chromium.org>
To: James M Snell <jasnell@gmail.com>
Cc: Poul-Henning Kamp <phk@phk.freebsd.dk>, Martin Thomson <martin.thomson@gmail.com>, "ietf-http-wg@w3.org" <ietf-http-wg@w3.org>, Roberto Peon <grmocg@gmail.com>, Hasan Khalil <hkhalil@google.com>
Content-Type: multipart/alternative; boundary=047d7b6d9e6e21e33d04dc60a8f1
X-Gm-Message-State: ALoCoQlAVxNwk3phl5l7Yd+6CJCim15F85vwhGYOUxRTrsskcJLofh954s3GEFlUNHzNhLLvaXNvuerRqD0C9ZaAdAZddAbabkUJudTuuROBihSkdvrbdD4PcN4ZEHgdE3AKkEiXjHuRgCcYPSj6bbJuq/aFQtgQb1VnvAmRItEfte/PVwQzpKlfPLyEKg+SObheRjnEnkPO
Received-SPF: pass client-ip=209.85.216.178; envelope-from=willchan@google.com; helo=mail-qc0-f178.google.com
X-W3C-Hub-Spam-Status: No, score=-4.2
X-W3C-Hub-Spam-Report: AWL=-2.073, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-1.315, SPF_PASS=-0.001
X-W3C-Scan-Sig: maggie.w3.org 1UarIy-0004st-9y 5e5fdf0261fe040dcb9f0d64bd01b402
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Proposal: New Frame Size Text (was: Re: Design Issue: Frame Size Items)
Archived-At: <http://www.w3.org/mid/CAA4WUYgfu=rcji-bdxNPsE9KCE4T67+vN9b0iojnvycx5R-StA@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/17919
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

Personally, I'm not sure if there's enough "rough" consensus here yet to
commit anything. Just my two cents there.

On Fri, May 10, 2013 at 2:07 PM, James M Snell <jasnell@gmail.com> wrote:

> This is now pull request #92: https://github.com/http2/http2-spec/pull/92
>
> On Thu, May 9, 2013 at 2:20 PM, James M Snell <jasnell@gmail.com> wrote:
> > On Thu, May 9, 2013 at 2:01 PM, William Chan (陈智昌)
> > <willchan@chromium.org> wrote:
> >> I discussed this with Roberto and Hasan. Here are some thoughts on
> including
> >> header blocks within flow control:
> >>
> >
> > Note that my proposal does not explicitly state that the header blocks
> > would be put covered by flow control, only that the current window
> > size be used to limit the maximum allowed size of those frames.
> > Whether or not headers count towards the flow control limits is a
> > separate (but definitely related) question... with that in mind, some
> > responses below...
>

Thanks for the clarification, I did indeed misunderstand that. What's
supposed to happen when the initial per-stream window is set to be smaller
than what's necessary for a HEADERS+PRIORITY frame? We just never send the
HTTP message then? Or are we going to move forward with the frame
continuation bit?


> >
> >> * In the proposed text, it appears that only the compressed header
> block is
> >> counting towards the window. Is that really what's desirable in order to
> >> properly control memory? Processing the header block requires
> decompressing
> >> it, and that's what will be kept in memory buffers until drained by the
> >> sink.
> >
> > Yes, I think it is desirable. The memory management for the
> > decompressor state is separate from the notion of how much data the
> > receiver is willing to accept over the network. It's likely, for
> > instance, that the data in a DATA frame could be compressed as well.
> > It would not really make sense to count the decompressed bytes there
> > towards the flow control limits.
>

I'm trying to figure out if I'm misunderstanding you or you're
misunderstanding the concern. Let me give a concrete example, an HTTP/2
proxy. When forwarding DATA frames, it doesn't have to decompress unless it
wants to inspect the payload for some reason. It just forwards onward to
the sink, unless the sink is not ready, in which case it buffers the frame
(agnostic of whether or not it's compressed). On the contrary, for a
HEADERS+PRIORITY frame, and IIUC, it *has* to decompress the header block
to update the compression context appropriately without introducing HoL
blocking with other things using the compression context. And if the sink
is not ready to accept the headers, and then it most likely has to keep the
decompressed header block around. That's my understanding at least, what
did I miss?


> >
> >> * One problem with having headers count toward the per-stream flow
> control
> >> window is that, in non-HTTP semantic layering, the header metadata
> could be
> >> used as a control channel for the stream. Similar to how it would be
> >> unfortunate to block connection control frames with the session flow
> control
> >> window, it might also be unfortunate to block stream header/control
> frames
> >> on the per-stream flow control window.
> >
> > Agreed. That said, however, because header blocks contain user
> > provided data, not including these in flow control opens the distinct
> > likelihood for abuse. For instance, if an application is not able to
> > get it's data passed through in DATA frames because of flow control
> > issues, they could simply bypass flow control by packaging the data up
> > into headers frames. Whether that's a realistic risk or not, I don't
> > know, but I've learned to never underestimate the ingenuity of
> > developers who are motivated enough to work around MUST and SHOULD
> > level requirements. :-)
>
>
> > - James
> >
> >> * As previously noted in other threads, it's unfortunate that stream
> headers
> >> aren't counted toward any flow control windows, since that means we may
> have
> >> to use the last-resort option to prevent further memory consumption -
> stop
> >> calling read().
> >>
> >>
> >> On Wed, May 8, 2013 at 9:12 PM, James M Snell <jasnell@gmail.com>
> wrote:
> >>>
> >>> Suggested replacement text for the current "Frame Size" discussion in
> >>> the spec...
> >>>
> >>> ...
> >>>    While the flow control protocol and framing mechanisms defined by
> >>> this specification are largely independent of one another, the flow
> >>> control WINDOW_SIZE places an upper limit on the total amount of data
> >>> an endpoint can send to a peer at any given time. DATA, HEADERS,
> >>> HEADERS+PRIORITY and PUSH_PROMISE frame sizes MUST NOT exceed the
> >>> current WINDOW_SIZE for the stream or connection and MUST NOT be
> >>> greater than 65,535 bytes. The 8 bytes of the frame header are not
> >>> counted toward this limit.
> >>>
> >>>    When a new connection is established, both endpoints are permitted
> >>> to begin sending frames prior to the establishment of an initial flow
> >>> control WINDOW_SIZE. Accordingly, there is a risk that an endpoint
> >>> might initially send frames that are too large for the peer to handle.
> >>> To mitigate this risk, it is RECOMMENDED that, until the initial
> >>> WINDOW_SIZE is established, the total size of individual
> >>> header-bearing frames not exceed the current TCP Maximum Segment Size
> >>> (MSS) and that individual DATA frames are no larger than 4096 bytes.
> >>> The 8-byte frame header is included in these limits.
> >>>
> >>> If an endpoint is unable to process a frame due to its size and the
> >>> frame specifies any stream identifier field value other than 0x0, the
> >>> endpoint MUST respond with a <xref target="StreamErrorHandler">stream
> >>> error</xref> using the FRAME_TOO_LARGE error code. If the stream
> >>> identifier field value is 0x0, the endpoint MUST send a <xref
> >>> target="ConnectionErrorHandler">connection error</xref> using the
> >>> FRAME_TOO_LARGE error code.
> >>> ...
> >>>
> >>> - James
> >>>
> >>>
> >>>
> >>> On Wed, May 8, 2013 at 1:56 PM, Poul-Henning Kamp <phk@phk.freebsd.dk>
> >>> wrote:
> >>> > In message
> >>> > <CABP7Rbc8rs4-ktyGKwVxVC4MztcvYtARqBDoyEBYujfcpo4YDw@mail.gmail.com>
> >>> > , James M Snell writes:
> >>> >
> >>> >>Going back through this, here's a counter proposal:
> >>> >>
> >>> >>Let's get rid of the 8192 frame size rule and simply say that the
> >>> >>maximum size for all DATA, HEADERS, HEADERS+PRIORITY and PUSH_PROMISE
> >>> >>frames is either 65,535 or the current flow control WINDOW_SIZE,
> >>> >>whichever is less.
> >>> >
> >>> > Hmm, *now* you're talking...
> >>> >
> >>> > I like it.
> >>> >
> >>> > --
> >>> > Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> >>> > phk@FreeBSD.ORG         | TCP/IP since RFC 956
> >>> > FreeBSD committer       | BSD since 4.3-tahoe
> >>> > Never attribute to malice what can adequately be explained by
> >>> > incompetence.
> >>
> >>
>