Re: Design Issue: Max Concurrent Streams Limit and Unidirectional Streams

William Chan (陈智昌) <willchan@chromium.org> Wed, 01 May 2013 19:59 UTC

Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 60F7C21F99A4 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Wed, 1 May 2013 12:59:43 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.676
X-Spam-Level:
X-Spam-Status: No, score=-9.676 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PhFUcTzY-8yA for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Wed, 1 May 2013 12:59:38 -0700 (PDT)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) by ietfa.amsl.com (Postfix) with ESMTP id B89C721F99B3 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Wed, 1 May 2013 12:59:37 -0700 (PDT)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1UXdBS-0007H2-T6 for ietf-http-wg-dist@listhub.w3.org; Wed, 01 May 2013 19:59:22 +0000
Resent-Date: Wed, 01 May 2013 19:59:22 +0000
Resent-Message-Id: <E1UXdBS-0007H2-T6@frink.w3.org>
Received: from lisa.w3.org ([128.30.52.41]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <willchan@google.com>) id 1UXdBI-0007Dk-Ef for ietf-http-wg@listhub.w3.org; Wed, 01 May 2013 19:59:12 +0000
Received: from mail-qe0-f41.google.com ([209.85.128.41]) by lisa.w3.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.72) (envelope-from <willchan@google.com>) id 1UXdBG-0005bl-Ra for ietf-http-wg@w3.org; Wed, 01 May 2013 19:59:12 +0000
Received: by mail-qe0-f41.google.com with SMTP id b10so1105291qen.28 for <ietf-http-wg@w3.org>; Wed, 01 May 2013 12:58:45 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=Li0VvicWz0CiT3sCZdmN7dT4OnU7WKrwudIcSKpwn50=; b=D5DWfnAkv0YDjV6mYJU91gBs3hgDfuVwFm9tEXgKe2vo9Q/82R+IoLWnCEtlJfNedN DzKKJZkINEGdiIezJpnD5Voe7pYs3sZPio8DOhb4DUDe1/p/X+M8BPJoZtq1Kz9QBvrt fHoClH88srXCm7NLJm6ZxGvdoHbuPrFq3d+Qh9ll/Fh7Pd3kh3DkoWzntUfHREQJqP+g oBRKb3f1VYUcdbJoJIaDSj6MqDQYNn/YNEQpIY4d8DfVr+hD0Zl/abvs+thhZ/OMDeW/ nKHKYs1Q5KQB0u42KYAyq911Z5NFqglvFt2cxRoKEn3yOL0q4vN6iELuLBry2aPhobzS lchw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=Li0VvicWz0CiT3sCZdmN7dT4OnU7WKrwudIcSKpwn50=; b=kYP7+Iy9Yg7BAhRYtlRRfROLJ0eQUZ+aIjpXJkl9nGCfVlmPBa/6oa4Oy9sQgd2V5Y sMhESXQzEmLO8WoV/S55I9YLHvyfhl+Y+fwpTOQy4lrweB1luSYSOlg9G2gW0lPxhqJ8 0/pLqEyFz8X1/dvy/Yjx+NDIBKKDPMvsGPXh0=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :x-gm-message-state; bh=Li0VvicWz0CiT3sCZdmN7dT4OnU7WKrwudIcSKpwn50=; b=FjWAmq/fLYiCpvP01HNTmTGNJpnVLaxWbbsHOL+wwxfeGnzGd7pPw/TqRf7aNWsFmQ ZwrCa37ru27ZfEIFpIQFgQ4d4Acr5p5B1Hj8tBgwKtndQVYmx2/OBVcHACb62n5EDK1v 5Vcjlx3tMaU78tddQpfnK2d+y5KjkVv82KwYDW/0a7kBxiCzGaEV1MnJqA1mqfqmvwtL BMT4XRljgYFDIrvIdjgrHrVqHwZaI6B6mb7p7EHwoBrp5fP8bmUOirKDVwTjdGypamQN 6ExFjvFH/RQFzqmvqeI2vnHIDRkd7pUlno92JmQqy6wjWghROTq5NC1XEhn1+6p5wZA5 GKTA==
MIME-Version: 1.0
X-Received: by 10.224.57.82 with SMTP id b18mr5109518qah.36.1367438325073; Wed, 01 May 2013 12:58:45 -0700 (PDT)
Sender: willchan@google.com
Received: by 10.229.180.4 with HTTP; Wed, 1 May 2013 12:58:44 -0700 (PDT)
In-Reply-To: <CAP+FsNc5QwETi8XR2+r9FgTSxLhA2MHH3cXu0Hq9yjcRKqOS5g@mail.gmail.com>
References: <CABP7RbdBe-Xkx+CMvpN=_oNAqm6SyLyL+XNHRUKSqn8mjSDw1Q@mail.gmail.com> <CAA4WUYgCiyWerT0tUUVKcbNPqdTGuXHd_MG59DjcUsEWst5t7g@mail.gmail.com> <CABkgnnVdU=cZ53Bqg5Un=E80NMpcgYO37DVmwUFW0O-i7SNf8w@mail.gmail.com> <CAA4WUYhz64FsEGgGhx91RfWwuPPxWdAkesOV-bmqWVWE7ZxdjA@mail.gmail.com> <CABP7RbcKQkn1o4WZscwNmSmm6YzqE_TKxPr4jnozNdaVqpZ7=A@mail.gmail.com> <CAA4WUYhF6rAZoYEaz4aJO6xawaJxzxGt=Bkg4H9eBOP-LBSRmQ@mail.gmail.com> <CAP+FsNezQzxdZEJY_2_0h_TR2pBbVsGyGBhQhKcm-65pt6S8rQ@mail.gmail.com> <CABP7RbevS8M0q9OxzPncqY_gE34q5-ymdg2hOX2SQgSUNkhzsw@mail.gmail.com> <CAA4WUYjAbuUqz9RdO+-p3a4EsyuS=Gv0rS-U-Vh+ZCjtDjFy6w@mail.gmail.com> <CAP+FsNec2LLZMjtGhSX-1q8qg66WtBoM5K0yMrs5m4VKXb5OVg@mail.gmail.com> <CAA4WUYgAT64jj=Am06MsA02A+eAcDrVbbgb4opO37bnMkWTPfg@mail.gmail.com> <CABP7Rbdgz=kRZPfjHK5UUfieq8uz=ToQZjFt1-+s9scj1CogmA@mail.gmail.com> <CAA4WUYjSjFKSdbj=QBLn0T4ufhzF1hUY=O=Qa2dfnkTzMXF0bg@mail.gmail.com> <CAP+FsNc5QwETi8XR2+r9FgTSxLhA2MHH3cXu0Hq9yjcRKqOS5g@mail.gmail.com>
Date: Wed, 1 May 2013 16:58:44 -0300
X-Google-Sender-Auth: tnduOOTUoKeG0m94caEX_FunhUY
Message-ID: <CAA4WUYiOm8WwK4Af5n9YDXPYYX3nUN1f2xgzAgGHMkKrnyHvTA@mail.gmail.com>
From: =?UTF-8?B?V2lsbGlhbSBDaGFuICjpmYjmmbrmmIwp?= <willchan@chromium.org>
To: Roberto Peon <grmocg@gmail.com>
Cc: James M Snell <jasnell@gmail.com>, HTTP Working Group <ietf-http-wg@w3.org>, Martin Thomson <martin.thomson@gmail.com>
Content-Type: multipart/alternative; boundary=089e01538d549e55a904dbad8bbd
X-Gm-Message-State: ALoCoQmdGvCyMWKy6Hf09lRxu962ne+Y8+EaQFxEz2JNKNMqmoo2BbdF4uYB4yFkr4bmLc7iVI0uunsHDnVDbSXG3K6qM10tiaU0Y0/ep1GZjZs8YRRK/yd75Ah8Paz92ZyD5DZ2C+SyfVQbzADWOcWqefY+SqVspfAxoekGlx0SJHCqmcIwhNLsQqP+PAo+jJzUNcO9ukOU
Received-SPF: pass client-ip=209.85.128.41; envelope-from=willchan@google.com; helo=mail-qe0-f41.google.com
X-W3C-Hub-Spam-Status: No, score=-4.8
X-W3C-Hub-Spam-Report: AWL=-1.401, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-2.57, SPF_PASS=-0.001
X-W3C-Scan-Sig: lisa.w3.org 1UXdBG-0005bl-Ra ab33f633ff44a72e4b1a8c34f8c6e86c
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Design Issue: Max Concurrent Streams Limit and Unidirectional Streams
Archived-At: <http://www.w3.org/mid/CAA4WUYiOm8WwK4Af5n9YDXPYYX3nUN1f2xgzAgGHMkKrnyHvTA@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/17770
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

I agree with that, although there's no current WebSocket API use case for
server initiated bidirectional streams. I could imagine it in the future
though.


On Wed, May 1, 2013 at 4:55 PM, Roberto Peon <grmocg@gmail.com>; wrote:

> I still want to be able to support the WS API over HTTP/2. It would be
> tragic to have N+1 connections instead of 1 when 1 works better anyway...
>
> -=R
>
>
> On Wed, May 1, 2013 at 10:46 AM, William Chan (陈智昌) <willchan@chromium.org
> > wrote:
>
>> The only benefit to that is supporting non-HTTP/2 application layering
>> semantics, which is intended not to change from HTTP/1.X. So there's
>> currently no use to allow the server to initiate streams with the
>> client=>server direction open.
>>
>> I consider the current trend of our discussions to tend towards
>> eliminating complexity and targeting for HTTP/2 application layering
>> semantics. I think if we have another use case come up that would require
>> supporting server initiated bidirectional streams, I think at that point
>> it'd be worthwhile to revisit how we do this.
>>
>> I'd like to hear from others if they disagree with my assessment of how
>> most people feel so far. FWIW, I personally would like us to support server
>> initiated bidirectional streams.
>>
>>
>> On Wed, May 1, 2013 at 2:26 PM, James M Snell <jasnell@gmail.com>; wrote:
>>
>>> Why not just bring the UNIDIRECTIONAL flag back as a PUSH_PROMISE
>>> frame-specific flag? If a PUSH_PROMISE frame has the unidirectional
>>> flag set, the stream is automatically half-closed in the return
>>> direction. If the flag is unset, the promised stream remains half-open
>>> until the client half-closes or a rst_stream is sent.
>>>
>>> On Mon, Apr 29, 2013 at 2:44 PM, William Chan (陈智昌)
>>> <willchan@chromium.org>; wrote:
>>> > Remember we originally *had* a flag for UNIDIRECTIONAL, which we
>>> removed
>>> > because it was redundant in the traditional HTTP use cases.
>>> >
>>> >
>>> > On Mon, Apr 29, 2013 at 6:39 PM, Roberto Peon <grmocg@gmail.com>;
>>> wrote:
>>> >>
>>> >> At worst, we burn a flag which states it is half-closed or
>>> unidirectional,
>>> >> or provide some other information which identifies the IANA port
>>> number for
>>> >> the overlayed protocol or something.
>>> >> Anyway, *shrug*.
>>> >> -=R
>>> >>
>>> >>
>>> >> On Mon, Apr 29, 2013 at 2:32 PM, William Chan (陈智昌)
>>> >> <willchan@chromium.org>; wrote:
>>> >>>
>>> >>> On Mon, Apr 29, 2013 at 6:17 PM, James M Snell <jasnell@gmail.com>;
>>> wrote:
>>> >>>>
>>> >>>> +1 on this.  I like this approach.
>>> >>>>
>>> >>>> On Apr 29, 2013 2:15 PM, "Roberto Peon" <grmocg@gmail.com>; wrote:
>>> >>>>>
>>> >>>>> I had thought to provide no explicit limit for PUSH_PROMISE, just
>>> as
>>> >>>>> there is no limit to the size of a webpage, or the number of links
>>> upon it.
>>> >>>>> The memory requirements for PUSH are similar or the same (push
>>> should
>>> >>>>> consume a single additional bit of overhead per url, when one
>>> considers that
>>> >>>>> the URL should be parsed, enqueued, etc.).
>>> >>>>> If the browser isn't done efficiently, or, the server is for some
>>> >>>>> unknown reason being stupid and attempting to DoS the browser with
>>> many
>>> >>>>> resources that it will never use, then the client sends RST_STREAM
>>> for the
>>> >>>>> ones it doesn't want, and makes a request on its own. all tidy.
>>> >>>
>>> >>>
>>> >>> I don't feel too strongly here. I do feel like this is more of an
>>> edge
>>> >>> case, possibly important for forward proxies (or reverse proxies
>>> speaking to
>>> >>> backends over a multiplexed channel like HTTP/2). It doesn't really
>>> matter
>>> >>> for my browser, so unless servers chime in and say they'd prefer a
>>> limit,
>>> >>> I'm fine with this.
>>> >>>
>>> >>>>>
>>> >>>>> As for PUSH'd streams, the easiest solution is likely to assume
>>> that
>>> >>>>> the stream starts out in a half-closed state.
>>> >>>
>>> >>>
>>> >>> I looked into our earlier email threads and indeed this is what we
>>> agreed
>>> >>> on (
>>> http://lists.w3.org/Archives/Public/ietf-http-wg/2013JanMar/1106.html).
>>> >>> I voiced some mild objection since if you view the HTTP/2 framing
>>> layer as a
>>> >>> transport for another application protocol, then bidirectional server
>>> >>> initiated streams might be nice. But in absence of any such
>>> protocol, this
>>> >>> is a nice simplification.
>>> >>>
>>> >>>>>
>>> >>>>> -=R
>>> >>>>>
>>> >>>>>
>>> >>>>> On Mon, Apr 29, 2013 at 12:33 PM, William Chan (陈智昌)
>>> >>>>> <willchan@chromium.org>; wrote:
>>> >>>>>>
>>> >>>>>> On Mon, Apr 29, 2013 at 3:46 PM, James M Snell <jasnell@gmail.com
>>> >
>>> >>>>>> wrote:
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> On Apr 29, 2013 11:36 AM, "William Chan (陈智昌)"
>>> >>>>>>> <willchan@chromium.org>; wrote:
>>> >>>>>>> >
>>> >>>>>>> [snip]
>>> >>>>>>>
>>> >>>>>>> >
>>> >>>>>>> >
>>> >>>>>>> > Oops, forgot about that. See, the issue with that is now we've
>>> made
>>> >>>>>>> > PUSH_PROMISE as potentially expensive as a HEADERS frame,
>>> since it does more
>>> >>>>>>> > than just simple stream id allocation. I guess it's not really
>>> a huge issue,
>>> >>>>>>> > since if it's used correctly (in the matter you described),
>>> then it
>>> >>>>>>> > shouldn't be too expensive. If clients attempt to abuse it,
>>> then servers
>>> >>>>>>> > should probably treat it in a similar manner as they treat
>>> people trying to
>>> >>>>>>> > abuse header compression in all other frames with the header
>>> block, and kill
>>> >>>>>>> > the connection accordingly.
>>> >>>>>>> >
>>> >>>>>>>
>>> >>>>>>> Not just "potentially" as expensive..   As soon as we get a push
>>> >>>>>>> promise we need to allocate state and hold onto it for an
>>> indefinite period
>>> >>>>>>> of time. We do not yet know exactly when that compression
>>> context can be let
>>> >>>>>>> go because it has not yet been bound to stream state.  Do push
>>> streams all
>>> >>>>>>> share the same compression state? Do those share the same
>>> compression state
>>> >>>>>>> as the originating stream? The answers might be obvious but they
>>> haven't yet
>>> >>>>>>> been written down.
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> I guess I don't see per-stream state as being that expensive.
>>> >>>>>> Compression contexts are a fixed state on a per-connection basis,
>>> meaning
>>> >>>>>> that additional streams don't add to that state. The main cost,
>>> as I see it,
>>> >>>>>> is the decompressed headers. I said potentially since that
>>> basically only
>>> >>>>>> means the URL (unless there are other headers important for
>>> caching due to
>>> >>>>>> Vary), and additional headers can come in the HEADERS frame. Also,
>>> >>>>>> PUSH_PROMISE doesn't require allocating other state, like
>>> backend/DB
>>> >>>>>> connections, if you only want to be able to handle
>>> (#MAX_CONCURRENT_STREAMs)
>>> >>>>>> of those backend connections in parallel.
>>> >>>>>>
>>> >>>>>> If they're not specified, then we should specify it, but I've
>>> always
>>> >>>>>> understood the header compression contexts to be directional and
>>> apply to
>>> >>>>>> all frames sending headers in a direction. Therefore there should
>>> be two
>>> >>>>>> compression contexts in a connection, one for header blocks being
>>> sent and
>>> >>>>>> one for header blocks being received. If this is controversial,
>>> let's fork a
>>> >>>>>> thread and discuss it.
>>> >>>>>>
>>> >>>>>>>
>>> >>>>>>> >>
>>> >>>>>>> >>
>>> >>>>>>> >> > As far as the potential problem above, the root problem is
>>> that
>>> >>>>>>> >> > when you
>>> >>>>>>> >> > have limits you can have hangs. We see this all the time
>>> today
>>> >>>>>>> >> > with browsers
>>> >>>>>>> >> > (it's only reason people do domain sharding so they can
>>> bypass
>>> >>>>>>> >> > limits). I'm
>>> >>>>>>> >> > not sure I see the value of introducing the new proposed
>>> limits.
>>> >>>>>>> >> > They don't
>>> >>>>>>> >> > solve the hangs, and I don't think the granularity
>>> addresses any
>>> >>>>>>> >> > of the
>>> >>>>>>> >> > costs in a finer grained manner. I'd like to hear
>>> clarification
>>> >>>>>>> >> > on what
>>> >>>>>>> >> > costs the new proposed limits will address.
>>> >>>>>>> >>
>>> >>>>>>> >> I don't believe that the proposal improves the situation
>>> enough
>>> >>>>>>> >> (or at
>>> >>>>>>> >> all) to justify the additional complexity.  That's something
>>> that
>>> >>>>>>> >> you
>>> >>>>>>> >> need to assess for yourself.  This proposal provides more
>>> granular
>>> >>>>>>> >> control, but it doesn't address the core problem, which is
>>> that
>>> >>>>>>> >> you
>>> >>>>>>> >> and I can only observe each other actions after some delay,
>>> which
>>> >>>>>>> >> means that we can't coordinate those actions perfectly.  Nor
>>> can
>>> >>>>>>> >> be
>>> >>>>>>> >> build a perfect model of the other upon which to observe and
>>> act
>>> >>>>>>> >> upon.
>>> >>>>>>> >>  The usual protocol issue.
>>> >>>>>>> >
>>> >>>>>>> >
>>> >>>>>>> > OK then. My proposal is to add a new limit for PUSH_PROMISE
>>> frames
>>> >>>>>>> > though, separately from the MAX_CONCURRENT_STREAMS limit,
>>> since PUSH_PROMISE
>>> >>>>>>> > exists as a promise to create a stream, explicitly so we don't
>>> have to count
>>> >>>>>>> > it toward the existing MAX_CONCURRENT_STREAMS limit (I
>>> searched the spec and
>>> >>>>>>> > this seems to be inadequately specced). Roberto and I
>>> discussed that before
>>> >>>>>>> > and may have written an email somewhere in spdy-dev@, but I
>>> don't think
>>> >>>>>>> > we've ever raised it here.
>>> >>>>>>> >
>>> >>>>>>>
>>> >>>>>>> Well,  there is an issue tracking it in the github repo now, at
>>> >>>>>>> least.  As currently defined in the spec,  it definitely needs
>>> to be
>>> >>>>>>> addressed.
>>> >>>>>>
>>> >>>>>> Great. You guys are way better than I am about tracking all known
>>> >>>>>> issues. I just have it mapped fuzzily in my head :)
>>> >>>>>
>>> >>>>>
>>> >>>
>>> >>
>>> >
>>>
>>
>>
>