Re: Design Issue: Max Concurrent Streams Limit and Unidirectional Streams

William Chan (陈智昌) <willchan@chromium.org> Wed, 01 May 2013 17:48 UTC

Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 5612121F9A46 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Wed, 1 May 2013 10:48:13 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -9.676
X-Spam-Level:
X-Spam-Status: No, score=-9.676 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hbNr-syGft90 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Wed, 1 May 2013 10:48:04 -0700 (PDT)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) by ietfa.amsl.com (Postfix) with ESMTP id 2E2FF21F9A6B for <httpbisa-archive-bis2Juki@lists.ietf.org>; Wed, 1 May 2013 10:48:01 -0700 (PDT)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1UXb7Z-0003qn-9w for ietf-http-wg-dist@listhub.w3.org; Wed, 01 May 2013 17:47:13 +0000
Resent-Date: Wed, 01 May 2013 17:47:13 +0000
Resent-Message-Id: <E1UXb7Z-0003qn-9w@frink.w3.org>
Received: from maggie.w3.org ([128.30.52.39]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <willchan@google.com>) id 1UXb7O-0003nF-PO for ietf-http-wg@listhub.w3.org; Wed, 01 May 2013 17:47:02 +0000
Received: from mail-qa0-f52.google.com ([209.85.216.52]) by maggie.w3.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.72) (envelope-from <willchan@google.com>) id 1UXb7N-0006iO-4C for ietf-http-wg@w3.org; Wed, 01 May 2013 17:47:02 +0000
Received: by mail-qa0-f52.google.com with SMTP id hg5so905143qab.4 for <ietf-http-wg@w3.org>; Wed, 01 May 2013 10:46:35 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=p+xlZHL7MvayaN/RtEqQQw/t5na32CmWP/Gu1gNayEg=; b=bi0mGIDhepNchEt2sURhV7Aq6G60i2P9FIWHG2DyTBvI7MDZr8Rh8Ib6TLhIyj80Rx MhKLMtahRdssfE1kM3URNcikapRplAro4ZfrKazfiQ+eyrZnoDvDRi/4iSZPWCew5egm iiuz5YNoeKlPyZsy+cprFQG05tN7fZcpDMep+hZ/YwnfK5vPZApGrKLFfhSxiAXvR/pM cf2gl9hNRtrcm8vwtSoKYtIloe427hyB9U4Pyqee/GstPNzVCtZmKbbXdQ7HLAWSzI3k tjyrDUb0C50eXcESI4145G6v58bUZHbifm+GgOGsfQzMRjwSG7tTP7QRLRbkZv0sE6hC 86oA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=p+xlZHL7MvayaN/RtEqQQw/t5na32CmWP/Gu1gNayEg=; b=og398owF17GxkgZDMv0rT1a5m4HdWw2cXzpA3WN45uLHehK8PO0EeDghAVq0U9O/eI KBCI4djntTgylPLGuvUsvh55+Had7T5rp/ebtXVrqcoEbEAT8hXNjbQIjfVb7A62C3lz Y3zh9LxWuZ/6cxif1NSMIpfhrkr5bcCcTk94I=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :x-gm-message-state; bh=p+xlZHL7MvayaN/RtEqQQw/t5na32CmWP/Gu1gNayEg=; b=IYctQM6Hn6s8JUYa8E0CGpZgh07av4Ia9YfPQvde7qnt5ZvoNRP0tXcTN1bQ39pcqM 3qH745189Ff0DczTuQ/Qa83MrbY3CxmRf0qFLcSe1sdLR/z7ZfAnYAOP7qm/zIGouFKC D6Q1s6duwuuWK5Ziwa6QC1S5bta1tlZRdUkZm8VN2Hxpf5hVmYLqccJ4Zq5hS3pTZ6ul UJBnLazqHafnaBPojRsCc/avfllwT7Z6DL75paeB/B2f8C93oxx7JO1X/TYQ1zq8ua8H QywG/SVUQ+11nsLmr0vT147MRNbCCsh+3MHA1uRJ+SJ0vpJWTxl83jmag/EnF7YfFtIE tw0w==
MIME-Version: 1.0
X-Received: by 10.224.222.14 with SMTP id ie14mr371728qab.34.1367430395215; Wed, 01 May 2013 10:46:35 -0700 (PDT)
Sender: willchan@google.com
Received: by 10.229.180.4 with HTTP; Wed, 1 May 2013 10:46:35 -0700 (PDT)
In-Reply-To: <CABP7Rbdgz=kRZPfjHK5UUfieq8uz=ToQZjFt1-+s9scj1CogmA@mail.gmail.com>
References: <CABP7RbdBe-Xkx+CMvpN=_oNAqm6SyLyL+XNHRUKSqn8mjSDw1Q@mail.gmail.com> <CAA4WUYgCiyWerT0tUUVKcbNPqdTGuXHd_MG59DjcUsEWst5t7g@mail.gmail.com> <CABkgnnVdU=cZ53Bqg5Un=E80NMpcgYO37DVmwUFW0O-i7SNf8w@mail.gmail.com> <CAA4WUYhz64FsEGgGhx91RfWwuPPxWdAkesOV-bmqWVWE7ZxdjA@mail.gmail.com> <CABP7RbcKQkn1o4WZscwNmSmm6YzqE_TKxPr4jnozNdaVqpZ7=A@mail.gmail.com> <CAA4WUYhF6rAZoYEaz4aJO6xawaJxzxGt=Bkg4H9eBOP-LBSRmQ@mail.gmail.com> <CAP+FsNezQzxdZEJY_2_0h_TR2pBbVsGyGBhQhKcm-65pt6S8rQ@mail.gmail.com> <CABP7RbevS8M0q9OxzPncqY_gE34q5-ymdg2hOX2SQgSUNkhzsw@mail.gmail.com> <CAA4WUYjAbuUqz9RdO+-p3a4EsyuS=Gv0rS-U-Vh+ZCjtDjFy6w@mail.gmail.com> <CAP+FsNec2LLZMjtGhSX-1q8qg66WtBoM5K0yMrs5m4VKXb5OVg@mail.gmail.com> <CAA4WUYgAT64jj=Am06MsA02A+eAcDrVbbgb4opO37bnMkWTPfg@mail.gmail.com> <CABP7Rbdgz=kRZPfjHK5UUfieq8uz=ToQZjFt1-+s9scj1CogmA@mail.gmail.com>
Date: Wed, 1 May 2013 14:46:35 -0300
X-Google-Sender-Auth: TMHCPqcRkAJxGtwc14ZzfbtIwyM
Message-ID: <CAA4WUYjSjFKSdbj=QBLn0T4ufhzF1hUY=O=Qa2dfnkTzMXF0bg@mail.gmail.com>
From: =?UTF-8?B?V2lsbGlhbSBDaGFuICjpmYjmmbrmmIwp?= <willchan@chromium.org>
To: James M Snell <jasnell@gmail.com>
Cc: Roberto Peon <grmocg@gmail.com>, HTTP Working Group <ietf-http-wg@w3.org>, Martin Thomson <martin.thomson@gmail.com>
Content-Type: multipart/alternative; boundary=20cf3074b45cf65cdd04dbabb21f
X-Gm-Message-State: ALoCoQmesVWxaVnLVzDPUF339BfVsZkODfERzVQX7Fq+x5euPh1zuUDqBp/FA5eoOJWHyEmSjVaDcZ7SEEgJfcg8X1KVy5uIqWggYa6AnrtHDcIWMEmbUYxoSW0kuGadGwhuKIG5dknfAUig7DylAt10MkfKiBmsoBIF6jyrNSVNBsxv6oCHTFWlWEiBhUyV2QOHfy2o4kxr
Received-SPF: pass client-ip=209.85.216.52; envelope-from=willchan@google.com; helo=mail-qa0-f52.google.com
X-W3C-Hub-Spam-Status: No, score=-4.8
X-W3C-Hub-Spam-Report: AWL=-1.413, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-2.57, SPF_PASS=-0.001
X-W3C-Scan-Sig: maggie.w3.org 1UXb7N-0006iO-4C 593a7914d2308a456ca12316db82b155
X-Original-To: ietf-http-wg@w3.org
Subject: Re: Design Issue: Max Concurrent Streams Limit and Unidirectional Streams
Archived-At: <http://www.w3.org/mid/CAA4WUYjSjFKSdbj=QBLn0T4ufhzF1hUY=O=Qa2dfnkTzMXF0bg@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/17766
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

The only benefit to that is supporting non-HTTP/2 application layering
semantics, which is intended not to change from HTTP/1.X. So there's
currently no use to allow the server to initiate streams with the
client=>server direction open.

I consider the current trend of our discussions to tend towards eliminating
complexity and targeting for HTTP/2 application layering semantics. I think
if we have another use case come up that would require supporting server
initiated bidirectional streams, I think at that point it'd be worthwhile
to revisit how we do this.

I'd like to hear from others if they disagree with my assessment of how
most people feel so far. FWIW, I personally would like us to support server
initiated bidirectional streams.


On Wed, May 1, 2013 at 2:26 PM, James M Snell <jasnell@gmail.com>; wrote:

> Why not just bring the UNIDIRECTIONAL flag back as a PUSH_PROMISE
> frame-specific flag? If a PUSH_PROMISE frame has the unidirectional
> flag set, the stream is automatically half-closed in the return
> direction. If the flag is unset, the promised stream remains half-open
> until the client half-closes or a rst_stream is sent.
>
> On Mon, Apr 29, 2013 at 2:44 PM, William Chan (陈智昌)
> <willchan@chromium.org>; wrote:
> > Remember we originally *had* a flag for UNIDIRECTIONAL, which we removed
> > because it was redundant in the traditional HTTP use cases.
> >
> >
> > On Mon, Apr 29, 2013 at 6:39 PM, Roberto Peon <grmocg@gmail.com>; wrote:
> >>
> >> At worst, we burn a flag which states it is half-closed or
> unidirectional,
> >> or provide some other information which identifies the IANA port number
> for
> >> the overlayed protocol or something.
> >> Anyway, *shrug*.
> >> -=R
> >>
> >>
> >> On Mon, Apr 29, 2013 at 2:32 PM, William Chan (陈智昌)
> >> <willchan@chromium.org>; wrote:
> >>>
> >>> On Mon, Apr 29, 2013 at 6:17 PM, James M Snell <jasnell@gmail.com>;
> wrote:
> >>>>
> >>>> +1 on this.  I like this approach.
> >>>>
> >>>> On Apr 29, 2013 2:15 PM, "Roberto Peon" <grmocg@gmail.com>; wrote:
> >>>>>
> >>>>> I had thought to provide no explicit limit for PUSH_PROMISE, just as
> >>>>> there is no limit to the size of a webpage, or the number of links
> upon it.
> >>>>> The memory requirements for PUSH are similar or the same (push should
> >>>>> consume a single additional bit of overhead per url, when one
> considers that
> >>>>> the URL should be parsed, enqueued, etc.).
> >>>>> If the browser isn't done efficiently, or, the server is for some
> >>>>> unknown reason being stupid and attempting to DoS the browser with
> many
> >>>>> resources that it will never use, then the client sends RST_STREAM
> for the
> >>>>> ones it doesn't want, and makes a request on its own. all tidy.
> >>>
> >>>
> >>> I don't feel too strongly here. I do feel like this is more of an edge
> >>> case, possibly important for forward proxies (or reverse proxies
> speaking to
> >>> backends over a multiplexed channel like HTTP/2). It doesn't really
> matter
> >>> for my browser, so unless servers chime in and say they'd prefer a
> limit,
> >>> I'm fine with this.
> >>>
> >>>>>
> >>>>> As for PUSH'd streams, the easiest solution is likely to assume that
> >>>>> the stream starts out in a half-closed state.
> >>>
> >>>
> >>> I looked into our earlier email threads and indeed this is what we
> agreed
> >>> on (
> http://lists.w3.org/Archives/Public/ietf-http-wg/2013JanMar/1106.html).
> >>> I voiced some mild objection since if you view the HTTP/2 framing
> layer as a
> >>> transport for another application protocol, then bidirectional server
> >>> initiated streams might be nice. But in absence of any such protocol,
> this
> >>> is a nice simplification.
> >>>
> >>>>>
> >>>>> -=R
> >>>>>
> >>>>>
> >>>>> On Mon, Apr 29, 2013 at 12:33 PM, William Chan (陈智昌)
> >>>>> <willchan@chromium.org>; wrote:
> >>>>>>
> >>>>>> On Mon, Apr 29, 2013 at 3:46 PM, James M Snell <jasnell@gmail.com>;
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>> On Apr 29, 2013 11:36 AM, "William Chan (陈智昌)"
> >>>>>>> <willchan@chromium.org>; wrote:
> >>>>>>> >
> >>>>>>> [snip]
> >>>>>>>
> >>>>>>> >
> >>>>>>> >
> >>>>>>> > Oops, forgot about that. See, the issue with that is now we've
> made
> >>>>>>> > PUSH_PROMISE as potentially expensive as a HEADERS frame, since
> it does more
> >>>>>>> > than just simple stream id allocation. I guess it's not really a
> huge issue,
> >>>>>>> > since if it's used correctly (in the matter you described), then
> it
> >>>>>>> > shouldn't be too expensive. If clients attempt to abuse it, then
> servers
> >>>>>>> > should probably treat it in a similar manner as they treat
> people trying to
> >>>>>>> > abuse header compression in all other frames with the header
> block, and kill
> >>>>>>> > the connection accordingly.
> >>>>>>> >
> >>>>>>>
> >>>>>>> Not just "potentially" as expensive..   As soon as we get a push
> >>>>>>> promise we need to allocate state and hold onto it for an
> indefinite period
> >>>>>>> of time. We do not yet know exactly when that compression context
> can be let
> >>>>>>> go because it has not yet been bound to stream state.  Do push
> streams all
> >>>>>>> share the same compression state? Do those share the same
> compression state
> >>>>>>> as the originating stream? The answers might be obvious but they
> haven't yet
> >>>>>>> been written down.
> >>>>>>
> >>>>>>
> >>>>>> I guess I don't see per-stream state as being that expensive.
> >>>>>> Compression contexts are a fixed state on a per-connection basis,
> meaning
> >>>>>> that additional streams don't add to that state. The main cost, as
> I see it,
> >>>>>> is the decompressed headers. I said potentially since that
> basically only
> >>>>>> means the URL (unless there are other headers important for caching
> due to
> >>>>>> Vary), and additional headers can come in the HEADERS frame. Also,
> >>>>>> PUSH_PROMISE doesn't require allocating other state, like backend/DB
> >>>>>> connections, if you only want to be able to handle
> (#MAX_CONCURRENT_STREAMs)
> >>>>>> of those backend connections in parallel.
> >>>>>>
> >>>>>> If they're not specified, then we should specify it, but I've always
> >>>>>> understood the header compression contexts to be directional and
> apply to
> >>>>>> all frames sending headers in a direction. Therefore there should
> be two
> >>>>>> compression contexts in a connection, one for header blocks being
> sent and
> >>>>>> one for header blocks being received. If this is controversial,
> let's fork a
> >>>>>> thread and discuss it.
> >>>>>>
> >>>>>>>
> >>>>>>> >>
> >>>>>>> >>
> >>>>>>> >> > As far as the potential problem above, the root problem is
> that
> >>>>>>> >> > when you
> >>>>>>> >> > have limits you can have hangs. We see this all the time today
> >>>>>>> >> > with browsers
> >>>>>>> >> > (it's only reason people do domain sharding so they can bypass
> >>>>>>> >> > limits). I'm
> >>>>>>> >> > not sure I see the value of introducing the new proposed
> limits.
> >>>>>>> >> > They don't
> >>>>>>> >> > solve the hangs, and I don't think the granularity addresses
> any
> >>>>>>> >> > of the
> >>>>>>> >> > costs in a finer grained manner. I'd like to hear
> clarification
> >>>>>>> >> > on what
> >>>>>>> >> > costs the new proposed limits will address.
> >>>>>>> >>
> >>>>>>> >> I don't believe that the proposal improves the situation enough
> >>>>>>> >> (or at
> >>>>>>> >> all) to justify the additional complexity.  That's something
> that
> >>>>>>> >> you
> >>>>>>> >> need to assess for yourself.  This proposal provides more
> granular
> >>>>>>> >> control, but it doesn't address the core problem, which is that
> >>>>>>> >> you
> >>>>>>> >> and I can only observe each other actions after some delay,
> which
> >>>>>>> >> means that we can't coordinate those actions perfectly.  Nor can
> >>>>>>> >> be
> >>>>>>> >> build a perfect model of the other upon which to observe and act
> >>>>>>> >> upon.
> >>>>>>> >>  The usual protocol issue.
> >>>>>>> >
> >>>>>>> >
> >>>>>>> > OK then. My proposal is to add a new limit for PUSH_PROMISE
> frames
> >>>>>>> > though, separately from the MAX_CONCURRENT_STREAMS limit, since
> PUSH_PROMISE
> >>>>>>> > exists as a promise to create a stream, explicitly so we don't
> have to count
> >>>>>>> > it toward the existing MAX_CONCURRENT_STREAMS limit (I searched
> the spec and
> >>>>>>> > this seems to be inadequately specced). Roberto and I discussed
> that before
> >>>>>>> > and may have written an email somewhere in spdy-dev@, but I
> don't think
> >>>>>>> > we've ever raised it here.
> >>>>>>> >
> >>>>>>>
> >>>>>>> Well,  there is an issue tracking it in the github repo now, at
> >>>>>>> least.  As currently defined in the spec,  it definitely needs to
> be
> >>>>>>> addressed.
> >>>>>>
> >>>>>> Great. You guys are way better than I am about tracking all known
> >>>>>> issues. I just have it mapped fuzzily in my head :)
> >>>>>
> >>>>>
> >>>
> >>
> >
>