Re: Design Issue: Max Concurrent Streams Limit and Unidirectional Streams

William Chan (陈智昌) <> Mon, 29 April 2013 21:45 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 79E1E21F9BAF for <>; Mon, 29 Apr 2013 14:45:30 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -9.676
X-Spam-Status: No, score=-9.676 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_HI=-8]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id kXeFGob8ukEd for <>; Mon, 29 Apr 2013 14:45:25 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id D910E21F9B99 for <>; Mon, 29 Apr 2013 14:45:24 -0700 (PDT)
Received: from lists by with local (Exim 4.72) (envelope-from <>) id 1UWvsY-00009U-9o for; Mon, 29 Apr 2013 21:44:58 +0000
Resent-Date: Mon, 29 Apr 2013 21:44:58 +0000
Resent-Message-Id: <>
Received: from ([]) by with esmtp (Exim 4.72) (envelope-from <>) id 1UWvsO-00008k-9b for; Mon, 29 Apr 2013 21:44:48 +0000
Received: from ([]) by with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.72) (envelope-from <>) id 1UWvsN-0006EY-4I for; Mon, 29 Apr 2013 21:44:48 +0000
Received: by with SMTP id k5so2863341qej.9 for <>; Mon, 29 Apr 2013 14:44:21 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=SFBG99vrqODYuCLr7uKM94UHORMrgB3x0S4t3Ln6dYQ=; b=N9zsjjPowkdI9X1aC/jD05Bum4ISnT2P7rr9Vya43+y+HGG4RxkqpQYv5u+dbHunhr JHinj/fMr7QhZlK9FIkZFZNilkJ6lUQr0/KEzNAXuf6TKufM7JASNiF6bLoTyZsN4XIy BlMOflYbp4nQvtHQpr1M4iWuBFpIBuO9glWqJb9VL1o02pWzKIKV+eVqfj13QVJixB37 C5ZSvamBXCSCw4k3FQ75t4CTLjsXf5TQm596LZ6QDOPrN64ZfNrH7v7fgIlBd73VmuqD qCsTFWxfeRFqDn3LeL33ij4lnEm7STiQp8yXMWDY8SLhE1hp6YkqV+p1BSfiFDe9uqTL vzpw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=google; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=SFBG99vrqODYuCLr7uKM94UHORMrgB3x0S4t3Ln6dYQ=; b=Sp6ZeAZGsVnFg4fAHLDabRdYkhRE7uYKAx56dQbTqF4tCO/4VgcZMvDaX4ZZfv5ST3 4Z4oQc7rynXwgvm41B0XjTER7chh78bECRSg7Wykq+5rkWOsUddEaYuxlu6Ab/JjzZ06 9WRJ20yIXwIrq4G+HSmQzDnAlUqno429S2qyE=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :x-gm-message-state; bh=SFBG99vrqODYuCLr7uKM94UHORMrgB3x0S4t3Ln6dYQ=; b=HEec2EoopJzQMMgaXrFPJmjYZgrh+EkM4FfGhom7IkoaZsyGb+M1ZW39eMpkYfy+L1 YpM+vn+HD6mNm0FgT0UTWfr+QZiSIvTUMp+jJGKBpus6HPLDSFxSTMf6r2zqPYYy8SDE 0F/WY2R78G2kp69nXXlJwPIDdIA+Fyes7PFuT+7uqXH1j+Gg6BsXxEDGVg2nM/t7HtJy Izl9HwI1iWV0xtNTDKX6jXVqp4vTRB0EE0IZjopZnOxZmLAM9hthiY4GUK7oBLGrWrez bIKrdYLo6O2Mq8lgcf9i5Faz1oAWAI0pckdhPsEQW8oIFlts1sHXy+wkF4Vikfb+En4/ Jo/g==
MIME-Version: 1.0
X-Received: by with SMTP id b18mr8053459qah.36.1367271861525; Mon, 29 Apr 2013 14:44:21 -0700 (PDT)
Received: by with HTTP; Mon, 29 Apr 2013 14:44:21 -0700 (PDT)
In-Reply-To: <>
References: <> <> <> <> <> <> <> <> <> <>
Date: Mon, 29 Apr 2013 18:44:21 -0300
X-Google-Sender-Auth: i5hqaNfwjiMGiEaFbNcUdpR4Fik
Message-ID: <>
From: "William Chan (陈智昌)" <>
To: Roberto Peon <>
Cc: James M Snell <>, HTTP Working Group <>, Martin Thomson <>
Content-Type: multipart/alternative; boundary="089e01538d549e35b204db86c979"
X-Gm-Message-State: ALoCoQk+y8Ievd5Te9ydV/Dv0UECb2qXQ9FWA6C5LwL5c2gLiBrgf/vyfKhr11w71QCRtVLPBSJAmJ4jkTrZmUArxE/nCj+pqvXdPo+/8WNyejfFAN2//UH/HYzo4C3TCrIpUFF9FL4xW9ugOGryJ8Mr4uoe8MIz19vhnCyNUHKGlv6zubmiHRciwGAXCQWC7SZ+AZokZkcq
Received-SPF: pass client-ip=;;
X-W3C-Hub-Spam-Status: No, score=-5.7
X-W3C-Hub-Spam-Report: AWL=-0.528, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RP_MATCHES_RCVD=-2.442, SPF_PASS=-0.001
X-W3C-Scan-Sig: 1UWvsN-0006EY-4I a6e0cc639c404fe9a1e8ac66b9b13db0
Subject: Re: Design Issue: Max Concurrent Streams Limit and Unidirectional Streams
Archived-At: <>
X-Mailing-List: <> archive/latest/17685
Precedence: list
List-Id: <>
List-Help: <>
List-Post: <>
List-Unsubscribe: <>

Remember we originally *had* a flag for UNIDIRECTIONAL, which we removed
because it was redundant in the traditional HTTP use cases.

On Mon, Apr 29, 2013 at 6:39 PM, Roberto Peon <> wrote:

> At worst, we burn a flag which states it is half-closed or unidirectional,
> or provide some other information which identifies the IANA port number for
> the overlayed protocol or something.
> Anyway, *shrug*.
> -=R
> On Mon, Apr 29, 2013 at 2:32 PM, William Chan (陈智昌) <
> > wrote:
>> On Mon, Apr 29, 2013 at 6:17 PM, James M Snell <> wrote:
>>> +1 on this.  I like this approach.
>>>  On Apr 29, 2013 2:15 PM, "Roberto Peon" <> wrote:
>>>> I had thought to provide no explicit limit for PUSH_PROMISE, just as
>>>> there is no limit to the size of a webpage, or the number of links upon it.
>>>> The memory requirements for PUSH are similar or the same (push should
>>>> consume a single additional bit of overhead per url, when one considers
>>>> that the URL should be parsed, enqueued, etc.).
>>>> If the browser isn't done efficiently, or, the server is for some
>>>> unknown reason being stupid and attempting to DoS the browser with many
>>>> resources that it will never use, then the client sends RST_STREAM for the
>>>> ones it doesn't want, and makes a request on its own. all tidy.
>> I don't feel too strongly here. I do feel like this is more of an edge
>> case, possibly important for forward proxies (or reverse proxies speaking
>> to backends over a multiplexed channel like HTTP/2). It doesn't really
>> matter for my browser, so unless servers chime in and say they'd prefer a
>> limit, I'm fine with this.
>>>> As for PUSH'd streams, the easiest solution is likely to assume that
>>>> the stream starts out in a half-closed state.
>>  I looked into our earlier email threads and indeed this is what we
>> agreed on (
>> I voiced some mild objection since if you view the HTTP/2 framing layer as
>> a transport for another application protocol, then bidirectional server
>> initiated streams might be nice. But in absence of any such protocol, this
>> is a nice simplification.
>>> -=R
>>>> On Mon, Apr 29, 2013 at 12:33 PM, William Chan (陈智昌) <
>>>>> wrote:
>>>>> On Mon, Apr 29, 2013 at 3:46 PM, James M Snell <>wrote:
>>>>>> On Apr 29, 2013 11:36 AM, "William Chan (陈智昌)" <>
>>>>>> wrote:
>>>>>> >
>>>>>> [snip]
>>>>>> >
>>>>>> >
>>>>>> > Oops, forgot about that. See, the issue with that is now we've made
>>>>>> PUSH_PROMISE as potentially expensive as a HEADERS frame, since it does
>>>>>> more than just simple stream id allocation. I guess it's not really a huge
>>>>>> issue, since if it's used correctly (in the matter you described), then it
>>>>>> shouldn't be too expensive. If clients attempt to abuse it, then servers
>>>>>> should probably treat it in a similar manner as they treat people trying to
>>>>>> abuse header compression in all other frames with the header block, and
>>>>>> kill the connection accordingly.
>>>>>> >
>>>>>> Not just "potentially" as expensive..   As soon as we get a push
>>>>>> promise we need to allocate state and hold onto it for an indefinite period
>>>>>> of time. We do not yet know exactly when that compression context can be
>>>>>> let go because it has not yet been bound to stream state.  Do push streams
>>>>>> all share the same compression state? Do those share the same compression
>>>>>> state as the originating stream? The answers might be obvious but they
>>>>>> haven't yet been written down.
>>>>> I guess I don't see per-stream state as being that expensive.
>>>>> Compression contexts are a fixed state on a per-connection basis, meaning
>>>>> that additional streams don't add to that state. The main cost, as I see
>>>>> it, is the decompressed headers. I said potentially since that basically
>>>>> only means the URL (unless there are other headers important for caching
>>>>> due to Vary), and additional headers can come in the HEADERS frame. Also,
>>>>> PUSH_PROMISE doesn't require allocating other state, like backend/DB
>>>>> connections, if you only want to be able to handle
>>>>> (#MAX_CONCURRENT_STREAMs) of those backend connections in parallel.
>>>>> If they're not specified, then we should specify it, but I've always
>>>>> understood the header compression contexts to be directional and apply to
>>>>> all frames sending headers in a direction. Therefore there should be two
>>>>> compression contexts in a connection, one for header blocks being sent and
>>>>> one for header blocks being received. If this is controversial, let's fork
>>>>> a thread and discuss it.
>>>>>>  >>
>>>>>> >>
>>>>>> >> > As far as the potential problem above, the root problem is that
>>>>>> when you
>>>>>> >> > have limits you can have hangs. We see this all the time today
>>>>>> with browsers
>>>>>> >> > (it's only reason people do domain sharding so they can bypass
>>>>>> limits). I'm
>>>>>> >> > not sure I see the value of introducing the new proposed limits.
>>>>>> They don't
>>>>>> >> > solve the hangs, and I don't think the granularity addresses any
>>>>>> of the
>>>>>> >> > costs in a finer grained manner. I'd like to hear clarification
>>>>>> on what
>>>>>> >> > costs the new proposed limits will address.
>>>>>> >>
>>>>>> >> I don't believe that the proposal improves the situation enough
>>>>>> (or at
>>>>>> >> all) to justify the additional complexity.  That's something that
>>>>>> you
>>>>>> >> need to assess for yourself.  This proposal provides more granular
>>>>>> >> control, but it doesn't address the core problem, which is that you
>>>>>> >> and I can only observe each other actions after some delay, which
>>>>>> >> means that we can't coordinate those actions perfectly.  Nor can be
>>>>>> >> build a perfect model of the other upon which to observe and act
>>>>>> upon.
>>>>>> >>  The usual protocol issue.
>>>>>> >
>>>>>> >
>>>>>> > OK then. My proposal is to add a new limit for PUSH_PROMISE frames
>>>>>> though, separately from the MAX_CONCURRENT_STREAMS limit, since
>>>>>> PUSH_PROMISE exists as a promise to create a stream, explicitly so we don't
>>>>>> have to count it toward the existing MAX_CONCURRENT_STREAMS limit (I
>>>>>> searched the spec and this seems to be inadequately specced). Roberto and I
>>>>>> discussed that before and may have written an email somewhere in spdy-dev@,
>>>>>> but I don't think we've ever raised it here.
>>>>>> >
>>>>>> Well,  there is an issue tracking it in the github repo now, at
>>>>>> least.  As currently defined in the spec,  it definitely needs to be
>>>>>> addressed.
>>>>> Great. You guys are way better than I am about tracking all known
>>>>> issues. I just have it mapped fuzzily in my head :)