Re: multiplexing -- don't do it

[William Chan (陈智昌) <[e-mail]>]

On Thu, Apr 5, 2012 at 4:05 PM, Peter Lepeska <[e-mail]> wrote:

> The idea is just a partial solution to the HOL blocking problem that at
> least decreases the likelihood that HOL issue will block high priority
> objects. I don't yet know the SPDY protocol well enough but the idea is
> that you would have two SPDY streams running over two TCP


I suggest you read up on SPDY a bit more first and come back. It may save a
lot of people time in explaining things :) At the minimum, read up on SPDY
sessions and streams (
http://willchan.github.com/SPDY-Specification/draft-mbelshe-spdy-00.html#rfc.section.2.1
).

connections. Anything that the browser deems as blocking page load, or for
> another reason high priority, will go over one SPDY stream and everything
> else will go over the other.


I think you mean using two SPDY sessions over separate TCP connections.
There's nothing in the SPDY specification which says you can't use multiple
connections, and that's up to the user-agent. That said, there are
downsides to doing so. I think we've already discussed many of the reasons
we want to use fewer connections previously, so I won't belabor those
points. I think the reason you bring up multiple connections is to work
around problems in the transport, which is why I think long-term we should
fix the transport layer too. It may be the case that a better client use of
SPDY would be to open two connections instead of a single one, but I would
push back very strongly against that in Chromium and only do it if we found
it was truly necessary.


> I agree that a UDP-based transport that supports out of order packets
> would completely solve HOL blocking for SPDY. My misunderstanding was that
> SCTP was such a transport but Mike set me straight. In any case, there are
> no obvious candidates for this so I agree we can't really design for it.
>
> Still I think we should support inter-object dependencies only if there's
> a real benefit, which I believe there is in the upstream but not so sure in
> the downstream, as compared to simply going binary.
>
> Peter
>
> On Thu, Apr 5, 2012 at 4:25 AM, William Chan (陈智昌) <[e-mail]>wrote:
>
>> On Thu, Apr 5, 2012 at 9:48 AM, Peter L <[e-mail]> wrote:
>>
>>> Thanks for the explanation on SCTP -- I have only a high level
>>> understanding and based on what you said I can't even claim that.
>>>
>>> In general, the in-flight HOL blocking problem is really serious for
>>> long fat networks: some mobile, intercontinental, satellite to name a few.
>>> So I'd like for HTTP 2.0 to at least hold the promise that it will be
>>> addressed. For instance, I proposed two SPDY streams, one high and one low
>>> priority. This would at least prevent low pri drops from blocking high pri
>>> objects. Do you have other better ideas?
>>>
>>
>> Can you explain this proposal further? It doesn't make sense to me as
>> stated (What do you mean by two SPDY streams? Are these actual SPDY streams
>> from the spec?). If HTTP/2.0 is layered on TCP, then it fundamentally will
>> have TCP-level HOL blocking. At the TCP level, there's no concept of
>> different priority objects, so any packet drop will delay other packets
>> behind it. SPDY did not attempt to address this problem, since there were
>> already so many problems higher in the stack. The real solution is to fix
>> the transport (new UDP-based protocol anyone?), but that's beyond the scope
>> of this work.
>>
>>
>>>
>>> Peter
>>>
>>>
>>> On Apr 4, 2012, at 6:05 PM, Mike Belshe <[e-mail]> wrote:
>>>
>>>
>>>
>>> On Wed, Apr 4, 2012 at 1:58 PM, Peter Lepeska <[e-mail]>wrote:
>>>
>>>> As long as SPDY is sent over TCP, it also suffers from HOL problems,
>>>> just not as bad as pipelining.
>>>>
>>>> I think SPDY (or whatever the HTTP 2.0 muxing protocol is) should
>>>> framed in such a way that if running over a protocol like SCTP, that solves
>>>> the HOL problems, we should be able to take advantage of it. Due to gzip
>>>> compression of headers, even if the transport allowed me to grab messages
>>>> out of order, I'd still have to wait for all packets prior in order to
>>>> decode the HTTP headers.
>>>>
>>>
>>>
>>> I think people have confusion about layering on top of transport
>>> protocols.  Any time you have an app protocol that wants to take advantage
>>> of new transport features you *MUST* change the definition of how the app
>>> protocol is bound to the lower level transport.  This absolutely applies to
>>> HTTP and TCP/SCTP (not talking about SPDY yet).  For example, RFC2616 does
>>> *not* specify how to use HTTP over SCTP, and a whole I-D exists for that:
>>> http://tools.ietf.org/html/draft-natarajan-http-over-sctp-01  This I-D
>>> defines one possible binding between HTTP and SCTP, but others could exist
>>> too.
>>>
>>> Why is this necessary?  Well, TCP has a single, bidirectional stream.
>>>  SCTP doesn't have bi-directional streams at all, and instead only has
>>> multiple, uni-directional streams.  So, if you want to leverage the new
>>> feature (streams) you need to define how you're going to bind onto it.
>>>  It's trivial, but doable.
>>>
>>> If we had SCTP, we wouldn't do MUX and SPDY.  It's redundant.  That's
>>> not to say that HTTP/2.0 is worse with multiplexing, its just to say that
>>> we wouldn't need to consider it if the transport already had it.  But SCTP
>>> is not viable today or even within the next decade.
>>>
>>> Google sponsored a project at the Univ of Delaware where they
>>> investigated SPDY over SCTP already.  A couple of bindings for SPDY over
>>> SCTP were considered:  use stream 0 for a control stream, and send the
>>> headers over that stream.  This has the nice property that it binds the
>>> stateful compression to a single, in-order stream.  Also, as you can
>>> imagine, it introduces a small amount of HoL there.  Another solution was
>>> to use N SCTP streams, with M mux'd SPDY streams on top of it.  Other such
>>> bindings could be considered.  Overall, SCTP was too immature to really
>>> benchmark.  The FreeBSD and Linux implementations of SCTP have many
>>> problems already.  The UoD guys might want to comment, I found the whole
>>> thing non-conclusive.
>>>
>>> My recommendation:
>>> Designing HTTP/2.0 for SCTP is a mistake and should NOT be a
>>> requirement.  SCTP is not a viable transport over the Internet today, and
>>> will not be in the foreseeable future.  When it is available, an
>>> appropriate binding for HTTP/2.0 can be determined, trivially, and we can
>>> worry about it then.  This is the same approach that was taken for HTTP
>>> with http://tools.ietf.org/html/draft-natarajan-http-over-sctp-01.
>>>
>>> Mike
>>>
>>>
>>>
>>>
>>>>
>>>> Peter
>>>>
>>>>
>>>> On Wed, Apr 4, 2012 at 9:07 AM, Patrick McManus <[e-mail]>wrote:
>>>>
>>>>> On Wed, 2012-04-04 at 07:02 +0000, Poul-Henning Kamp wrote:
>>>>> > In message <[e-mail]>, Willy Tarreau writes:
>>>>> >
>>>>> > >> I'm starting to get data back, but not in a state that I'd
>>>>> reliably
>>>>> > >> release. That said, there are very clear indicators of
>>>>> intermediaries
>>>>> > >> causing problems, especially when the pipeline depth exceeds 3
>>>>> requests.
>>>>> >
>>>>> > I always thought that the problem in HTTP/1.x is that you can never
>>>>> > quite be sure if there is an un-warranted entity comming after at
>>>>> GET,
>>>>>
>>>>> its not uncommon to have the consumer RST the whole TCP session when
>>>>> asked to recv too far beyond the current request it is processing. For
>>>>> some devices "too far" appears to be defined as "any new packet". I
>>>>> presume some variation of this is where Will's data point comes from.
>>>>> (Often 3 uncompressed requests fit in 1 packet).
>>>>>
>>>>> That class of bug sounds absurd, but its really a pretty common
>>>>> pattern.
>>>>> As an example: hosts that fail TLS False Start (for which I understand
>>>>> second hand that Chrome needs to keep a black-list), react badly
>>>>> because
>>>>> there is TCP data queued when they are in a state that the expect their
>>>>> peer to be quiet. Same pattern.
>>>>>
>>>>> The lesson to me is that you want to define a tight set of
>>>>> functionality
>>>>> that is reasonably testable up front - and that's what you can depend
>>>>> widely on later. Using anything beyond that demands excessive levels of
>>>>> pain, complexity, and cleverness.
>>>>>
>>>>> (and all this pipelining talk as if it were equivalent to spdy mux is
>>>>> kind of silly. Pipelining's intrinsic HOL problems are at least as bad
>>>>> of an issue as the interop bugs.)
>>>>>
>>>>> -Patrick
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>