Re: http/2 prioritization/fairness bug with proxies

[Nico Williams <[e-mail]>]

On Thu, Feb 14, 2013 at 3:42 PM, Roberto Peon <[e-mail]> wrote:
> On Thu, Feb 14, 2013 at 12:26 PM, Nico Williams <[e-mail]>
> wrote:
>> But it's possible that we're talking about different thing.  One thing
>> is priority for server processing of requests.  Another is for proxies
>> -- here we have to start worrying about the multiplexing issues that
>> SSHv1 and SSHv2 have had, and since I think we ware talking about
>> proxies I keep coming back to these issues.
>
> Priority and flow control are separate issues.
> Priorities are an expression of the order in which the browser wants/needs
> resources.

Ah, if that's all priorities do, then there's no issue.  However, if
they relate to traffic classes (bulk vs. non-bulk) then we have the
SSH problems.

> Flow control is for receivers which are memory constrained to be sure that
> they won't overrun memory requirements.

It's for sinks that are slow.  All receivers are memory constrained, after all.

>> > From an implementation standpoint: I'm already running out of ephemeral
>> > port
>> > space. I *do Not* want to use more connections.
>>
>> It's certainly what browsers do already, and have for many years.
>> What's the problem?
>
> Browsers do this because HTTP effectively limits you to one outstanding
> request per connection, which is effectively one request per RTT per
> connection.

Pipelining allows multiple requests to be sent before responses are
written back.  The problem is that responses must be written in the
same order as the requests.  The solution to that would be to add an
XID (like basically any RPC protocol would, e.g., ONC RPC).

> Allowing a larger concurrency per connection than 1 is what the multiplexing
> does.

I suspect that's not the only reason that browsers make multiple
concurrent connections to servers.  To be fair, the SSH problems can't
the reason either.

> Browsers are most often not the ones likely to run out of ephemeral port
> space-- that is a problem for proxies and servers.

Ah!  This is a good thing to point out.  But note that proxies are in
a decent position to know if SCTP can work (namely, they can cache
which servers it's worked for and which it hasn't).  So an SCTP
binding of HTTP might still be useful.

>> > From an implementation standpoint: It is often impossible to figure out
>> > that
>> > k connections belong to a single client at all loadbalancers without
>> > sacrificing orders of magnitude of performance. Orders of magnitude, not
>> > just a factor of X.
>>
>> But I'm not saying you need to do that.  Nor am I implying it.
>
> If there are multiple connections, then the cost of coordination for this
> kind of loadbalancing at large sites is orders of magnitude higher.
> You're telling me that you wish to use multiple connections. Unless my
> statement above is a lie, it must then follow that it will cost orders of
> magnitude more for the same coordination.

I'm telling you that if SSH multiplexing/flow control issues arise
then the best solution is to have two TCP connections: one for bulk,
and one for non-bulk.  Bulk/not-bulk adds a single boolean input to
the routing process.  I don't see how that raises the cost of load
balancing by one order of magnitude, much less two, not unless issues
like scarcity of ephemeral ports are the root cause (load balancers
are unlikely to have this problem, but proxies are).

>> > Browsers have used multiple connections because of limitations in HTTP,
>> > which we're already solving with multiplexing.
>>
>> I strongly suspect that you're not solving them.  It's not just "oh,
>> let's multiplex".  You need to watch out for the issues other
>> protocols that multiplex different flows on single TCP flows have had.
>>  You haven't demonstrated a grasp of those issues.
>
>
> Nico, I've been working closely with browser folks for quite a while (years)
> and have a good grounding in what is suboptimal today, after all, SPDY
> started as a project with a server/proxy developer (me) and a browser
> developer (Mike Belshe).

You're right, I did not express myself very well.  I apologize.  I am
concerned with any notion of flow control layered above TCP.  I'm not
as concerned about priorities now that I understand their purpose (I
did suspect we might be talking about different things.)

Nico
--