Re: http/2 prioritization/fairness bug with proxies

[William Chan (陈智昌) <[e-mail]>]

First, I totally agree SPDY/4 prioritization changes are far more reaching.
Let's not talk about them yet. I share your complexity concern and agree we
need data before proceeding with many of those features, and I plan on
getting data.

As far as grouping, I think it's definitely a bug for the case where you
have a forward proxy with users sharing the same HTTP/2.0 connection to an
origin server. You can have many high priority short-lived streams which
can starve out others, unless we implement the vague notion of "don't
starve out streams", which is difficult when it might be a sustained rate
of high priority short-lived streams. It's easier in your latter case of
infinite, high-priority streams.

You're right that the high priority, infinite stream can starve streams
within the same group. I don't think this means that grouping is not
required, but that grouping is potentially insufficient. For intragroup
starvation, I think it's debatable about whether the server should be smart
and not allow streams to _completely_ starve other streams within a group,
or that clients should have a reprioritization facility. I think this is a
discussion worth having, but I'd personally classify it as separate from
whether or not the grouping feature is necessary.

On Sun, Feb 10, 2013 at 2:06 PM, Mike Belshe <[e-mail]> wrote:

>
>
>
> On Mon, Feb 4, 2013 at 5:47 PM, William Chan (陈智昌) <[e-mail]>wrote:
>
>> I'm sorry if I am unclear in any way. Please continue to
>> challenge/question my comments/assertions so I can clarify my position
>> as appropriate.
>>
>> Just to be clear here, I stand by that it's a protocol bug currently.
>>
>
> +0.5.  :-)
>
> Mark - I think we could open a issue ticket with the current HTTP/2.0
> draft that this is a bug which will present itself for servers that
> implement naively.  This isn't strictly a "bug", since the spec says the
> server can do whatever it wants with priorities, but this is subtle enough
> (surfacing primarily in http/2 -> http/2 proxy situations), that many
> server implementors won't think of it unless we mention it in the spec.
>
> The bare minimum would be to simply document it and tell servers not to
> starve out streams.  However, this is probably a wimpy approach and I think
> we can do better.
>
> The SPDY/4 prioritization changes are far more reaching than just fixing
> this bug.  While I like grouping as a feature, I don't believe it actually
> fixes this bug:  a browser could open a high priority, infinite stream,
> which is competing across a shared proxy backend with other streams; unless
> the user manually switches tabs (or does something to force changed
> groups), the starvation can still occur.
>
> Mike
>
>
>
>
>
>> I agree with adding more hooks to convey advisory priority semantics.
>> That said, "advisory" is open to interpretation. I agree that the
>> sender should ultimately be in control of how it orders responses, and
>> indeed there are of course many situations where it's best for the
>> sender to ignore the advisory priority. Yet, if the advisory priority
>> semantics are generally not respected, then clients will not be able
>> to rely on them, and will be forced to implement prioritization at a
>> higher layer, which suffers from the link underutilization vs
>> contention tradeoff I highlighted earlier.
>>
>> I appreciate the concern that we're adding complexity by introducing
>> new semantics. I am arguing that because the existing mechanisms for
>> addressing starvation are suboptimal, we should treat this as a
>> protocol bug and thus change the protocol in such a way as to fix this
>> problem. My suggestion for doing so was adding new priority "grouping"
>> semantics. I am hopeful that these new semantics will not introduce an
>> inordinate amount of specification, as the primary idea is that the
>> current SPDY priority levels would apply within a "group". I think we
>> can come up with a way to define a group that will be relatively easy
>> to spec.
>>
>> SPDY/4 introduces other prioritization semantics beyond just grouping,
>> but I wanted to focus on this one first, as I believe this is a bug
>> that we *need* to fix. The other SPDY/4 priority changes are of a
>> performance optimization nature, and I believe they will need to be
>> justified by data. I have no plans to raise them up in this group
>> until we have said data.
>>
>> On Tue, Feb 5, 2013 at 5:34 AM, Poul-Henning Kamp <[e-mail]>
>> wrote:
>> > Content-Type: text/plain; charset=ISO-8859-1
>> > --------
>> > In message <[e-mail]>,
>> Ben Nive
>> > n-Jenkins writes:
>> >
>> >>So the idea is the protocol contains enough 'hooks' to sufficiently
>> >>express the different priorities between & within groups that folks
>> >>would like to express but isn't prescriptive about how anyone uses or
>> >>implements different prioritisation, scheduling, etc schemes.
>> >
>> > That was clearly not how the original poster presented it:
>> >
>> >         "I consider all those options as suboptimal, and thus
>> >         consider this issue to be a protocol bug. Our SPDY/4
>> >         prioritization proposal addresses this by [...]"
>> >
>> > --
>> > Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
>> > [e-mail]         | TCP/IP since RFC 956
>> > FreeBSD committer       | BSD since 4.3-tahoe
>> > Never attribute to malice what can adequately be explained by
>> incompetence.
>>
>>
>