Re: http/2 prioritization/fairness bug with proxies

Roberto Peon <grmocg@gmail.com> Fri, 08 February 2013 07:26 UTC

Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6947721F89AE for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 7 Feb 2013 23:26:02 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.531
X-Spam-Level:
X-Spam-Status: No, score=-10.531 tagged_above=-999 required=5 tests=[AWL=0.067, BAYES_00=-2.599, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([64.170.98.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zipyE0teiWjq for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 7 Feb 2013 23:26:01 -0800 (PST)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) by ietfa.amsl.com (Postfix) with ESMTP id C97B321F89AA for <httpbisa-archive-bis2Juki@lists.ietf.org>; Thu, 7 Feb 2013 23:26:00 -0800 (PST)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1U3iJt-0007iV-MK for ietf-http-wg-dist@listhub.w3.org; Fri, 08 Feb 2013 07:24:25 +0000
Resent-Date: Fri, 08 Feb 2013 07:24:25 +0000
Resent-Message-Id: <E1U3iJt-0007iV-MK@frink.w3.org>
Received: from maggie.w3.org ([128.30.52.39]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <grmocg@gmail.com>) id 1U3iJi-0007Yl-O4 for ietf-http-wg@listhub.w3.org; Fri, 08 Feb 2013 07:24:15 +0000
Received: from mail-lb0-f176.google.com ([209.85.217.176]) by maggie.w3.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.72) (envelope-from <grmocg@gmail.com>) id 1U3iJf-0006ab-Ey for ietf-http-wg@w3.org; Fri, 08 Feb 2013 07:24:14 +0000
Received: by mail-lb0-f176.google.com with SMTP id s4so2718479lbc.21 for <ietf-http-wg@w3.org>; Thu, 07 Feb 2013 23:23:44 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=5wYsjIs+N7VnRjYbFLnxrTuMBOKoaxmlcnc9DvYoCm8=; b=J08C9ixy4W4Ywnd9dT9K05Iaqij8Y+un5FS7P67r0E7QLi9Qnqwr7pHeBhufVMIS8T 9Ov9PDnu8kWaMbikfXI6zq1TDIfXTSRmyfH5oiL7x79gwJDjjjZBhjBLcTSZl6aI/HKb yEkVSgAgq38actwXqhfluDf2qLTbFZpiRfD9ZcA6oVfQ8F/vMl7D7N7GxFgKKOcUXiMA v9KDhMKeFsSPdMtcl8jB8Bxg7CyRJROFP5bKOo5iiWAi3nEtBxFhYO1BQP89oCnBUwHG 3P5oAMnY7L7JGEjUo88AnfZcj2y9hK6aHefTO6AET48ZU/Zlm+tDmuhEphWbnohaJp4t xcUw==
MIME-Version: 1.0
X-Received: by 10.152.133.52 with SMTP id oz20mr3895304lab.30.1360308224159; Thu, 07 Feb 2013 23:23:44 -0800 (PST)
Received: by 10.112.81.5 with HTTP; Thu, 7 Feb 2013 23:23:43 -0800 (PST)
In-Reply-To: <CAOeYYRf6V+oh3ZULG1458wLL-YpgzBJbyQEPiqdo5t_snT=aZQ@mail.gmail.com>
References: <CAA4WUYjiBZpShKKFfHQnixc94aOLrck0oR4ykARB=hF5h8nkfA@mail.gmail.com> <3430.1359961022@critter.freebsd.dk> <510F72CE.8030003@treenet.co.nz> <CAA4WUYiBJrLjM0-vurFOuJfUaabXtK=W8N5z28yshSfrvD9crg@mail.gmail.com> <1516.1360002578@critter.freebsd.dk> <42A54D15-0AA3-4172-94F7-E94C86E84D7F@niven-jenkins.co.uk> <2346.1360010079@critter.freebsd.dk> <CAA4WUYiyu+JvFuKooqa4xVdCJP=Mngu9dgHjhH99_SEac1kCZQ@mail.gmail.com> <CAOeYYRf6V+oh3ZULG1458wLL-YpgzBJbyQEPiqdo5t_snT=aZQ@mail.gmail.com>
Date: Thu, 07 Feb 2013 23:23:43 -0800
Message-ID: <CAP+FsNdg+7nTtvofK-h8P+3-YwfH4+fKWWGLefZVX_Rb_4aoBQ@mail.gmail.com>
From: Roberto Peon <grmocg@gmail.com>
To: Ashok Kumar <ashokkumar.j@gmail.com>
Cc: "William Chan (陈智昌)" <willchan@chromium.org>, Poul-Henning Kamp <phk@phk.freebsd.dk>, Ben Niven-Jenkins <ben@niven-jenkins.co.uk>, Amos Jeffries <squid3@treenet.co.nz>, HTTP Working Group <ietf-http-wg@w3.org>
Content-Type: multipart/alternative; boundary="f46d0435c24c7c6fce04d531702a"
Received-SPF: pass client-ip=209.85.217.176; envelope-from=grmocg@gmail.com; helo=mail-lb0-f176.google.com
X-W3C-Hub-Spam-Status: No, score=-3.5
X-W3C-Hub-Spam-Report: AWL=-2.694, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001
X-W3C-Scan-Sig: maggie.w3.org 1U3iJf-0006ab-Ey 8aa66a3feae8fd68ce46d5bf483b9ff8
X-Original-To: ietf-http-wg@w3.org
Subject: Re: http/2 prioritization/fairness bug with proxies
Archived-At: <http://www.w3.org/mid/CAP+FsNdg+7nTtvofK-h8P+3-YwfH4+fKWWGLefZVX_Rb_4aoBQ@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/16459
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

It should always keep the pipe full, but it should be using small enough
frames that it can respond with alacrity to a higher priority resource.
So, in generally, the algorithm should be something like:

WriteToAStream():
  for priority-bucket in active-priority-list:
    for stream in priority-bucket:
      if is_readable(stream):
        outbound.WriteData(stream, 2**14)
        return

other variations are possible, but that is the basic idea.


On Thu, Feb 7, 2013 at 9:50 PM, Ashok Kumar <ashokkumar.j@gmail.com> wrote:

> On Feb 4, 2013 8:05 AM, "Amos Jeffries" <squid3 <squid3@treenet.co.nz>@<squid3@treenet.co.nz>
> treenet.co.nz <squid3@treenet.co.nz>> wrote:
> > Priority 1 - user actioned fetch - requires fast answer
> > Priority 2 - background/automated fetch in user-visible window -
> requires fast answer but treat as bulk traffic.
> > Priority 3 - automated software fetch - treat as low-speed traffic.
> > Priority 4 - idle software probe - may drop if necessary.
>
> I still feel fixed priorities have better use, from a server point of
> view. The description may need to be tweaked a bit from the user-agent's
> needs perspective. A number will always have different meaning from
> different clients and server is not best place to guess who is more
> important.
>
> One tangential question, please spare me if this is not the right place.
> How is priority supposed to work from a server's perspective. Let's say
> Apache has a MaxClient of 1000. Will it stop accepting normal requests
> after it has used up 80% quota and will wait for higher priority requests?
> May be a trivial implementation detail but this has been bothering me for
> sometime and I thought this might help in deciding how priorities should be
> sent/interpreted.
>
>
>
>
> On Tue, Feb 5, 2013 at 7:17 AM, William Chan (陈智昌) <willchan@chromium.org>wrote:
>
>> I'm sorry if I am unclear in any way. Please continue to
>> challenge/question my comments/assertions so I can clarify my position
>> as appropriate.
>>
>> Just to be clear here, I stand by that it's a protocol bug currently.
>> I agree with adding more hooks to convey advisory priority semantics.
>> That said, "advisory" is open to interpretation. I agree that the
>> sender should ultimately be in control of how it orders responses, and
>> indeed there are of course many situations where it's best for the
>> sender to ignore the advisory priority. Yet, if the advisory priority
>> semantics are generally not respected, then clients will not be able
>> to rely on them, and will be forced to implement prioritization at a
>> higher layer, which suffers from the link underutilization vs
>> contention tradeoff I highlighted earlier.
>>
>> I appreciate the concern that we're adding complexity by introducing
>> new semantics. I am arguing that because the existing mechanisms for
>> addressing starvation are suboptimal, we should treat this as a
>> protocol bug and thus change the protocol in such a way as to fix this
>> problem. My suggestion for doing so was adding new priority "grouping"
>> semantics. I am hopeful that these new semantics will not introduce an
>> inordinate amount of specification, as the primary idea is that the
>> current SPDY priority levels would apply within a "group". I think we
>> can come up with a way to define a group that will be relatively easy
>> to spec.
>>
>> SPDY/4 introduces other prioritization semantics beyond just grouping,
>> but I wanted to focus on this one first, as I believe this is a bug
>> that we *need* to fix. The other SPDY/4 priority changes are of a
>> performance optimization nature, and I believe they will need to be
>> justified by data. I have no plans to raise them up in this group
>> until we have said data.
>>
>> On Tue, Feb 5, 2013 at 5:34 AM, Poul-Henning Kamp <phk@phk.freebsd.dk>
>> wrote:
>> > Content-Type: text/plain; charset=ISO-8859-1
>> > --------
>> > In message <42A54D15-0AA3-4172-94F7-E94C86E84D7F@niven-jenkins.co.uk>,
>> Ben Nive
>> > n-Jenkins writes:
>> >
>> >>So the idea is the protocol contains enough 'hooks' to sufficiently
>> >>express the different priorities between & within groups that folks
>> >>would like to express but isn't prescriptive about how anyone uses or
>> >>implements different prioritisation, scheduling, etc schemes.
>> >
>> > That was clearly not how the original poster presented it:
>> >
>> >         "I consider all those options as suboptimal, and thus
>> >         consider this issue to be a protocol bug. Our SPDY/4
>> >         prioritization proposal addresses this by [...]"
>> >
>> > --
>> > Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
>> > phk@FreeBSD.ORG         | TCP/IP since RFC 956
>> > FreeBSD committer       | BSD since 4.3-tahoe
>> > Never attribute to malice what can adequately be explained by
>> incompetence.
>>
>>
>
>
> --
> .- ... .... --- -.-
>