Return-Path: X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F1B3A21F8E4E for ; Tue, 28 May 2013 13:14:34 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -10.373 X-Spam-Level: X-Spam-Status: No, score=-10.373 tagged_above=-999 required=5 tests=[AWL=-0.075, BAYES_00=-2.599, HTML_MESSAGE=0.001, MIME_8BIT_HEADER=0.3, RCVD_IN_DNSWL_HI=-8] Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g2kGzNR6AhAs for ; Tue, 28 May 2013 13:14:28 -0700 (PDT) Received: from frink.w3.org (frink.w3.org [128.30.52.56]) by ietfa.amsl.com (Postfix) with ESMTP id 637FB21F8B35 for ; Tue, 28 May 2013 13:14:28 -0700 (PDT) Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from ) id 1UhQH2-0007t8-Vs for ietf-http-wg-dist@listhub.w3.org; Tue, 28 May 2013 20:13:37 +0000 Resent-Date: Tue, 28 May 2013 20:13:36 +0000 Resent-Message-Id: Received: from maggie.w3.org ([128.30.52.39]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from ) id 1UhQGp-0007sF-Vl for ietf-http-wg@listhub.w3.org; Tue, 28 May 2013 20:13:24 +0000 Received: from mail-ob0-f173.google.com ([209.85.214.173]) by maggie.w3.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.72) (envelope-from ) id 1UhQGk-0006cX-Vt for ietf-http-wg@w3.org; Tue, 28 May 2013 20:13:23 +0000 Received: by mail-ob0-f173.google.com with SMTP id wc20so1883003obb.18 for ; Tue, 28 May 2013 13:12:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=KYmxamdVd9HhZrKsLyHlDAjrGs6YkTA9tslsV/MiiLc=; b=oqZWltGgb9CLeGhfgSaAwVF9vmkRLANBQydfM4U62g9uMEDNRJ4jX/jDd/K2AkQxVp KswD4sheXLkk+Q11sf8WdjWfIZ0ISdBslmx4Ol4MbCw5cr/cZ2/k07hEm1xQFzzt3hOl FpPgvBM9Ns7Jr7ZGK6kJ3BWeF05tgXdevnTZtwlGLSVo5lpD4cH+a2UDPWQpRrNIkXGN +QSJNwl41l6eTUX31Cdnr0iqFuT715snC10FHHhZbPryTTW8P8tV8CW6tx4Slh/omW5V Nqik82VXa8Mfvm/UUPJeCklHRAAZehRP7MMEAhVOKu0iZue+TYq1AkqYtB7xd7b4KlB/ I/ag== MIME-Version: 1.0 X-Received: by 10.182.153.5 with SMTP id vc5mr21879364obb.32.1369771973074; Tue, 28 May 2013 13:12:53 -0700 (PDT) Received: by 10.76.169.68 with HTTP; Tue, 28 May 2013 13:12:53 -0700 (PDT) In-Reply-To: References: Date: Tue, 28 May 2013 13:12:53 -0700 Message-ID: From: Roberto Peon To: =?UTF-8?B?V2lsbGlhbSBDaGFuICjpmYjmmbrmmIwp?= Cc: James M Snell , HTTP Working Group , Patrick McManus Content-Type: multipart/alternative; boundary=089e013a0adae0e4b204ddcce3f9 Received-SPF: pass client-ip=209.85.214.173; envelope-from=grmocg@gmail.com; helo=mail-ob0-f173.google.com X-W3C-Hub-Spam-Status: No, score=-3.5 X-W3C-Hub-Spam-Report: AWL=-2.677, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001 X-W3C-Scan-Sig: maggie.w3.org 1UhQGk-0006cX-Vt e6690ecaef72d80330c8a6f21c4de9df X-Original-To: ietf-http-wg@w3.org Subject: Re: Proposal - Reduce HTTP2 frame length from 16 to 12 bits Archived-At: Resent-From: ietf-http-wg@w3.org X-Mailing-List: archive/latest/18124 X-Loop: ietf-http-wg@w3.org Resent-Sender: ietf-http-wg-request@w3.org Precedence: list List-Id: List-Help: List-Post: List-Unsubscribe: --089e013a0adae0e4b204ddcce3f9 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable responses inline On Tue, May 28, 2013 at 12:16 PM, William Chan (=E9=99=88=E6=99=BA=E6=98=8C= ) wrote: > On Tue, May 28, 2013 at 11:50 AM, James M Snell wrote= : > >> On Tue, May 28, 2013 at 11:41 AM, Roberto Peon wrote: >> > As a reverse proxy, I've seen properties for which 4k writes/reads wer= e >> too >> > small and induced latency increases. >> > >> >> I haven't played with this part too much yet but this is my general >> suspicion also. >> > > Can you guys clarify this in more detail? Specifically, where the latency > comes from. I have ideas, but I'd rather than an authoritative explanatio= n. > It always comes down to the cost of the context switches (i.e. syscalls) and the locking that must be done in the lower layers of the IO stack. > >> >> > Admittedly, frame size doesn't have to be the same as read/write size, >> but >> > it certainly does encourage that implementation (which is, I think, th= e >> > point of smaller max frame size that you proposed). >> > > You're right that it does encourage that implementation. Just like a > larger length encourages just naively breaking up frames into that max > frame size and thus hurt responsiveness. Which one is likelier to cause > worse overall "performance" (I know this is vague, since people care abou= t > different aspects of perf)? What we want to do is have the most reasonabl= e > default behavior, with the ability for performant implementations to tune > without unreasonable difficulty. I believe we're mostly focusing here on > optimizing the naive implementations, not the highly tuned implementation= s. > Remember that I'm the one who proposed the smaller max frame size in the first place (now a fair while ago)? :) My sweet-spot number was 16k, as I knew that I could saturate a 10G nic with 16k frames/writes and have enough CPU left over to do some actual work. The amount of overhead goes up more than linearly with the decrease in frame size thanks to contention, etc. > > >> > >> > I propose we keep the 16 bit frame size and instead allow the (now >> > negotiated setting of) max frame size to default to 12 bits worth, wit= h >> that >> > going upwards out downwards when a settings frame arrives from the oth= er >> > side indicating it's max receive size. HK >> > >> >> Honestly, I'd prefer to do away with frame size negotiation altogether >> because of the potential for path mtu style issues. Keeping the 16-bit >> size for now with strong encouragement (SHOULD, perhaps?) for keeping >> sizes around 12-bit lengths for the most common cases seems like the >> right approach. >> >> -- James >> > Unlike TCP/IP, max frame size is a point-to-point thing, as the primitive we mux is streams, not frames. Frames are the way we accomplish the muxing. Why would there be any path MTU like thing? -=3DR > >> > This would give the best chance that the code would be written in such >> a way >> > as to adapt with the times as they change. >> > -=3DR >> > >> > On May 28, 2013 10:01 AM, "William Chan (=E9=99=88=E6=99=BA=E6=98=8C)"= >> > wrote: >> >> >> >> Can you clarify what you mean by a documented performance metric for >> >> non-browser use cases? I don't think Patrick said anything browser >> specific. >> >> He provided some serialization latency numbers and noted that they ar= e >> high >> >> enough to impact responsiveness. And then he provided numbers on >> overhead. >> >> >> >> I, for one, find the responsiveness argument compelling for browsers. >> I'm >> >> not completely sure 0.2% is low enough overhead for everyone, but I >> wouldn't >> >> complain about it. And in absence of complaints, I guess I'd support >> moving >> >> forward with only 12 bits for length. >> >> >> >> >> >> On Tue, May 28, 2013 at 9:22 AM, James M Snell >> wrote: >> >>> >> >>> Currently, my only challenge with this is that, so far, we have not >> >>> seen any documented performance metrics for non-browser based uses. >> >>> .That said, I don't really have the time currently to put together a >> >>> comprehensive set of such metrics so it wouldn't be polite of me to >> >>> insist on them ;-) ... perhaps for now we ought to keep the 16-bit >> >>> size but include a recommendation about not exceeding 12-bits, then >> >>> see what more implementation experience does for us. >> >>> >> >>> On Tue, May 28, 2013 at 7:20 AM, Patrick McManus < >> mcmanus@ducksong.com> >> >>> wrote: >> >>> > Hi All, >> >>> > >> >>> > I've been looking at a lot of spdy frames lately, and I've noticed >> what >> >>> > I >> >>> > consider a common implementation problem that I think a good http/= 2 >> >>> > spec >> >>> > could help with. I'm commonly seeing frames large enough to >> interfere >> >>> > with >> >>> > effective prioritization. I've seen this from at least 3 different >> >>> > servers. >> >>> > >> >>> > The HTTP/2 draft has a max frame size of 16 bits, which is a huge >> >>> > improvement from spdy's 24. I propose we reduce it further to 12. >> (i.e. >> >>> > 4096 >> >>> > bytes). >> >>> > >> >>> > The muxxed approach of multiple streams onto one connection done i= n >> >>> > HTTP/2 >> >>> > has great advantages, but the one downside of it is that it create= s >> >>> > head of >> >>> > line blocking problems between those streams dictated by frame >> >>> > granularity. >> >>> > With small frames this is pretty manageable, with extremely large >> ones >> >>> > we've >> >>> > recreated the same head of line problems that HTTP/1 pipelines hav= e. >> >>> > The >> >>> > server needs to be able to respond quickly to higher priority >> events >> >>> > (including cancellations) and once it has written a frame header t= o >> the >> >>> > wire >> >>> > it is committed to the entire frame for how ever long it takes to >> >>> > serialize >> >>> > it. IMO the shorter that time, the better. >> >>> > >> >>> > Our spec can help implementations do the right thing here by >> limiting >> >>> > the >> >>> > max frame size to 12 bits. >> >>> > >> >>> > It takes 500msec to serialize 64KB at 1Mbit/sec... 125msec at >> >>> > 4Mbit/sec. >> >>> > Those are some pretty notable task-switch times. Dropping the fram= e >> to >> >>> > 4096 >> >>> > cuts them to 32msec and 8 msec.. that's much more responsive, at t= he >> >>> > cost of >> >>> > 120 extra bytes of transfer (< 1msec at 1Mbit/sec). >> >>> > >> >>> > In general - the smaller the better as long as the overhead doesn'= t >> get >> >>> > to >> >>> > be too large. At 8 in 4096 (~.2%) I think that's acceptable. Its >> >>> > roughly the >> >>> > same overhead as a VLAN tag. >> >>> > >> >>> > Obviously this makes a continuation bit for control frames >> absolutely >> >>> > mandatory, but I think we're already in that spot with 16 bit fram= e >> >>> > lengths. >> >>> > >> >>> > -Patrick >> >>> > >> >>> > >> >>> >> >> >> > >> > > --089e013a0adae0e4b204ddcce3f9 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
responses inline


On Tue, May 28, 2013 at 12:16 PM, William Chan (=E9= =99=88=E6=99=BA=E6=98=8C) <willchan@chromium.org> wrote:=
On Tue, May 28, 2013 a= t 11:50 AM, James M Snell <jasnell@gmail.com> wrote:
On Tue, May 28, 2013 at 11:41 AM, Rober= to Peon <grmocg@gm= ail.com> wrote:
> As a reverse proxy, I've seen properties for which 4k writes/reads= were too
> small and induced latency increases.
>

I haven't played with this part too much yet but this is my gener= al
suspicion also.

Can you guys clar= ify this in more detail? Specifically, where the latency comes from. I have= ideas, but I'd rather than an authoritative explanation.

It always comes down to = the cost of the context switches (i.e. syscalls) and the locking that must = be done in the lower layers of the IO stack.

=C2=A0

> Admittedly, frame size doesn't have to be the same as read/write s= ize, but
> it certainly does encourage that implementation (which is, I think, th= e
> point of smaller max frame size that you proposed).

You're right that it does encourage that = implementation. Just like a larger length encourages just naively breaking = up frames into that max frame size and thus hurt responsiveness. Which one = is likelier to cause worse overall "performance" (I know this is = vague, since people care about different aspects of perf)? What we want to = do is have the most reasonable default behavior, with the ability for perfo= rmant implementations to tune without unreasonable difficulty. I believe we= 're mostly focusing here on optimizing the naive implementations, not t= he highly tuned implementations.

Remember that I= 9;m the one who proposed the smaller max frame size in the first place (now= a fair while ago)? :)
My sweet-spot number was 16k, as I k= new that I could saturate a 10G nic with 16k frames/writes and have enough = CPU left over to do some actual work. The amount of overhead goes up more t= han linearly with the decrease in frame size thanks to contention, etc.
=C2=A0
=C2=A0
>
> I propose we keep the 16 bit frame size and instead allow the (now
> negotiated setting of) max frame size to default to 12 bits worth, wit= h that
> going upwards out downwards when a settings frame arrives from the oth= er
> side indicating it's max receive size. HK
>

Honestly, I'd prefer to do away with frame size negotiation altog= ether
because of the potential for path mtu style issues. Keeping the 16-bit
size for now with strong encouragement (SHOULD, perhaps?) for keeping
sizes around 12-bit lengths for the most common cases =C2=A0seems like the<= br> right approach.

-- James

Unlike TCP/IP, max frame size is a point-t= o-point thing, as the primitive we mux is streams, not frames. Frames are t= he way we accomplish the muxing.
Why would there be any path MTU like thing?

=
-=3DR
=C2=A0

> This would give the best chance that the code would be written in such= a way
> as to adapt with the times as they change.
> -=3DR
>
> On May 28, 2013 10:01 AM, "William Chan (=E9=99=88=E6=99=BA=E6=98= =8C)" <w= illchan@chromium.org>
> wrote:
>>
>> Can you clarify what you mean by a documented performance metric f= or
>> non-browser use cases? I don't think Patrick said anything bro= wser specific.
>> He provided some serialization latency numbers and noted that they= are high
>> enough to impact responsiveness. And then he provided numbers on o= verhead.
>>
>> I, for one, find the responsiveness argument compelling for browse= rs. I'm
>> not completely sure 0.2% is low enough overhead for everyone, but = I wouldn't
>> complain about it. And in absence of complaints, I guess I'd s= upport moving
>> forward with only 12 bits for length.
>>
>>
>> On Tue, May 28, 2013 at 9:22 AM, James M Snell <jasnell@gmail.com> wrote: >>>
>>> Currently, my only challenge with this is that, so far, we hav= e not
>>> seen any documented performance metrics for non-browser based = uses.
>>> .That said, I don't really have the time currently to put = together a
>>> comprehensive set of such metrics so it wouldn't be polite= of me to
>>> insist on them ;-) ... perhaps for now we ought to keep the 16= -bit
>>> size but include a recommendation about not exceeding 12-bits,= then
>>> see what more implementation experience does for us.
>>>
>>> On Tue, May 28, 2013 at 7:20 AM, Patrick McManus <mcmanus@ducksong.com= >
>>> wrote:
>>> > Hi All,
>>> >
>>> > I've been looking at a lot of spdy frames lately, and= I've noticed what
>>> > I
>>> > consider a common implementation problem that I think a g= ood http/2
>>> > spec
>>> > could help with. I'm commonly seeing frames large eno= ugh to interfere
>>> > with
>>> > effective prioritization. I've seen this from at leas= t 3 different
>>> > servers.
>>> >
>>> > The HTTP/2 draft has a max frame size of 16 bits, which i= s a huge
>>> > improvement from spdy's 24. I propose we reduce it fu= rther to 12. (i.e.
>>> > 4096
>>> > bytes).
>>> >
>>> > The muxxed approach of multiple streams onto one connecti= on done in
>>> > HTTP/2
>>> > has great advantages, but the one downside of it is that = it creates
>>> > head of
>>> > line blocking problems between those streams dictated by = frame
>>> > granularity.
>>> > With small frames this is pretty manageable, with extreme= ly large ones
>>> > we've
>>> > recreated the same head of line problems that HTTP/1 pipe= lines have.
>>> > The
>>> > server needs to =C2=A0be able to respond quickly to highe= r priority events
>>> > (including cancellations) and once it has written a frame= header to the
>>> > wire
>>> > it is committed to the entire frame for how ever long it = takes to
>>> > serialize
>>> > it. IMO the shorter that time, the better.
>>> >
>>> > Our spec can help implementations do the right thing here= by limiting
>>> > the
>>> > max frame size to 12 bits.
>>> >
>>> > It takes 500msec to serialize 64KB at 1Mbit/sec... 125mse= c at
>>> > 4Mbit/sec.
>>> > Those are some pretty notable task-switch times. Dropping= the frame to
>>> > 4096
>>> > cuts them to 32msec and 8 msec.. that's much more res= ponsive, at the
>>> > cost of
>>> > 120 extra bytes of transfer (< 1msec at 1Mbit/sec). >>> >
>>> > In general - the smaller the better as long as the overhe= ad doesn't get
>>> > to
>>> > be too large. At 8 in 4096 (~.2%) I think that's acce= ptable. Its
>>> > roughly the
>>> > same overhead as a VLAN tag.
>>> >
>>> > Obviously this makes a continuation bit for control frame= s absolutely
>>> > mandatory, but I think we're already in that spot wit= h 16 bit frame
>>> > lengths.
>>> >
>>> > -Patrick
>>> >
>>> >
>>>
>>
>


--089e013a0adae0e4b204ddcce3f9--