Re: HTTP router point-of-view concerns
Amos Jeffries <squid3@treenet.co.nz> Thu, 11 July 2013 10:10 UTC
Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 97CDF21F99DB for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 11 Jul 2013 03:10:45 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.553
X-Spam-Level:
X-Spam-Status: No, score=-10.553 tagged_above=-999 required=5 tests=[AWL=0.046, BAYES_00=-2.599, RCVD_IN_DNSWL_HI=-8]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S6nQmZXQchBi for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 11 Jul 2013 03:10:39 -0700 (PDT)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) by ietfa.amsl.com (Postfix) with ESMTP id F18CA21F99C2 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Thu, 11 Jul 2013 03:10:38 -0700 (PDT)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1UxDoa-00012W-4e for ietf-http-wg-dist@listhub.w3.org; Thu, 11 Jul 2013 10:09:32 +0000
Resent-Date: Thu, 11 Jul 2013 10:09:32 +0000
Resent-Message-Id: <E1UxDoa-00012W-4e@frink.w3.org>
Received: from lisa.w3.org ([128.30.52.41]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <squid3@treenet.co.nz>) id 1UxDoR-00011r-I2 for ietf-http-wg@listhub.w3.org; Thu, 11 Jul 2013 10:09:23 +0000
Received: from ip-58-28-153-233.static-xdsl.xnet.co.nz ([58.28.153.233] helo=treenet.co.nz) by lisa.w3.org with esmtp (Exim 4.72) (envelope-from <squid3@treenet.co.nz>) id 1UxDoP-0007UR-PH for ietf-http-wg@w3.org; Thu, 11 Jul 2013 10:09:23 +0000
Received: from [192.168.1.218] (ip202-27-218-168.satlan.co.nz [202.27.218.168]) by treenet.co.nz (Postfix) with ESMTP id 931CBE6F7F; Thu, 11 Jul 2013 22:08:57 +1200 (NZST)
Message-ID: <51DE8435.9010103@treenet.co.nz>
Date: Thu, 11 Jul 2013 22:08:53 +1200
From: Amos Jeffries <squid3@treenet.co.nz>
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130620 Thunderbird/17.0.7
MIME-Version: 1.0
To: Christian Parpart <trapni@gmail.com>
CC: ietf-http-wg@w3.org
References: <CA+qvzFPUpcm6kUtJx+rTw8Dpp4Gtx4Bmr3XPDhjNsjchUfN9_w@mail.gmail.com> <51DE1E32.9010801@treenet.co.nz> <CA+qvzFOrZEaaXKRYeaU=_X7vmemmjybGX9xLe3=VtEh6b8Nwgw@mail.gmail.com>
In-Reply-To: <CA+qvzFOrZEaaXKRYeaU=_X7vmemmjybGX9xLe3=VtEh6b8Nwgw@mail.gmail.com>
Content-Type: text/plain; charset="ISO-8859-1"; format="flowed"
Content-Transfer-Encoding: 7bit
Received-SPF: pass client-ip=58.28.153.233; envelope-from=squid3@treenet.co.nz; helo=treenet.co.nz
X-W3C-Hub-Spam-Status: No, score=-3.5
X-W3C-Hub-Spam-Report: AWL=-3.449, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001
X-W3C-Scan-Sig: lisa.w3.org 1UxDoP-0007UR-PH 22c758979668675a950c8395ff05bf62
X-Original-To: ietf-http-wg@w3.org
Subject: Re: HTTP router point-of-view concerns
Archived-At: <http://www.w3.org/mid/51DE8435.9010103@treenet.co.nz>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/18691
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>
On 11/07/2013 9:49 p.m., Christian Parpart wrote: > On Thu, Jul 11, 2013 at 4:53 AM, Amos Jeffries <squid3@treenet.co.nz > <mailto:squid3@treenet.co.nz>> wrote: > > On 11/07/2013 10:52 a.m., Christian Parpart wrote: > > Hey guys, > > I am just new to this list and I've just recently started > working through the HTTP/2 draft and all the blog articles > found about. > That being said, I myself have also very concerns others have > noticed as well, and would like to deeply show my intention to > help standing aside :) > > I am the implementor of one of the many HTTP servers that's > being used in production, and one major feature is HTTP > routing / load balancing, > and I would really love to implement an HTTP/1.1 successor > that is (to be) officially labeled HTTP/2[.0]. > > However, as a varnish [1] and a BSD [2] guy also raised there > hands on, is the lag of easy extraction of envelop information > of an HTTP request message, such as method, path, but most > certainly the host. > > Please forgive me if this is already on-topic somewhere > hidden, however, I would really highly encourage you to consider > adding some dedicated frame type for this kind of envelope > information. > > With that in mind, one might say that an HTTP/2 stream is not > initiated by a HEADERS frame but the ENVELOPE frame that > contains the actual > uncompressed and unmystified but compact information about > this request message. > > One humble proposal might indeed be: > > type: (something unassigned) > flags: bit 1 = END_STREAM /* this HTTP message is complete > with just these envelope frame, e.g. a simple GET, and no need > for user-agent etc. */ > id: unique stream ID, semantics like any other stream ID > body: a key/value table of the envelope data > > ENVELOPE FRAME DATA: > > The envelope data table is a simple table of key/value pairs > where the key is an 8bit value identifying the entry > and a variable length value that is interpreted depending on > the key. The list of provided envelope fields ends > as the end of the envelope frame has been reached. that means, > an envelope must always fit into a single (first) frame. > > ENVELOPE KEY/VALUE FIELDS: > > * :scheme => uint8: 0x01 > o http => uint8: 0x01 > o https => uint8: 0x02 > o custom => same as in method (if this is distinction is > really > demanded) > * :method => uint8: 0x02 > o GET => uint8: 0x01 > o POST => uint8: 0x02 > o PUT => uint8: 0x03 > o DELETE => uint8:0x04 > o custom => uint8: 0xFF, followed by one uint8 encoding > the size > > of the following bytes declaring the plaintext method > value, > e.g. "PROPFIND" > * :path => uint8: 0x03, uint16 length in network byte order, > > followed by $length octects declaring the path's value. > * :host => uint8: 0x04, uint16: length in network byte order, > > $length octets declaring the host's value. > * :route => uint8: 0x05, uint8: length, $length octets that > declare > > this value. (field is only specified if known, thus > previousely > announced by the remote server or this frame is part of a > response > and we are to announce a routing identifier) > * :status => uint8: 0x06, uint16: code in network byte order > /* if > > HTTP/2 considers starting response streams the same way */ > > This is the exact information an HTTP client > (scheme,method,path,host) or server (status) MUST currently > sent as part of the (first) HEADERS frame. > So the change I propose is, to extract this information from > the HEADERS frame and put it into its own frame that also > initiates the stream implicitly. > > Having this in mind, it is a pleasure to implement HTTP > routers because those now don't have to decode the full > HEADERS frames but just decode the ENVELOPE frame and pass any > continued frame to the directed next-hop server. > > > Sadly the compression draft as written is completely incompatible > with this type of load balancing. It operates a *stateful* > compressor, such that every single HEADERS frame being received > has to be decoded in order to re-encode using a separate > connection-specific stateful compressor on the next-hop > connections. This is mandatory for the compressed frames > regardless of whether the ENVELOPE header is used to provide > uncompressed details. > > > I hope we did not misunderstand each other, with "ENVELOPE" I meant a > frame on its own that 1.) initiates a stream and 2.) contains the most > important envelope information needed to route a message. I think I understand you. We considered something very similar early in the arguments against original SPDY design. Consider that the client may send the load banacer a set of HEADERS after your ENVELOPE block. One of these HEADERS (doesn't matter which one) contains a header which is added to the decompressor state table and then used by teh following HEADERS blocks. Your load balancer which does not do compression may send that HEADERS lock to server #1 and the others which depend on it to server #2, #3... etc. The net result is corrupted decompression state at the servers #2 etc, and over time unpredictable HTTP syntax being received and acted on by all servers. > The best load balancers can do under the current compression > draft is to avoid complex re-encoding by emiting only Literal > header representations and skipping all the traffic optimizations > compression offers. Which converts them into near perfect DDoS > bandwidth-amplification sources. > > > here I don't think that will be much of the case, since HTTP/2 header > encoding should still be a way smaller in bytes than HTTP/1.1 and prior. Only in compressed form. The Literal form is equivalent to non-compressed HTTP/1.1 syntax. So the bandwidth is being expanded from HTTP/2 size up to HTTP/1 size. Low input causing the relay to generate large output is the very definition of amplification. > > The sad state of affairs is that the *only* type of middleware > which benefits from the proposed HTTP/2 is those which performs > transparent interception and passive monitoring/recording of users > traffic (ie the worst kind). Anything which starts modifying or > manipulating (ie doing something useful for the ISP or CDN) MUST > implement a full compressor/decompressor pair in order to keep the > HTTP/2 statefulness in order. > > > If HTTP/2 now would just put the routing information (method, path, > host, scheme [, routing-key]) cleartext upfront (still binary) an > active HTTP router could just parse this and pass the remaining as-is > (e.g. unmodified) to the next-hop. Although, if it then still whishes > to add special headers (such as Via), it could do so via the trailing > HEADERS frame) - if I am not too wrong. See my example about the HEADERS block state inter-dependency. Amos
- Re: HTTP router point-of-view concerns Poul-Henning Kamp
- HTTP router point-of-view concerns Christian Parpart
- Re: HTTP router point-of-view concerns Amos Jeffries
- Re: HTTP router point-of-view concerns Roberto Peon
- Re: HTTP router point-of-view concerns Amos Jeffries
- Re: HTTP router point-of-view concerns Christian Parpart
- Re: HTTP router point-of-view concerns Amos Jeffries
- Re: HTTP router point-of-view concerns Michael Sweet
- Re: HTTP router point-of-view concerns Martin Thomson
- Re: HTTP router point-of-view concerns James M Snell
- Re: HTTP router point-of-view concerns Sam Pullara
- Re: HTTP router point-of-view concerns Roberto Peon
- Re: HTTP router point-of-view concerns Roberto Peon
- Re: HTTP router point-of-view concerns Amos Jeffries
- Re: HTTP router point-of-view concerns Sam Pullara
- Re: HTTP router point-of-view concerns Patrick McManus
- Re: HTTP router point-of-view concerns Roberto Peon
- Re: HTTP router point-of-view concerns James M Snell
- Re: HTTP router point-of-view concerns Roberto Peon
- Re: HTTP router point-of-view concerns James M Snell
- Re: HTTP router point-of-view concerns Roberto Peon
- Re: HTTP router point-of-view concerns Sam Pullara
- Re: HTTP router point-of-view concerns Roberto Peon
- Re: HTTP router point-of-view concerns Martin Thomson
- Re: HTTP router point-of-view concerns Poul-Henning Kamp
- Re: HTTP router point-of-view concerns Mark Nottingham
- Re: HTTP router point-of-view concerns Mike Belshe
- Re: HTTP router point-of-view concerns Gábor Molnár
- Re: HTTP router point-of-view concerns Gábor Molnár
- Re: HTTP router point-of-view concerns Poul-Henning Kamp
- Re: HTTP router point-of-view concerns Michael Sweet
- Re: HTTP router point-of-view concerns Christian Parpart
- Re: HTTP router point-of-view concerns Willy Tarreau
- Re: HTTP router point-of-view concerns Patrick McManus
- Re: HTTP router point-of-view concerns Jeff Pinner
- Re: HTTP router point-of-view concerns Martin Thomson
- Re: HTTP router point-of-view concerns Roberto Peon
- Re: HTTP router point-of-view concerns Ludin, Stephen
- Re: HTTP router point-of-view concerns Roberto Peon
- Re: HTTP router point-of-view concerns James M Snell
- Re: HTTP router point-of-view concerns Amos Jeffries
- Re: HTTP router point-of-view concerns Amos Jeffries
- Re: HTTP router point-of-view concerns Roberto Peon
- Re: HTTP router point-of-view concerns Yoav Nir
- Re: HTTP router point-of-view concerns Willy Tarreau
- Re: HTTP router point-of-view concerns Poul-Henning Kamp
- Re: HTTP router point-of-view concerns Poul-Henning Kamp
- Re: HTTP router point-of-view concerns Willy Tarreau
- Re: HTTP router point-of-view concerns Poul-Henning Kamp
- Re: HTTP router point-of-view concerns Sam Pullara
- Re: HTTP router point-of-view concerns Willy Tarreau
- Re: HTTP router point-of-view concerns Mark Delany
- Re: HTTP router point-of-view concerns Poul-Henning Kamp
- Re: HTTP router point-of-view concerns Poul-Henning Kamp
- Re: HTTP router point-of-view concerns Willy Tarreau
- Re: HTTP router point-of-view concerns Willy Tarreau
- Re: HTTP router point-of-view concerns Poul-Henning Kamp
- Re: HTTP router point-of-view concerns Willy Tarreau
- Re: HTTP router point-of-view concerns Yoav Nir
- Re: HTTP router point-of-view concerns Poul-Henning Kamp
- Re: HTTP router point-of-view concerns Yoav Nir
- Re: HTTP router point-of-view concerns Willy Tarreau
- Re: HTTP router point-of-view concerns Poul-Henning Kamp
- Re: HTTP router point-of-view concerns Stephen Farrell
- Re: HTTP router point-of-view concerns Willy Tarreau
- Re: HTTP router point-of-view concerns Sam Pullara
- Re: HTTP router point-of-view concerns Nicolas Mailhot
- Re: HTTP router point-of-view concerns Nicolas Mailhot
- Re: HTTP router point-of-view concerns Nicolas Mailhot
- Re: HTTP router point-of-view concerns Martin Nilsson
- Re: HTTP router point-of-view concerns Nico Williams
- Re: HTTP router point-of-view concerns Nico Williams
- Re: HTTP router point-of-view concerns Poul-Henning Kamp
- Re: HTTP router point-of-view concerns Nico Williams
- Re: HTTP router point-of-view concerns Poul-Henning Kamp
- Re: HTTP router point-of-view concerns Nico Williams