Re: HTTP/2 and Websockets
Yutaka Hirano <yhirano@google.com> Fri, 21 November 2014 06:44 UTC
Return-Path: <ietf-http-wg-request@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 450971AD0D3 for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 20 Nov 2014 22:44:59 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.973
X-Spam-Level:
X-Spam-Status: No, score=-6.973 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FM_FORGED_GMAIL=0.622, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RP_MATCHES_RCVD=-0.594, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SvQYmhP4s8qr for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Thu, 20 Nov 2014 22:44:54 -0800 (PST)
Received: from frink.w3.org (frink.w3.org [128.30.52.56]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 3C5AE1A0076 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Thu, 20 Nov 2014 22:44:53 -0800 (PST)
Received: from lists by frink.w3.org with local (Exim 4.72) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1Xrhuo-0005hE-PS for ietf-http-wg-dist@listhub.w3.org; Fri, 21 Nov 2014 06:41:58 +0000
Resent-Date: Fri, 21 Nov 2014 06:41:58 +0000
Resent-Message-Id: <E1Xrhuo-0005hE-PS@frink.w3.org>
Received: from maggie.w3.org ([128.30.52.39]) by frink.w3.org with esmtp (Exim 4.72) (envelope-from <yhirano@google.com>) id 1Xrhue-0005ek-UT for ietf-http-wg@listhub.w3.org; Fri, 21 Nov 2014 06:41:48 +0000
Received: from mail-yh0-f50.google.com ([209.85.213.50]) by maggie.w3.org with esmtps (TLS1.0:RSA_ARCFOUR_SHA1:16) (Exim 4.72) (envelope-from <yhirano@google.com>) id 1Xrhub-0001V5-KN for ietf-http-wg@w3.org; Fri, 21 Nov 2014 06:41:48 +0000
Received: by mail-yh0-f50.google.com with SMTP id 29so2275016yhl.9 for <ietf-http-wg@w3.org>; Thu, 20 Nov 2014 22:41:19 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ZqVVycc2MQsmNJShoAro3sWZHhxczIxavdgiQ9SSrZM=; b=jw8qqTVlNAk/127ZaDAwmSsOxatbHvM2XR39Yif1h/+yn6MoLdiAN1JBtMLErXIaSF AH5yB0Z12sSAKZDigy/DaiHrgjlsBd9IP2suNsg8GIbsytwShQ50HKXeUaWFWQuoPEDf HWGNyg6ssKf/DOaEbG9Gfb1s2VOxdOBDqR3dwWVXq3clSeJwP8LQWQT3Dyr5967Cnprg EiDrkoMr4ZdL2bCoPIg4iNt1RDT9+GlynfvV/U3LB7GHTTQ8LuJM8IuzLbxxQ7LgLyXl vz2nMphvaIQGi+0PazX8Emf10BMg0DhzftIV3gzs6VdlilU0fBX/KiF6yOF2GqSU3Qbb E8Mw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=ZqVVycc2MQsmNJShoAro3sWZHhxczIxavdgiQ9SSrZM=; b=IwMNzx5ZMeQKC8SIA1tplbXd55KcpRYvJg89lqpJ/4Nc3XhYD0n9v6qjhbdb2PLVQe 5UmwCRmddKoZC7D1IhXjm/sWn4JCKxV3HFRWTrvVgQOvPTzF/Kh7BkWaLtzuwO4RGOqf BvVLumA1p4Nc+itw/fjoQahGLngGRBQH7zq2ror5j/JToK0NfEp9EqovOv6qFJYX69Z7 0tXn2so3riqK9d6ghbL8I+JM/MXxrmCNR/2sb6+IYRqwlV9M4Swaln/7gBfoawLFy8zx itoOPBITwy0/Wqi8ooNT1EHOW6IWSb39an9mkFVcvn91sp421gPesZLsdMYZh0B5XyMc NS9w==
X-Gm-Message-State: ALoCoQlScITgVI25SHKbDgtGh3igiFclkDv36pQnV0z9RTqAc3urCNLiCI+QoXxJWD0ThvIKLYBe
MIME-Version: 1.0
X-Received: by 10.236.41.138 with SMTP id h10mr289039yhb.182.1416552079681; Thu, 20 Nov 2014 22:41:19 -0800 (PST)
Received: by 10.170.196.14 with HTTP; Thu, 20 Nov 2014 22:41:19 -0800 (PST)
In-Reply-To: <7597002B-B6DE-49FC-9B25-741D5130569F@warmcat.com>
References: <CAJ3HoZ1gHH1MmY=NY68HBcFV7u74qKtkdWDe4i93MkjnXE+sPg@mail.gmail.com> <CABihn6GAu7pViZ3JxHAm1-SOeFLvxOQAuhrzwROGGY2jiCg4Cg@mail.gmail.com> <CAJ3HoZ3wfFMT2=duF22FyQGgdPzJwfmSSw0t82c9uvTkcsm1wg@mail.gmail.com> <CABihn6FwK7uZpg6v7u9byTw6xNsTjFsWfv_QyPMReTtumaXJWA@mail.gmail.com> <542BD960.7020407@treenet.co.nz> <CAJ3HoZ1B5hGrhzAnVYnDbYWV_BPmqMKsOhvM7VvS+2xFNT7krw@mail.gmail.com> <543D024D.2070001@treenet.co.nz> <CAJ3HoZ0aRLhB4Pf3zZobrzZYqYK8FguqMpNEvqoJ82FyM_XKHg@mail.gmail.com> <6D965D58-91C9-4DE0-AA2F-7559C0594368@warmcat.com> <CABihn6FPAWKABgJKHYNg5f1hOZsUTngKnhBkaB6cCNkG75cWCw@mail.gmail.com> <53882B6B-244E-4ADF-95DB-2E9DC4542C21@warmcat.com> <CABihn6E9OBTxSVvLHmhiwXcmwrDCo8w2DbYJwaNCaYvjSjUZqw@mail.gmail.com> <7597002B-B6DE-49FC-9B25-741D5130569F@warmcat.com>
Date: Fri, 21 Nov 2014 15:41:19 +0900
Message-ID: <CABihn6Fuxk13djry5FdRX+SUXGpUE01k6PrCn3oohFpsiT1EqA@mail.gmail.com>
From: Yutaka Hirano <yhirano@google.com>
To: Andy Green <andy@warmcat.com>
Cc: Robert Collins <robertc@robertcollins.net>, Amos Jeffries <squid3@treenet.co.nz>, HTTP Working Group <ietf-http-wg@w3.org>
Content-Type: multipart/alternative; boundary="089e013a2622840f0f050858bb02"
Received-SPF: pass client-ip=209.85.213.50; envelope-from=yhirano@google.com; helo=mail-yh0-f50.google.com
X-W3C-Hub-Spam-Status: No, score=-4.2
X-W3C-Hub-Spam-Report: AWL=-1.516, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01
X-W3C-Scan-Sig: maggie.w3.org 1Xrhub-0001V5-KN d0492fbdbf560cd3355f9c590f92a479
X-Original-To: ietf-http-wg@w3.org
Subject: Re: HTTP/2 and Websockets
Archived-At: <http://www.w3.org/mid/CABihn6Fuxk13djry5FdRX+SUXGpUE01k6PrCn3oohFpsiT1EqA@mail.gmail.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/28079
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>
On Fri, Nov 21, 2014 at 12:38 PM, Andy Green <andy@warmcat.com> wrote: > > > On 21 November 2014 11:30:46 GMT+08:00, Yutaka Hirano <yhirano@google.com> > wrote: > >On Fri, Nov 21, 2014 at 11:42 AM, Andy Green <andy@warmcat.com> wrote: > > > >> > >> > >> On 21 November 2014 10:02:28 GMT+08:00, Yutaka Hirano > ><yhirano@google.com> > >> wrote: > >> >> > >> >> I think you have to take the atomic large ws frame thing as a > >genuine > >> >> problem because http2 has the transmit credit concept. So even if > >> >you > >> >> buffered one ws frame, you can't sit there spewing as much http2 > >DATA > >> >frame > >> >> as it needs to atomically encapsulate it, without sizing your > >http2 > >> >frame > >> >> to suit the tx credit. > >> >> But it's OK because the implementation can transparently fragment > >the > >> >ws > >> >> data using the ws message semantics... I think there's no choice > >but > >> >to > >> >> take that approach. > >> > > >> >Sorry I don't understand what you are proposing. Can you explain? > >> > >> I'm agreeing with what was already written by someone else on the > >thread. > >> > >> Talking about buffering huge ws frames until you have enough to issue > >it > >> all in one big http2 DATA frame will not fly. > >> > >> If you're using this putative ws-over-http2 scheme, and you get given > >a > >> huge ws frame to transmit, you should fragment it using RFC6455 > >message > >> semantics to some implementation-defined limit that is friendly for > >mux'd > >> http2 transport. > >> > >Thanks. > > > >Strictly speaking, RFC6455 allows an extension to give meaning to > >WebSocket > >frames, so merging / fragmenting frames breaks such extensions. > >We discussed this problem in HyBi and many of us said "don't care". > > Yeah extensions except for compression have just not come into existence. > > So I also see it as don't care. I'm not even sure it's true since the > intention during ws discussion was an intermediary can fragment frames same > as how tcp packets may be fragmented. > > >In any case, an http/2 frame cannot be bigger than 2^24 (or 2^14 > >without an > >explicit permission), so I think we don't have to worry about DoS. > > ... I don't see that. If I keep spamming 16MB frames even on one stream > on a consumer link your latency goes to pieces and if something is doing > multiple instances of it even a big pipe will feel pain. > > The spec says: Length: The length of the frame payload expressed as an unsigned 24-bit integer. Values greater than 2^14 (16,384) MUST NOT be sent unless the receiver has set a larger value for SETTINGS_MAX_FRAME_SIZE. So by default 2^14 is the upper limit. Larger frames will be used only if the receiver wants. > >> >I guess it can be done in parallel with http2 coming to an end > >rather > >> >than > >> >> trying to block it, just by defining some new optional frame > >types... > >> > > >> >I agree to define a ws-dedicated frame type and use it. > >> > >> Super... has anyone proposed how to map RFC6455 to http2 framing in > >detail > >> yet? > >> > >I think not. > >I did list several ways at [1], but I deleted it from the next > > Alright I will go look at these. > > The discussion is moribund somebody should probably issue a 'stalking > horse' since it's easier for people to jump in and tell you you're doing it > wrong ^^ > > -Andy > > >version[2] > >because HTTP/2 situation had changed. > > > >1: > >http://tools.ietf.org/html/draft-hirano-httpbis-websocket-over-http2-00 > >2: > >http://tools.ietf.org/html/draft-hirano-httpbis-websocket-over-http2-01 > > > > > >> > >> -Andy > >> > >> > > >> >On Fri, Nov 21, 2014 at 10:47 AM, Andy Green <andy@warmcat.com> > >wrote: > >> > > >> >> > >> >> > >> >> On 21 November 2014 04:11:53 GMT+08:00, Robert Collins < > >> >> robertc@robertcollins.net> wrote: > >> >> >On 15 October 2014 00:00, Amos Jeffries <squid3@treenet.co.nz> > >> >wrote: > >> >> >> -----BEGIN PGP SIGNED MESSAGE----- > >> >> >> Hash: SHA1 > >> >> >> > >> >> >> On 14/10/2014 11:01 p.m., Robert Collins wrote: > >> >> >>> On 1 October 2014 23:37, Amos Jeffries wrote: > >> >> >>> > >> >> >>>>> All the implementor discussion I've seen during the > >> >> >>>>>> HTTP/2 discussions has focused on how intermediaries want > >to > >> >> >>>>>> be scalable: and buffering is anti-scaling. So - is it a > >> >> >>>>>> pragmatic concern, or do we expect DATA stream buffering to > >> >> >>>>>> take place [outside of protocol gateways converting to > >> >> >>>>>> HTTP/1.1 where non upload can require buffering - and note > >> >> >>>>>> that such a gateway can't carry ws anyway unless its aware > >of > >> >> >>>>>> it, and if its aware of it, it can make sure it does not > >> >> >>>>>> buffer]. > >> >> >>>> > >> >> >>>> > >> >> >>>> I think the problem is not buffering in HTTP/2 per-se but the > >> >> >>>> DATA frame (de-)aggregation that can happen if the frames are > >> >> >>>> buffered by general network conditions (ie in TCP > >bottlenecks). > >> >> >>>> This would not be good for a 1:1 relationship between DATA > >and > >> >ws > >> >> >>>> frames. > >> >> >>>> > >> >> >>>> Amos > >> >> >>> > >> >> >>> So hang on a second here. If we say that ws frames can't be > >split > >> >> >>> over multiple HTTP/2 frames that implies that we have to > >buffer > >> >> >>> them until there is enough in the window to transmit a > >> >potentially > >> >> >>> very large packet all at once. It also conflicts with RFC6455 > >- > >> >the > >> >> >>> specific intent there is to not be a stream based system. > >> >> >> > >> >> >> If a ws frame *has* to be that long, not doing so would block > >the > >> >> >> entire HTTP/2 connection until all bytes of that frame were > >> >delivered > >> >> >> anyway. So you trade off buffering that single frame at the > >> >sender, > >> >> >> versus blocking all HTTP/2 traffic end-to-end. > >> >> >> > >> >> >> If the ws data is so critical to get transmitted fast why is > >that > >> >> >> single ws frame so large to begin with? surely it would be > >> >> >transmitted > >> >> >> faster as a sequence of WS + *WSDATA frames emited as the > >payload > >> >was > >> >> >> available to send. > >> >> > > >> >> >I agree that its inconsistent which is why I don't think it > >matters > >> >> > >> >> I am the author of libwebsockets, we are adding http2 support at > >the > >> >> moment. The basic http2 serving is done and works for http, but > >> >we're all > >> >> dressed up and nowhere to go in terms of treating websocket > >> >connections as > >> >> just another kind of http2, since the framing is "TBD". I am > >sorry I > >> >am a > >> >> bit late to the party. > >> >> > >> >> I think you have to take the atomic large ws frame thing as a > >genuine > >> >> problem because http2 has the transmit credit concept. So even if > >> >you > >> >> buffered one ws frame, you can't sit there spewing as much http2 > >DATA > >> >frame > >> >> as it needs to atomically encapsulate it, without sizing your > >http2 > >> >frame > >> >> to suit the tx credit. > >> >> > >> >> But it's OK because the implementation can transparently fragment > >the > >> >ws > >> >> data using the ws message semantics... I think there's no choice > >but > >> >to > >> >> take that approach. > >> >> > >> >> Otherwise you get into being able to DoS even an http2 "big pipe > >> >> aggregation" by just one mux element spewing an endless ws frame > >and > >> >> blocking every other mux'd connection... it cannot be right. > >> >> > >> >> >and mapping down to h2 frames as a sequence of octets would be > >fine. > >> >> >But you seem to both agree with my reasoning and disagree with my > >> >> >conclusion. This is confusing. > >> >> > > >> >> >>> I was suggesting that we just treat the HTTP/2 stream like the > >> >TCP > >> >> >>> connection in RFC 6455 - the conversation from stream to > >message > >> >> >>> based semantics and so on can take place above that in the ws > >> >> >>> implementation - and that we should still apply the > >transmission > >> >> >>> windows etc to ws streams. > >> >> > >> >> Yes ---^ this is how it has to be I think. > >> >> > >> >> >> If you do that you loose any and all benefits from HTTP/2 > >frames. > >> >> >> Everything from ws frame headers to data content becomes > >> >semantically > >> >> >> identical to the opaque payload of a DATA frame on an HTTP/2 > >> >CONNECT > >> >> >> request. I believe Yutaka is seeking to get away from that > >> >situation > >> >> >> where DATA frames may be split, joined or buffered at any > >point. > >> >> > > >> >> >Sorry, I just don't follow that. We have a primitive which > >appears > >> >to > >> >> >fit ws entirely, with the only caveat being that we haven't > >defined > >> >> >the mapping from the high level frames to the h2 primitives. If > >the > >> >> > >> >> Yeah. > >> >> > >> >> >spec identifies how ws is negotiated and framed within h2, its > >not > >> >> >opaque at all. And ws implementations that support raw ws (which > >> >> >they'll do for quite some time...) have to deal with tcp which > >> >offers > >> >> >no better semantics than this. > >> >> > >> >> Right now if I understood it the ws connections can still > >negotiate > >> >> themselves transparently inside http2 mux connections, using the > >> >RFC6455 > >> >> upgrade on their individual session ID, do the extra RTT and tx > >data > >> >> masking. > >> >> > >> >> Formalizing how to encapsulate the same thing in http2 doesn't buy > >> >much > >> >> above that... the benefit we can get is map the RFC6455 framing on > >to > >> >http2 > >> >> native framing and get rid of the duplication simple encapsulation > >> >has (for > >> >> many small frames, it would be really painful overhead actually). > >So > >> >if we > >> >> will do anything, it should indeed be define how to map RFC6455 > >> >framing on > >> >> http2 framing. > >> >> > >> >> I guess it can be done in parallel with http2 coming to an end > >rather > >> >than > >> >> trying to block it, just by defining some new optional frame > >types... > >> >> > >> >> -Andy > >> >> > >> >> >-Rob > >> >> > >> >> > >> >> > >> > >> > >
- HTTP/2 and Websockets Robert Collins
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Ilari Liusvaara
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Greg Wilkins
- Re: HTTP/2 and Websockets Ilari Liusvaara
- Re: HTTP/2 and Websockets Robert Collins
- Re: HTTP/2 and Websockets Ilari Liusvaara
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Amos Jeffries
- Re: HTTP/2 and Websockets Robert Collins
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Amos Jeffries
- Re: HTTP/2 and Websockets Robert Collins
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Robert Collins
- Re: HTTP/2 and Websockets Robert Collins
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Amos Jeffries
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Amos Jeffries
- Re: HTTP/2 and Websockets Jason Greene
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Jason Greene
- Re: HTTP/2 and Websockets Jason Greene
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Jason Greene
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Amos Jeffries
- Re: HTTP/2 and Websockets Robert Collins
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Amos Jeffries
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Martin Thomson
- Re: HTTP/2 and Websockets Jason Greene
- Re: HTTP/2 and Websockets Martin Thomson
- Re: HTTP/2 and Websockets Roberto Peon
- Re: HTTP/2 and Websockets Willy Tarreau
- Re: HTTP/2 and Websockets Jason Greene
- Re: HTTP/2 and Websockets Jason Greene
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Martin Thomson
- Re: HTTP/2 and Websockets Willy Tarreau
- Re: HTTP/2 and Websockets Amos Jeffries
- Re: HTTP/2 and Websockets Greg Wilkins
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Andy Green
- Re: HTTP/2 and Websockets Amos Jeffries
- Re: HTTP/2 and Websockets Roberto Peon
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Yutaka Hirano
- Re: HTTP/2 and Websockets Martin Thomson
- Re: HTTP/2 and Websockets Roberto Peon
- Re: HTTP/2 and Websockets Brad Fitzpatrick
- Re: HTTP/2 and Websockets Roberto Peon
- Re: HTTP/2 and Websockets Glen Knowles
- Re: HTTP/2 and Websockets Roberto Peon
- Re: HTTP/2 and Websockets Wenbo Zhu