Re: HTTP/2 and Websockets

[Andy Green <[e-mail]>]

On 21 November 2014 14:41:19 GMT+08:00, Yutaka Hirano <[e-mail]> wrote:
>On Fri, Nov 21, 2014 at 12:38 PM, Andy Green <[e-mail]> wrote:
>
>>
>>
>> On 21 November 2014 11:30:46 GMT+08:00, Yutaka Hirano
><[e-mail]>
>> wrote:
>> >On Fri, Nov 21, 2014 at 11:42 AM, Andy Green <[e-mail]>
>wrote:
>> >
>> >>
>> >>
>> >> On 21 November 2014 10:02:28 GMT+08:00, Yutaka Hirano
>> ><[e-mail]>
>> >> wrote:
>> >> >>
>> >> >> I think you have to take the atomic large ws frame thing as a
>> >genuine
>> >> >> problem because http2 has the transmit credit concept.  So even
>if
>> >> >you
>> >> >> buffered one ws frame, you can't sit there spewing as much
>http2
>> >DATA
>> >> >frame
>> >> >> as it needs to atomically encapsulate it, without sizing your
>> >http2
>> >> >frame
>> >> >> to suit the tx credit.
>> >> >> But it's OK because the implementation can transparently
>fragment
>> >the
>> >> >ws
>> >> >> data using the ws message semantics... I think there's no
>choice
>> >but
>> >> >to
>> >> >> take that approach.
>> >> >
>> >> >Sorry I don't understand what you are proposing. Can you explain?
>> >>
>> >> I'm agreeing with what was already written by someone else on the
>> >thread.
>> >>
>> >> Talking about buffering huge ws frames until you have enough to
>issue
>> >it
>> >> all in one big http2 DATA frame will not fly.
>> >>
>> >> If you're using this putative ws-over-http2 scheme, and you get
>given
>> >a
>> >> huge ws frame to transmit, you should fragment it using RFC6455
>> >message
>> >> semantics to some implementation-defined limit that is friendly
>for
>> >mux'd
>> >> http2 transport.
>> >>
>> >Thanks.
>> >
>> >Strictly speaking, RFC6455 allows an extension to give meaning to
>> >WebSocket
>> >frames, so merging / fragmenting frames breaks such extensions.
>> >We discussed this problem in HyBi and many of us said "don't care".
>>
>> Yeah extensions except for compression have just not come into
>existence.
>>
>> So I also see it as don't care.  I'm not even sure it's true since
>the
>> intention during ws discussion was an intermediary can fragment
>frames same
>> as how tcp packets may be fragmented.
>>
>> >In any case, an http/2 frame cannot be bigger than 2^24 (or 2^14
>> >without an
>> >explicit permission), so I think we don't have to worry about DoS.
>>
>> ... I don't see that.  If I keep spamming 16MB frames even on one
>stream
>> on a consumer link your latency goes to pieces and if something is
>doing
>> multiple instances of it even a big pipe will feel pain.
>>
>>
>The spec says:
>  Length:
>The length of the frame payload expressed as an unsigned 24-bit
>integer.
>Values greater than 2^14 (16,384) MUST NOT be sent unless the receiver
>has
>set a larger value for SETTINGS_MAX_FRAME_SIZE.
>So by default 2^14 is the upper limit. Larger frames will be used only
>if
>the receiver wants.

Okay.  But if buffering whole ws frames no matter how big was the plan, and the one end wanted to send a 16MB ws frame, the other end would say, "sure" you can send me >16K and he'd spam his 16MB frames.  Default 16K limit doesn't help much because the server is complicit in thinking sending huge ws frames is good.

I think we're agreeing, that is not what anybody wants to see.

-Andy

>> >> >I guess it can be done in parallel with http2 coming to an end
>> >rather
>> >> >than
>> >> >> trying to block it, just by defining some new optional frame
>> >types...
>> >> >
>> >> >I agree to define a ws-dedicated frame type and use it.
>> >>
>> >> Super... has anyone proposed how to map RFC6455 to http2 framing
>in
>> >detail
>> >> yet?
>> >>
>> >I think not.
>> >I did list several ways at [1], but I deleted it from the next
>>
>> Alright I will go look at these.
>>
>> The discussion is moribund somebody should probably issue a 'stalking
>> horse' since it's easier for people to jump in and tell you you're
>doing it
>> wrong ^^
>>
>> -Andy
>>
>> >version[2]
>> >because HTTP/2 situation had changed.
>> >
>> >1:
>>
>>http://tools.ietf.org/html/draft-hirano-httpbis-websocket-over-http2-00
>> >2:
>>
>>http://tools.ietf.org/html/draft-hirano-httpbis-websocket-over-http2-01
>> >
>> >
>> >>
>> >> -Andy
>> >>
>> >> >
>> >> >On Fri, Nov 21, 2014 at 10:47 AM, Andy Green <[e-mail]>
>> >wrote:
>> >> >
>> >> >>
>> >> >>
>> >> >> On 21 November 2014 04:11:53 GMT+08:00, Robert Collins <
>> >> >> [e-mail]> wrote:
>> >> >> >On 15 October 2014 00:00, Amos Jeffries <[e-mail]>
>> >> >wrote:
>> >> >> >> -----BEGIN PGP SIGNED MESSAGE-----
>> >> >> >> Hash: SHA1
>> >> >> >>
>> >> >> >> On 14/10/2014 11:01 p.m., Robert Collins wrote:
>> >> >> >>> On 1 October 2014 23:37, Amos Jeffries wrote:
>> >> >> >>>
>> >> >> >>>>> All the implementor discussion I've seen during the
>> >> >> >>>>>> HTTP/2 discussions has focused on how intermediaries
>want
>> >to
>> >> >> >>>>>> be scalable: and buffering is anti-scaling. So - is it a
>> >> >> >>>>>> pragmatic concern, or do we expect DATA stream buffering
>to
>> >> >> >>>>>> take place [outside of protocol gateways converting to
>> >> >> >>>>>> HTTP/1.1 where non upload can require buffering - and
>note
>> >> >> >>>>>> that such a gateway can't carry ws anyway unless its
>aware
>> >of
>> >> >> >>>>>> it, and if its aware of it, it can make sure it does not
>> >> >> >>>>>> buffer].
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> I think the problem is not buffering in HTTP/2 per-se but
>the
>> >> >> >>>> DATA frame (de-)aggregation that can happen if the frames
>are
>> >> >> >>>> buffered by general network conditions (ie in TCP
>> >bottlenecks).
>> >> >> >>>> This would not be good for a 1:1 relationship between DATA
>> >and
>> >> >ws
>> >> >> >>>> frames.
>> >> >> >>>>
>> >> >> >>>> Amos
>> >> >> >>>
>> >> >> >>> So hang on a second here. If we say that ws frames can't be
>> >split
>> >> >> >>> over multiple HTTP/2 frames that implies that we have to
>> >buffer
>> >> >> >>> them until there is enough in the window to transmit a
>> >> >potentially
>> >> >> >>> very large packet all at once. It also conflicts with
>RFC6455
>> >-
>> >> >the
>> >> >> >>> specific intent there is to not be a stream based system.
>> >> >> >>
>> >> >> >> If a ws frame *has* to be that long, not doing so would
>block
>> >the
>> >> >> >> entire HTTP/2 connection until all bytes of that frame were
>> >> >delivered
>> >> >> >> anyway. So you trade off buffering that single frame at the
>> >> >sender,
>> >> >> >> versus blocking all HTTP/2 traffic end-to-end.
>> >> >> >>
>> >> >> >> If the ws data is so critical to get transmitted fast why is
>> >that
>> >> >> >> single ws frame so large to begin with? surely it would be
>> >> >> >transmitted
>> >> >> >> faster as a sequence of WS + *WSDATA frames emited as the
>> >payload
>> >> >was
>> >> >> >> available to send.
>> >> >> >
>> >> >> >I agree that its inconsistent which is why I don't think it
>> >matters
>> >> >>
>> >> >> I am the author of libwebsockets, we are adding http2 support
>at
>> >the
>> >> >> moment.  The basic http2 serving is done and works for http,
>but
>> >> >we're all
>> >> >> dressed up and nowhere to go in terms of treating websocket
>> >> >connections as
>> >> >> just another kind of http2, since the framing is "TBD".  I am
>> >sorry I
>> >> >am a
>> >> >> bit late to the party.
>> >> >>
>> >> >> I think you have to take the atomic large ws frame thing as a
>> >genuine
>> >> >> problem because http2 has the transmit credit concept.  So even
>if
>> >> >you
>> >> >> buffered one ws frame, you can't sit there spewing as much
>http2
>> >DATA
>> >> >frame
>> >> >> as it needs to atomically encapsulate it, without sizing your
>> >http2
>> >> >frame
>> >> >> to suit the tx credit.
>> >> >>
>> >> >> But it's OK because the implementation can transparently
>fragment
>> >the
>> >> >ws
>> >> >> data using the ws message semantics... I think there's no
>choice
>> >but
>> >> >to
>> >> >> take that approach.
>> >> >>
>> >> >> Otherwise you get into being able to DoS even an http2 "big
>pipe
>> >> >> aggregation" by just one mux element spewing an endless ws
>frame
>> >and
>> >> >> blocking every other mux'd connection... it cannot be right.
>> >> >>
>> >> >> >and mapping down to h2 frames as a sequence of octets would be
>> >fine.
>> >> >> >But you seem to both agree with my reasoning and disagree with
>my
>> >> >> >conclusion. This is confusing.
>> >> >> >
>> >> >> >>> I was suggesting that we just treat the HTTP/2 stream like
>the
>> >> >TCP
>> >> >> >>> connection in RFC 6455 - the conversation from stream to
>> >message
>> >> >> >>> based semantics and so on can take place above that in the
>ws
>> >> >> >>> implementation - and that we should still apply the
>> >transmission
>> >> >> >>> windows etc to ws streams.
>> >> >>
>> >> >> Yes ---^ this is how it has to be I think.
>> >> >>
>> >> >> >> If you do that you loose any and all benefits from HTTP/2
>> >frames.
>> >> >> >> Everything from ws frame headers to data content becomes
>> >> >semantically
>> >> >> >> identical to the opaque payload of a DATA frame on an HTTP/2
>> >> >CONNECT
>> >> >> >> request. I believe Yutaka is seeking to get away from that
>> >> >situation
>> >> >> >> where DATA frames may be split, joined or buffered at any
>> >point.
>> >> >> >
>> >> >> >Sorry, I just don't follow that. We have a primitive which
>> >appears
>> >> >to
>> >> >> >fit ws entirely, with the only caveat being that we haven't
>> >defined
>> >> >> >the mapping from the high level frames to the h2 primitives.
>If
>> >the
>> >> >>
>> >> >> Yeah.
>> >> >>
>> >> >> >spec identifies how ws is negotiated and framed within h2, its
>> >not
>> >> >> >opaque at all. And ws implementations that support raw ws
>(which
>> >> >> >they'll do for quite some time...) have to deal with tcp which
>> >> >offers
>> >> >> >no better semantics than this.
>> >> >>
>> >> >> Right now if I understood it the ws connections can still
>> >negotiate
>> >> >> themselves transparently inside http2 mux connections, using
>the
>> >> >RFC6455
>> >> >> upgrade on their individual session ID, do the extra RTT and tx
>> >data
>> >> >> masking.
>> >> >>
>> >> >> Formalizing how to encapsulate the same thing in http2 doesn't
>buy
>> >> >much
>> >> >> above that... the benefit we can get is map the RFC6455 framing
>on
>> >to
>> >> >http2
>> >> >> native framing and get rid of the duplication simple
>encapsulation
>> >> >has (for
>> >> >> many small frames, it would be really painful overhead
>actually).
>> >So
>> >> >if we
>> >> >> will do anything, it should indeed be define how to map RFC6455
>> >> >framing on
>> >> >> http2 framing.
>> >> >>
>> >> >> I guess it can be done in parallel with http2 coming to an end
>> >rather
>> >> >than
>> >> >> trying to block it, just by defining some new optional frame
>> >types...
>> >> >>
>> >> >> -Andy
>> >> >>
>> >> >> >-Rob
>> >> >>
>> >> >>
>> >> >>
>> >>
>> >>
>>
>>