Re: HTTP/2 and Websockets

[Yutaka Hirano <[e-mail]>]

On Fri, Nov 21, 2014 at 11:42 AM, Andy Green <[e-mail]> wrote:

>
>
> On 21 November 2014 10:02:28 GMT+08:00, Yutaka Hirano <[e-mail]>
> wrote:
> >>
> >> I think you have to take the atomic large ws frame thing as a genuine
> >> problem because http2 has the transmit credit concept.  So even if
> >you
> >> buffered one ws frame, you can't sit there spewing as much http2 DATA
> >frame
> >> as it needs to atomically encapsulate it, without sizing your http2
> >frame
> >> to suit the tx credit.
> >> But it's OK because the implementation can transparently fragment the
> >ws
> >> data using the ws message semantics... I think there's no choice but
> >to
> >> take that approach.
> >
> >Sorry I don't understand what you are proposing. Can you explain?
>
> I'm agreeing with what was already written by someone else on the thread.
>
> Talking about buffering huge ws frames until you have enough to issue it
> all in one big http2 DATA frame will not fly.
>
> If you're using this putative ws-over-http2 scheme, and you get given a
> huge ws frame to transmit, you should fragment it using RFC6455 message
> semantics to some implementation-defined limit that is friendly for mux'd
> http2 transport.
>
Thanks.

Strictly speaking, RFC6455 allows an extension to give meaning to WebSocket
frames, so merging / fragmenting frames breaks such extensions.
We discussed this problem in HyBi and many of us said "don't care".
In any case, an http/2 frame cannot be bigger than 2^24 (or 2^14 without an
explicit permission), so I think we don't have to worry about DoS.


>
> >I guess it can be done in parallel with http2 coming to an end rather
> >than
> >> trying to block it, just by defining some new optional frame types...
> >
> >I agree to define a ws-dedicated frame type and use it.
>
> Super... has anyone proposed how to map RFC6455 to http2 framing in detail
> yet?
>
I think not.
I did list several ways at [1], but I deleted it from the next version[2]
because HTTP/2 situation had changed.

1: http://tools.ietf.org/html/draft-hirano-httpbis-websocket-over-http2-00
2: http://tools.ietf.org/html/draft-hirano-httpbis-websocket-over-http2-01


>
> -Andy
>
> >
> >On Fri, Nov 21, 2014 at 10:47 AM, Andy Green <[e-mail]> wrote:
> >
> >>
> >>
> >> On 21 November 2014 04:11:53 GMT+08:00, Robert Collins <
> >> [e-mail]> wrote:
> >> >On 15 October 2014 00:00, Amos Jeffries <[e-mail]>
> >wrote:
> >> >> -----BEGIN PGP SIGNED MESSAGE-----
> >> >> Hash: SHA1
> >> >>
> >> >> On 14/10/2014 11:01 p.m., Robert Collins wrote:
> >> >>> On 1 October 2014 23:37, Amos Jeffries wrote:
> >> >>>
> >> >>>>> All the implementor discussion I've seen during the
> >> >>>>>> HTTP/2 discussions has focused on how intermediaries want to
> >> >>>>>> be scalable: and buffering is anti-scaling. So - is it a
> >> >>>>>> pragmatic concern, or do we expect DATA stream buffering to
> >> >>>>>> take place [outside of protocol gateways converting to
> >> >>>>>> HTTP/1.1 where non upload can require buffering - and note
> >> >>>>>> that such a gateway can't carry ws anyway unless its aware of
> >> >>>>>> it, and if its aware of it, it can make sure it does not
> >> >>>>>> buffer].
> >> >>>>
> >> >>>>
> >> >>>> I think the problem is not buffering in HTTP/2 per-se but the
> >> >>>> DATA frame (de-)aggregation that can happen if the frames are
> >> >>>> buffered by general network conditions (ie in TCP bottlenecks).
> >> >>>> This would not be good for a 1:1 relationship between DATA and
> >ws
> >> >>>> frames.
> >> >>>>
> >> >>>> Amos
> >> >>>
> >> >>> So hang on a second here. If we say that ws frames can't be split
> >> >>> over multiple HTTP/2 frames that implies that we have to buffer
> >> >>> them until there is enough in the window to transmit a
> >potentially
> >> >>> very large packet all at once. It also conflicts with RFC6455 -
> >the
> >> >>> specific intent there is to not be a stream based system.
> >> >>
> >> >> If a ws frame *has* to be that long, not doing so would block the
> >> >> entire HTTP/2 connection until all bytes of that frame were
> >delivered
> >> >> anyway. So you trade off buffering that single frame at the
> >sender,
> >> >> versus blocking all HTTP/2 traffic end-to-end.
> >> >>
> >> >> If the ws data is so critical to get transmitted fast why is that
> >> >> single ws frame so large to begin with? surely it would be
> >> >transmitted
> >> >> faster as a sequence of WS + *WSDATA frames emited as the payload
> >was
> >> >> available to send.
> >> >
> >> >I agree that its inconsistent which is why I don't think it matters
> >>
> >> I am the author of libwebsockets, we are adding http2 support at the
> >> moment.  The basic http2 serving is done and works for http, but
> >we're all
> >> dressed up and nowhere to go in terms of treating websocket
> >connections as
> >> just another kind of http2, since the framing is "TBD".  I am sorry I
> >am a
> >> bit late to the party.
> >>
> >> I think you have to take the atomic large ws frame thing as a genuine
> >> problem because http2 has the transmit credit concept.  So even if
> >you
> >> buffered one ws frame, you can't sit there spewing as much http2 DATA
> >frame
> >> as it needs to atomically encapsulate it, without sizing your http2
> >frame
> >> to suit the tx credit.
> >>
> >> But it's OK because the implementation can transparently fragment the
> >ws
> >> data using the ws message semantics... I think there's no choice but
> >to
> >> take that approach.
> >>
> >> Otherwise you get into being able to DoS even an http2 "big pipe
> >> aggregation" by just one mux element spewing an endless ws frame and
> >> blocking every other mux'd connection... it cannot be right.
> >>
> >> >and mapping down to h2 frames as a sequence of octets would be fine.
> >> >But you seem to both agree with my reasoning and disagree with my
> >> >conclusion. This is confusing.
> >> >
> >> >>> I was suggesting that we just treat the HTTP/2 stream like the
> >TCP
> >> >>> connection in RFC 6455 - the conversation from stream to message
> >> >>> based semantics and so on can take place above that in the ws
> >> >>> implementation - and that we should still apply the transmission
> >> >>> windows etc to ws streams.
> >>
> >> Yes ---^ this is how it has to be I think.
> >>
> >> >> If you do that you loose any and all benefits from HTTP/2 frames.
> >> >> Everything from ws frame headers to data content becomes
> >semantically
> >> >> identical to the opaque payload of a DATA frame on an HTTP/2
> >CONNECT
> >> >> request. I believe Yutaka is seeking to get away from that
> >situation
> >> >> where DATA frames may be split, joined or buffered at any point.
> >> >
> >> >Sorry, I just don't follow that. We have a primitive which appears
> >to
> >> >fit ws entirely, with the only caveat being that we haven't defined
> >> >the mapping from the high level frames to the h2 primitives. If the
> >>
> >> Yeah.
> >>
> >> >spec identifies how ws is negotiated and framed within h2, its not
> >> >opaque at all. And ws implementations that support raw ws (which
> >> >they'll do for quite some time...) have to deal with tcp which
> >offers
> >> >no better semantics than this.
> >>
> >> Right now if I understood it the ws connections can still negotiate
> >> themselves transparently inside http2 mux connections, using the
> >RFC6455
> >> upgrade on their individual session ID, do the extra RTT and tx data
> >> masking.
> >>
> >> Formalizing how to encapsulate the same thing in http2 doesn't buy
> >much
> >> above that... the benefit we can get is map the RFC6455 framing on to
> >http2
> >> native framing and get rid of the duplication simple encapsulation
> >has (for
> >> many small frames, it would be really painful overhead actually).  So
> >if we
> >> will do anything, it should indeed be define how to map RFC6455
> >framing on
> >> http2 framing.
> >>
> >> I guess it can be done in parallel with http2 coming to an end rather
> >than
> >> trying to block it, just by defining some new optional frame types...
> >>
> >> -Andy
> >>
> >> >-Rob
> >>
> >>
> >>
>
>