Re: HTTP/2 and Websockets

[Andy Green <[e-mail]>]

On 21 November 2014 11:38:26 GMT+08:00, Andy Green <[e-mail]> wrote:
>
>
>On 21 November 2014 11:30:46 GMT+08:00, Yutaka Hirano
><[e-mail]> wrote:
>>On Fri, Nov 21, 2014 at 11:42 AM, Andy Green <[e-mail]> wrote:
>>
>>>
>>>
>>> On 21 November 2014 10:02:28 GMT+08:00, Yutaka Hirano
>><[e-mail]>
>>> wrote:
>>> >>
>>> >> I think you have to take the atomic large ws frame thing as a
>>genuine
>>> >> problem because http2 has the transmit credit concept.  So even
>if
>>> >you
>>> >> buffered one ws frame, you can't sit there spewing as much http2
>>DATA
>>> >frame
>>> >> as it needs to atomically encapsulate it, without sizing your
>>http2
>>> >frame
>>> >> to suit the tx credit.
>>> >> But it's OK because the implementation can transparently fragment
>>the
>>> >ws
>>> >> data using the ws message semantics... I think there's no choice
>>but
>>> >to
>>> >> take that approach.
>>> >
>>> >Sorry I don't understand what you are proposing. Can you explain?
>>>
>>> I'm agreeing with what was already written by someone else on the
>>thread.
>>>
>>> Talking about buffering huge ws frames until you have enough to
>issue
>>it
>>> all in one big http2 DATA frame will not fly.
>>>
>>> If you're using this putative ws-over-http2 scheme, and you get
>given
>>a
>>> huge ws frame to transmit, you should fragment it using RFC6455
>>message
>>> semantics to some implementation-defined limit that is friendly for
>>mux'd
>>> http2 transport.
>>>
>>Thanks.
>>
>>Strictly speaking, RFC6455 allows an extension to give meaning to
>>WebSocket
>>frames, so merging / fragmenting frames breaks such extensions.
>>We discussed this problem in HyBi and many of us said "don't care".
>
>Yeah extensions except for compression have just not come into
>existence.
>
>So I also see it as don't care.  I'm not even sure it's true since the
>intention during ws discussion was an intermediary can fragment frames
>same as how tcp packets may be fragmented.
>
>>In any case, an http/2 frame cannot be bigger than 2^24 (or 2^14
>>without an
>>explicit permission), so I think we don't have to worry about DoS.
>
>... I don't see that.  If I keep spamming 16MB frames even on one
>stream on a consumer link your latency goes to pieces and if something
>is doing multiple instances of it even a big pipe will feel pain.
>
>>> >I guess it can be done in parallel with http2 coming to an end
>>rather
>>> >than
>>> >> trying to block it, just by defining some new optional frame
>>types...
>>> >
>>> >I agree to define a ws-dedicated frame type and use it.
>>>
>>> Super... has anyone proposed how to map RFC6455 to http2 framing in
>>detail
>>> yet?
>>>
>>I think not.
>>I did list several ways at [1], but I deleted it from the next
>
>Alright I will go look at these.

5.2.1 is like pure tunneling, it's simple alright but it's bloated for small frames.

5.2.2 has a needless flushing thing on top... in a real ws setup pieces of data can arrive any old way they should be passed on as they come or coalsced according to intermediary policy, I don't see the point.

5.3.3 headers + data per websocket frame... super inefficient

Has it already been discussed to just merge http2 frame with ws frame?

 - use the headers scheme you defined to negotiate the connection + ws protocol

 - There's an http2 frame type called like WSDATA

 - WSDATA's 8-bit flags field, is the ws flags [fin][rsv1-3][opcode]

 - WSDATA's payload, is the ws frame payload... the ws framing disappears completely into the http2 framing.

... what problems does this create?

-Andy

>The discussion is moribund somebody should probably issue a 'stalking
>horse' since it's easier for people to jump in and tell you you're
>doing it wrong ^^
>
>-Andy
>
>>version[2]
>>because HTTP/2 situation had changed.
>>
>>1:
>>http://tools.ietf.org/html/draft-hirano-httpbis-websocket-over-http2-00
>>2:
>>http://tools.ietf.org/html/draft-hirano-httpbis-websocket-over-http2-01
>>
>>
>>>
>>> -Andy
>>>
>>> >
>>> >On Fri, Nov 21, 2014 at 10:47 AM, Andy Green <[e-mail]>
>>wrote:
>>> >
>>> >>
>>> >>
>>> >> On 21 November 2014 04:11:53 GMT+08:00, Robert Collins <
>>> >> [e-mail]> wrote:
>>> >> >On 15 October 2014 00:00, Amos Jeffries <[e-mail]>
>>> >wrote:
>>> >> >> -----BEGIN PGP SIGNED MESSAGE-----
>>> >> >> Hash: SHA1
>>> >> >>
>>> >> >> On 14/10/2014 11:01 p.m., Robert Collins wrote:
>>> >> >>> On 1 October 2014 23:37, Amos Jeffries wrote:
>>> >> >>>
>>> >> >>>>> All the implementor discussion I've seen during the
>>> >> >>>>>> HTTP/2 discussions has focused on how intermediaries want
>>to
>>> >> >>>>>> be scalable: and buffering is anti-scaling. So - is it a
>>> >> >>>>>> pragmatic concern, or do we expect DATA stream buffering
>to
>>> >> >>>>>> take place [outside of protocol gateways converting to
>>> >> >>>>>> HTTP/1.1 where non upload can require buffering - and note
>>> >> >>>>>> that such a gateway can't carry ws anyway unless its aware
>>of
>>> >> >>>>>> it, and if its aware of it, it can make sure it does not
>>> >> >>>>>> buffer].
>>> >> >>>>
>>> >> >>>>
>>> >> >>>> I think the problem is not buffering in HTTP/2 per-se but
>the
>>> >> >>>> DATA frame (de-)aggregation that can happen if the frames
>are
>>> >> >>>> buffered by general network conditions (ie in TCP
>>bottlenecks).
>>> >> >>>> This would not be good for a 1:1 relationship between DATA
>>and
>>> >ws
>>> >> >>>> frames.
>>> >> >>>>
>>> >> >>>> Amos
>>> >> >>>
>>> >> >>> So hang on a second here. If we say that ws frames can't be
>>split
>>> >> >>> over multiple HTTP/2 frames that implies that we have to
>>buffer
>>> >> >>> them until there is enough in the window to transmit a
>>> >potentially
>>> >> >>> very large packet all at once. It also conflicts with RFC6455
>>-
>>> >the
>>> >> >>> specific intent there is to not be a stream based system.
>>> >> >>
>>> >> >> If a ws frame *has* to be that long, not doing so would block
>>the
>>> >> >> entire HTTP/2 connection until all bytes of that frame were
>>> >delivered
>>> >> >> anyway. So you trade off buffering that single frame at the
>>> >sender,
>>> >> >> versus blocking all HTTP/2 traffic end-to-end.
>>> >> >>
>>> >> >> If the ws data is so critical to get transmitted fast why is
>>that
>>> >> >> single ws frame so large to begin with? surely it would be
>>> >> >transmitted
>>> >> >> faster as a sequence of WS + *WSDATA frames emited as the
>>payload
>>> >was
>>> >> >> available to send.
>>> >> >
>>> >> >I agree that its inconsistent which is why I don't think it
>>matters
>>> >>
>>> >> I am the author of libwebsockets, we are adding http2 support at
>>the
>>> >> moment.  The basic http2 serving is done and works for http, but
>>> >we're all
>>> >> dressed up and nowhere to go in terms of treating websocket
>>> >connections as
>>> >> just another kind of http2, since the framing is "TBD".  I am
>>sorry I
>>> >am a
>>> >> bit late to the party.
>>> >>
>>> >> I think you have to take the atomic large ws frame thing as a
>>genuine
>>> >> problem because http2 has the transmit credit concept.  So even
>if
>>> >you
>>> >> buffered one ws frame, you can't sit there spewing as much http2
>>DATA
>>> >frame
>>> >> as it needs to atomically encapsulate it, without sizing your
>>http2
>>> >frame
>>> >> to suit the tx credit.
>>> >>
>>> >> But it's OK because the implementation can transparently fragment
>>the
>>> >ws
>>> >> data using the ws message semantics... I think there's no choice
>>but
>>> >to
>>> >> take that approach.
>>> >>
>>> >> Otherwise you get into being able to DoS even an http2 "big pipe
>>> >> aggregation" by just one mux element spewing an endless ws frame
>>and
>>> >> blocking every other mux'd connection... it cannot be right.
>>> >>
>>> >> >and mapping down to h2 frames as a sequence of octets would be
>>fine.
>>> >> >But you seem to both agree with my reasoning and disagree with
>my
>>> >> >conclusion. This is confusing.
>>> >> >
>>> >> >>> I was suggesting that we just treat the HTTP/2 stream like
>the
>>> >TCP
>>> >> >>> connection in RFC 6455 - the conversation from stream to
>>message
>>> >> >>> based semantics and so on can take place above that in the ws
>>> >> >>> implementation - and that we should still apply the
>>transmission
>>> >> >>> windows etc to ws streams.
>>> >>
>>> >> Yes ---^ this is how it has to be I think.
>>> >>
>>> >> >> If you do that you loose any and all benefits from HTTP/2
>>frames.
>>> >> >> Everything from ws frame headers to data content becomes
>>> >semantically
>>> >> >> identical to the opaque payload of a DATA frame on an HTTP/2
>>> >CONNECT
>>> >> >> request. I believe Yutaka is seeking to get away from that
>>> >situation
>>> >> >> where DATA frames may be split, joined or buffered at any
>>point.
>>> >> >
>>> >> >Sorry, I just don't follow that. We have a primitive which
>>appears
>>> >to
>>> >> >fit ws entirely, with the only caveat being that we haven't
>>defined
>>> >> >the mapping from the high level frames to the h2 primitives. If
>>the
>>> >>
>>> >> Yeah.
>>> >>
>>> >> >spec identifies how ws is negotiated and framed within h2, its
>>not
>>> >> >opaque at all. And ws implementations that support raw ws (which
>>> >> >they'll do for quite some time...) have to deal with tcp which
>>> >offers
>>> >> >no better semantics than this.
>>> >>
>>> >> Right now if I understood it the ws connections can still
>>negotiate
>>> >> themselves transparently inside http2 mux connections, using the
>>> >RFC6455
>>> >> upgrade on their individual session ID, do the extra RTT and tx
>>data
>>> >> masking.
>>> >>
>>> >> Formalizing how to encapsulate the same thing in http2 doesn't
>buy
>>> >much
>>> >> above that... the benefit we can get is map the RFC6455 framing
>on
>>to
>>> >http2
>>> >> native framing and get rid of the duplication simple
>encapsulation
>>> >has (for
>>> >> many small frames, it would be really painful overhead actually).
>
>>So
>>> >if we
>>> >> will do anything, it should indeed be define how to map RFC6455
>>> >framing on
>>> >> http2 framing.
>>> >>
>>> >> I guess it can be done in parallel with http2 coming to an end
>>rather
>>> >than
>>> >> trying to block it, just by defining some new optional frame
>>types...
>>> >>
>>> >> -Andy
>>> >>
>>> >> >-Rob
>>> >>
>>> >>
>>> >>
>>>
>>>