Re: [hybi] Frame size

[Jamie Lokier <[e-mail]>]

Mike Belshe wrote:
> 
>    As an anecdote:  every single place that we thought we should use a
>    16bit length in SPDY has come back to haunt us and will ultimately get
>    changed to 32bits.  When thinking of protocols to use now and also in
>    2020, we really should not limit anything to 16bits unless it is in
>    some critically bandwidth-sensitive part of the protocol.  Remember
>    that bandwidth is cheap (and growing fast!)  Round Trip Times are
>    expensive (and not shrinking!)

I agree.


>    It seems to me that we're over engineering here for what the
>    websockets API is trying to do.
>    I disagree that 32bit lengths means that you need to support chunking.
>     The application can decide when it wants to chunk or not.

I think the point of chunks are

  - to permit the transport implementation to make network-aware
    decisions (such as aligning with TCP segments)

  - to permit intermediaries to re-chunk when that is advantageous
    (such as aligning with the next hop TCP segments)

  - to carry information for smarter buffering decisions at intermediaries
    (to reduce latency - eager forwarding of every byte when received
    is not ideal) (and I know you care about latency)

  - as a realistic basis for multiplexing, even if that's
    an extension not in the base protocol

  - to leave room for transport control messages inserted by the
    transport (such as graceful close and transport error indicators),
    so that applications don't have implement them

>    Also, why does a buffer need to be contained in RAM?  Just
>    because we have a long length doesn't mean that the payload can't
>    stream.  Lots of protocols work this way today, like HTTP.

WebSocket API spec says that messages must be completely collected
before they are signalled as DOM events.  They cannot be streamed in
that API.

About buffering *frames* as opposed to messages, I agree.  There is no
reason that frames need to be buffered whole before assembling them
into messages or forwarding them to the next hop.  Intermediaries
should not buffer whole frames of arbitrary length, they should
forward them, and probably change the frame boundaries at the moment
of forwarding to avoid blocking control messages later.

>     And what happens if you
>    stream a 10GB message in a single web page?  The browser just hangs up
>    the phone at some point.  There is no graceful error.

Why?  There is no reason why the browser cannot stop reading, and send
the graceful error message...

And that is important: Look at it the other way (although either way
around this can occur).  You're sending a 10GB message to the server
in something akin to HTTP PUT (but over this protocol), and that
particular server is being shut down, times out or whatever.  If the
server just hangs up, the browser doesn't know if it's safe to
retransmit that POST request, so it must report an error.  But if the
server sends a graceful "sorry I'm closing" message, the browser can
retry automatically (like it does for HTTP GETs).

That is currently why pipelined HTTP is so severely broken and close
to unusable...  Let's not repeat that debacle.

>    While this is a theoretical problem (which Greg raised), it is
>    not a practical one.  Unfortunately, once you decide that 32bit
>    lengths are not good enough, you're running into lots of
>    subproblems: fragmentation/reassembly, max frame size, etc.  I
>    think we're just overthinknig the needs to the WebSockets API.  I
>    propose we make lengths be 32bits and drop
>    fragmentation/reassembly from this protocol.

So you're committing to being unable to send 5GB messages?

That seems inconsistent with the mention of future networks.  Today, a
5GB message takes just 5 seconds on a decent 10gig ethernet.  That is
well within the usability range of WebSocket and browser applications,
although typically there isn't enough RAM and processing isn't fast
enough to do something useful inside a browser application with that
kind of data.

In 2020, a 5GB message will take a fraction of a second to transmit,
<1% of RAM, and the CPU will be able to process all of it quickly
enough to do useful things, such as updating a 3D model in Javascript.
I see no benefit to forbidding their use while permitting 3GB messages.

-- Jamie