Re: [hybi] frame length encoding

[John Tamplin <[e-mail]>]

On Sun, Aug 22, 2010 at 3:57 AM, Brian <[e-mail]> wrote:

> As much as I feel that 63-bits is ridiculously large for the frame length,
> I'd be happy to accept it and just move on to the next item of discussion --
> though the WS server I'm writing right now will consider any frame with a
> non-zero bit in the high 31-bits of the long long length field to be an
> error condition, and immediately drop the rogue/attacking/misbehaving
> client.  I don't want to be bound to treat absurdly large frames as
> potentially legitimate by an overzealous specification.  Furthermore, I'm
> writing my server with Node.js, which, being a JavaScript-based environment
> itself, doesn't even have a 64-bit integer type of any sort.  I *can't*
> correctly handle a 63-bit frame.  Ah, the irony of a protocol meant for
> JavaScript applications that's impossible to fully implemented in JavaScript
> (at least efficiently -- no bignum libraries allowed.)  And who knows what
> would happen with the actual frame data in, say, a JavaScript String, with a
> frame of a few terabytes?  Assuming anyone had that much RAM?   We're
> talking about JavaScript clients here, so no streaming to disk, etc....
>
> In my not-so-humble opinion, trying to accommodate sending large files
> through WebSockets is a complete and utter misdirection.  The protocol
> exists to provide real-time interactivity support to browsers.  That's what
> it's meant for.  If someone wants to send large chunks of data (>4GB) in one
> frame, they should find another protocol, because they're clearly not
> planning to process it in JavaScript in a web-browser client.  And if
> they're not running in the browser, there's no reason they have to use WS.
>  Making abusers of the protocol chunk at 4GB boundaries is more than
> reasonable when you consider that the vast majority of users won't begin to
> consider sending fragments anywhere near even a fraction of that size.  This
> is not and should not be an all-things to all-people protocol.  We shouldn't
> be designing for these edge cases just because the protocol could
> theoretically enable any number of diverse applications.
>

Just because the JS API today passes the messages totally in RAM, there is
no reason that it can be extended to support a stream-like API if processing
larger messages becomes justified.  However, at the framing level, we have
to have something in place such that v1.0 implementations written against
this spec can interoperate with later extensions, so it seems prudent to
make sure the protocol is sufficient for any reasonably forseeable needs.
 There were several strong arguments that 32-bits isn't enough, and the next
logical step up is 63/64-bits.

Personally, I would be perfectly happy with a 32-bit maximum frame length,
and let senders fragment larger messages.  However, there were significant
objections (see the discussions link from
http://www.ietf.org/mail-archive/web/hybi/current/msg03399.html for details)
about being required to fragment a message when its size was known, and the
fact that it made "fire and forget" of sending the message contents with a
single sendfile() call impossible (when larger than 4G).  Thus, it seemed
likely the only way to gain consensus was to support a maximum frame length
much larger than 32 bits.

I agree in the spec we need to clarify behavior regarding frame size.  I
suspect we will wind up adding some way of declaring maximum acceptable
frame size during the handshake, but we have yet to start working on trying
to nail that down.

-- 
John A. Tamplin
Software Engineer (GWT), Google