Re: [hybi] Interface for Large Frames

[Willy Tarreau <[e-mail]>]

Hi Bob,

On Wed, Jun 22, 2011 at 02:58:47PM -0700, Bob Gezelter wrote:
> The FILE methods in stdio.h include a "number of bytes" parameter. In
> the context of the WebSocket protocol, the natural unit would appear to
> be the frame. Once the frame has been retrieved by a user, then a
> character-by-character state machine may (emphasis MAY) be appropriate,
> however, the interface is best (most efficient) if entire frames can be
> transferred at a time.
> 
> If programs must pull one character at a time, the efficiency will be
> poor, with a large amount of overhead being consumed moving up and down
> the call tree.

Obviously, and noone is recommending to do that! But the most common
read/write streaming loop is the appropriate way to do this in my
opinion, as it's done in any stream-based processor :

process_frame(int socket, int bytesleft)
{
    while (bytesleft > 0) {
        int bytes = read(socket, buffer, buffer_size);
        process(buffer, bytes);
        bytesleft -= bytes;
    }
}

Large frames will mostly be used for video streaming or file transfers.
It totally makes sense to use a loop similar to the above to write bytes
to disk (in case of file upload) or to pass the stream to the video
decoder. Here we're just facing a streaming protocol (which was the
goal of WebSockets since the beginning) so I don't see what's new here.

It is possible that the frames make people think they should process
them at once, but frames are just here to make streaming easier to
handle. It can be easier to think as an uninterrupted stream with
some occasional checkpoints.

Regards,
Willy