Re: [hybi] Interface for Large Frames

Willy Tarreau <> Wed, 22 June 2011 22:15 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id B17221F0C45 for <>; Wed, 22 Jun 2011 15:15:10 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -4.37
X-Spam-Status: No, score=-4.37 tagged_above=-999 required=5 tests=[AWL=-2.927, BAYES_00=-2.599, HELO_IS_SMALL6=0.556, J_CHICKENPOX_46=0.6]
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id j2zTqizcTkJu for <>; Wed, 22 Jun 2011 15:15:10 -0700 (PDT)
Received: from ( []) by (Postfix) with ESMTP id B0F471F0C41 for <>; Wed, 22 Jun 2011 15:15:09 -0700 (PDT)
Received: (from willy@localhost) by mail.home.local (8.14.4/8.14.4/Submit) id p5MMF1mS024529; Thu, 23 Jun 2011 00:15:01 +0200
Date: Thu, 23 Jun 2011 00:15:01 +0200
From: Willy Tarreau <>
To: Bob Gezelter <>
Message-ID: <>
References: <>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <>
User-Agent: Mutt/
Subject: Re: [hybi] Interface for Large Frames
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Server-Initiated HTTP <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 22 Jun 2011 22:15:11 -0000

Hi Bob,

On Wed, Jun 22, 2011 at 02:58:47PM -0700, Bob Gezelter wrote:
> The FILE methods in stdio.h include a "number of bytes" parameter. In
> the context of the WebSocket protocol, the natural unit would appear to
> be the frame. Once the frame has been retrieved by a user, then a
> character-by-character state machine may (emphasis MAY) be appropriate,
> however, the interface is best (most efficient) if entire frames can be
> transferred at a time.
> If programs must pull one character at a time, the efficiency will be
> poor, with a large amount of overhead being consumed moving up and down
> the call tree.

Obviously, and noone is recommending to do that! But the most common
read/write streaming loop is the appropriate way to do this in my
opinion, as it's done in any stream-based processor :

process_frame(int socket, int bytesleft)
    while (bytesleft > 0) {
        int bytes = read(socket, buffer, buffer_size);
        process(buffer, bytes);
        bytesleft -= bytes;

Large frames will mostly be used for video streaming or file transfers.
It totally makes sense to use a loop similar to the above to write bytes
to disk (in case of file upload) or to pass the stream to the video
decoder. Here we're just facing a streaming protocol (which was the
goal of WebSockets since the beginning) so I don't see what's new here.

It is possible that the frames make people think they should process
them at once, but frames are just here to make streaming easier to
handle. It can be easier to think as an uninterrupted stream with
some occasional checkpoints.