Re: [hybi] Framing Take VI (a compromise proposal)

[John Tamplin <[e-mail]>]

On Sat, Aug 14, 2010 at 4:03 PM, Scott Ferguson <[e-mail]> wrote:

> Well, the complexity is more of a spec issue than a coding one because none
> of these proposals are hugely complicated to implement. (Implementing them
> efficiently is a different matter.)
>
> A Jamie-style proposal less complex because it leaves the basic frame clean
> without needing the extension length or the extension payload and without
> the extra complete-length field. The frame would look like:


There were objections raised about exactly this sort of thing before:

   - a receiver would prefer to allocate the buffer at once for a message
   rather than having to reallocate/copy it, which means the sender should send
   the complete message length with the first fragment if it is available
   - having a short length that was "mostly enough" would encourage buggy
   implementations that don't support or test the long length case


> 0                   1                   2                   3
> 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> +-+-+-+---------+-------------------------------+---------------+
> |I|F|B|opcode(5)| length(16)                    |               |
> |N|I|I|         |                               |               |
> |I|N|G|         |                               |               |
> +-+-+-+---------+-------------------------------+ - - - - - - - +
> | big-length(48) if BIG = 1                                     |
> + - - - - - - - +-----------------------------------------------+
> |               | payload(length)                               |
> +---------------+                                               |
> |                                                               |
> +---------------------------------------------------------------+
>
> As a measure of complexity, compare the features that can vary the
> frame-header size:
>
>  a) short/long payload length
>  b) extension data
>  c) short/long extension length
>  d) complete message-length
>
> A Jamie-style extension only has the single complexity-feature (a) to the
> frame-header, while your proposal adds 3 more (b,c,d). Since the Jamie-style
> extension supports all the extension capabilities that yours does, there's
> no added capabilities with the extra complexity added by b,c,d.
>

The point of having a specified way of passing extension data in the framing
is so intermediaries can know where the payload data is even if they don't
understand the extensions that were negotiated.  I personally don't feel
terribly strongly about this.

Regarding d, as mentioned above those arguing against fragmentation objected
to not having the full message length, and after proposing that addition
they seemed happy with the compromise.  As fragmentation support has been
accepted as a requirement


> As a separate issue, your code's signature doesn't capture the essence of
> the problem, because you really need to consider all the APIs when
> evaluating a framing proposal, not just one that happens to be easy to
> implement for the proposal. Your example was a specialized API that really
> only works for utf-8 encoded 8-bit C-strings.
>
> The basic sender APIs that framing needs to support are:
>
> a)  sendTextMessage(char []buffer, int offset, int length); // JavaScript
> 16-bit text send
>

The complication of UTF16->UTF8 conversion is the same in any other
length-based framing options, though supporting fragmentation means you can
have a fixed-size send buffer rather than having to allocate all of it at
once.


> a*) sendTextMessage(byte []buffer, int offset, int length); // utf-8
> encoded text (like PHP)
>
> b)  sendBinaryMessage(byte []buffer, int offset, int length); // binary
> single-buffer send
>

Other than the choice of opcode, these are identical, right?


> c)  class BinaryMessageOutputStream {
>   write(byte []buffer, int offset, int length);
>   close();
>  }
>
> d)  class TextMessageWriter {
>   write(char []buffer, int offset, int length);
>   close();
>  }
>

Again, I don't see these as showing any differences between the various
framing proposals, and fragmentation makes it easier to support streams as
you just write a fragment when the buffer fills.


> The receiver variations are
>
> a) char []receiveTextMessage()
>
> b) byte []receiveBinaryMessage() // if returning utf-8 8-bit strings, this
> is the text receive
>
> c) class BinaryMessageInputStream {
>     int read(byte []buffer, int offset, int length);
>   }
>
> d) class TextMessageReader {
>     int read(char []buffer, int offset, int length);
>   }
>

I would think there would also be an event-driven API, since that is what
the JS API is going to look like -- where there is an onMessage event called
with a complete message after it has been received.  The only public
server-side API I know of (Jetty7) also operates this way.


> A good frame design will support all 5 sender API styles and 4 receiver API
> styles in a straightforward/efficient manner. You can't just pick one API
> that happens to make a frame proposal work.
>

Can you explain how your proposed framing is any different for any of these
APIs?  The difference is about the size of the short frame length, leaving
out the extension information which isn't going to be used in the base
protocol anyway, and leaving out the overall message length which means you
have to guess a buffer size for fragmented messages and grow it if necessary
(or fail the receive if using a too-small caller-supplied buffer).

-- 
John A. Tamplin
Software Engineer (GWT), Google