Re: [hybi] frame length encoding

[Pieter Hintjens <[e-mail]>]

On Sat, Aug 21, 2010 at 9:07 PM, John Tamplin <[e-mail]> wrote:

> So, this is exactly the same as option #1, except it allocates RSV2 to the
> short length field.  As such, it doesn't address the primary complaint
> people had about option #1, which is that an implementation that flushes a
> buffer a WebSocket frame is likely to use something in the 4-64k buffer
> size, and would therefore have to use the 8-byte version.

John, thanks for taking the time to collect all the knowledge we've
gathered so far and steer this group towards consensus.

The advantage of #0 is simplicity.  Frames can be read and written
with the utter minimum of logic: no special cases, no ceilings, no bit
mangling.

Simplicity of the framing is also an essential part of reducing CPU
cycles needed to process frames, which matters in low-latency
scenarios.

Our main use case for high performance messaging is market data and
the generally-accepted average size of a FIX message is 157 bytes.  We
also have Twitter messages at 140 bytes and SMS text messages at 160
bytes.  This argues (gently) that a "short" message is under 255 bytes
rather than 127.

As for 64-bit long frames, it's again a matter of simplicity.  With
32-bit frames some use cases MUST switch to fragmentation.

We've been using the 8/64 framing model for several years in ZeroMQ
and it has proven to be robust and optimally fast, see
http://www.zeromq.org/whitepapers:design-v01#toc5.

At the same time if the group's consensus is towards a more complex
length encoding we'd look at adopting that in ZeroMQ.

-
Pieter Hintjens
iMatix - www.imatix.com