Re: [hybi] how do we move forward on agreeing on framing?

[John Tamplin <[e-mail]>]

On Thu, Aug 19, 2010 at 1:02 PM, gustav trede <[e-mail]>wrote:

> Only one thing that i fail to understand, regarding the payload length
> encoding:
> What technical reason makes it so much better then the original binary
> frame length encoding( if (byte[i]&0x80)==0x80) to detect the length bytes)
> that its rather massive waste of bandwidth is justified ?.
>

There are a couple of problems with it:

   - you can have an arbitrary number of leading 0x80 bytes, so there is no
   single representation of a length
   - you can have an arbitrary number of length bytes, which means to
   properly support the spec you have to implement multi-precision arithmetic.
    In practice, implementations will likely use 32 or 64-bit ints, and then
   some attacker will exploit those implementations.  With this proposal, every
   implementor knows they must support 64-bit lengths

If you fix those problems by disallowing leading 0x80 bytes and making the
maximum number of length bytes 9 (for the same 63-bit maximum length), you
now have 9 steps in the length, and I don't think they are justified.  I
think the 8 bytes of length is not significant for a larger message, so in
the worst case you have 127 payload bytes, so the overall message length is
137 bytes (2 byte fixed header, 8 byte extended length, payload) -- giving
an overhead of 7.3%.  More typical payload sizes will result in overheads of
the extra length bytes of well under half a percent.

The two length encoding mechanisms compare like this (ignoring the
opcode/flags byte, and any TCP/IP headers), with overall overhead
percentages for each:

   - 0-126 octets - both take one additional byte
   - 127 octets - v76 takes 1 byte, this proposal takes 9 (1.6%, 7.3%)
   - 128-16383 octets - v76 takes 2 bytes, this proposal takes 9 (2.3%-.02%,
   7.2%-.06%)
   - 16384-2097152 octets - v76 takes 3 bytes, this proposal takes 9
   (.03%-.0002%, .06%-.0005%)
   - ...

You can already see that there is no significant difference in overhead for
the larger packet sizes, yet we have the complexity of all these different
size length values for little benefit.  When you consider TCP/IP headers as
well, the difference in overhead is negligible.

Yes, it is more efficient to have a 2-byte length format for small packets
just above 127 bytes, but is that worth the additional complexity of extra
length steps?  Profiling of real-world WebSocket apps (admittedly, there are
few currently) suggests that there will be lots of very small packets
(especially once compression is supported), and a few larger packets.  So I
think that matches well with the proposal to provide exactly two steps of
length sizes.

-- 
John A. Tamplin
Software Engineer (GWT), Google