Re: [hybi] frame length encoding

[John Tamplin <[e-mail]>]

On Sat, Aug 21, 2010 at 1:31 PM, Shelby Moore <[e-mail]> wrote:

> I assume that reserved bit is in the size field because all the other
> 8-bit groups of bit fields were allocated? That wastes 7-bits. We could
> recover the 7-bits and keep the reserved bit, if we are willing to have a
> header which is not byte aligned (1 extra bit in length), but the code
> gets more convoluted:
>

It isn't in the size field, it is a separate reserved bit.  It is where it
is because there was a request that the opcode be aligned on a nibble
boundary for easily reading a hex dump (admittedly minor benefit) -- any of
the fields could be rearranged however you desire without impacting anything
functionally.  The base header has 16 bits total, none of them wasted
(though 3 reserved, plus additional opcode values reserved).


> Why did you decide to put a reserved bit in middle of a size field if that
> bit is not already allocated to specific use?
>

The reserved bits are reserved because we haven't decided how to use them.
 Some discussions we have had about frame-by-frame compression or extension
metadata would need a single reserved bit in the header, the opcode field
could be extended, and there are likely things we haven't thought of yet
that could use it.


> And just in case there is ever a use case for the frames larger than 126
> but less than 65535 in size.
>
> Agreed it is only the 126 frame length that has any penalty.
>
> And if we don't need the extra reserved bit (or if we can use a 1 bit
> longer header), then there is no penalty at all.
>
> But I guess the logic is that it is better throw away 7 bits for the 126
> to 65535 frame byte sizes, than to add 1 bit to frame smaller than 126
> bytes in size?
>
> 7 bits / (126 x 8 bits) = 0.7%
>
> The break even point is 126 / 7 = 18 byte frame sizes.


Depending on the app, more than 98% of real-world WebSocket frames should
fit into 126 bytes after we add compression support.  Paying one bit for
those 98% seems a poor trade to save 7 bits on the <<2% frames which would
be between 127 and 64k.  There is also a threshold effect here -- if we
never need that reserved bit, then it was free to use it.  However, if we
turn out to have needed it, it will cost us a minimum of 1 byte to get it
back by adding an extension opcode.


> We would need to know the distribution of frame sizes we expect to make a
> decision of what is more cost effective.  But it is minor in either case,
> and the bit twiddling for non-byte aligned headers has some cost mostly in
> terms of software error.


I published such data for GWT Quake and Wave in April -
http://www.ietf.org/mail-archive/web/hybi/current/msg01810.html - since
then, I have add percentiles of frames <=126 bytes, assuming stream-based
compression with a 12-bit window: 98.88%/99.03% for GWT Quake S->C (2
trials), 99.98%/99.97% GWT Quake C->S, 98.22% Wave C->S, 98.31% Wave S->C.

Okay but we could get it back if we want to have non-byte aligned headers.


No, I think that is a non-starter, as it adds a great deal of complexity for
very little gain.

-- 
John A. Tamplin
Software Engineer (GWT), Google