Re: [hybi] frame length encoding

["Shelby Moore" <[e-mail]>]

> On Sat, Aug 21, 2010 at 11:14 AM, Shelby Moore <[e-mail]>
> wrote:
>
>> Correction:
>>
>> long long len = readByte();
>> if (len >= 128)
>>     len = ((len & 0x7f) << 8) + readByte();
>> else if (len == 127)
>>    len = readLongLong();
>>
>
> Ok, so you are suggesting using RSV2 to differentiate 7/63-bit from 15-bit
> lengths, and then within the 7/63-bit option using a value of 127 to do
> so,
> right?

Yes.

I assume that reserved bit is in the size field because all the other
8-bit groups of bit fields were allocated? That wastes 7-bits. We could
recover the 7-bits and keep the reserved bit, if we are willing to have a
header which is not byte aligned (1 extra bit in length), but the code
gets more convoluted:

long long read = readUnsignedshort();
long long len = read >> 7;
read &= 0x7f;
if (len >= 128)
    len = ((len & 0x7f) << 8) + (read << 8) + ((read = readByte()) >> 7);
else if (len == 127)
   len = (read << 8) + ((read = readLongLong()) >> 7);
read &= 0x7f;

> I still dislike using up one of the reserved bits, as it reduced our
> expansion options.


Why did you decide to put a reserved bit in middle of a size field if that
bit is not already allocated to specific use?

Apologies I have not yet read the entire frame header design discussion.


>  Also, as posted previously, I think once compression
> is
> supported, an overwhelming majority of real-world WebSocket messages will
> fit in ~126 bytes.  That is why I am mostly indifferent between option 1
> and
> 2 -- I don't think the 2-byte length value will get used very often (I
> expect mostly small frames with some very large frames where an entire
> file
> is sent in one message), but the cost of adding it over option 1 is
> relatively small (packets of exactly 126 bytes now take 2 bytes longer,
> everything else is the same).


And just in case there is ever a use case for the frames larger than 126
but less than 65535 in size.

Agreed it is only the 126 frame length that has any penalty.

And if we don't need the extra reserved bit (or if we can use a 1 bit
longer header), then there is no penalty at all.

But I guess the logic is that it is better throw away 7 bits for the 126
to 65535 frame byte sizes, than to add 1 bit to frame smaller than 126
bytes in size?

7 bits / (126 x 8 bits) = 0.7%

The break even point is 126 / 7 = 18 byte frame sizes.  We would need to
know the distribution of frame sizes we expect to make a decision of what
is more cost effective.  But it is minor in either case, and the bit
twiddling for non-byte aligned headers has some cost mostly in terms of
software error.


> I do prefer your proposal over Option #3, which also uses one reserved
> bit,

Agreed, gets rid of the error case.

> but otherwise I still prefer either #1 or #2 because I think removing 1/3
> of
> our expansion capability is excessive for what is gained.

Okay but we could get it back if we want to have non-byte aligned headers.


> The reason I feel strongly about keeping reserved bits available for
> expansion, is that we have not yet agreed on how extensions should work.
> If
> we decide to go the nested opcode approach (an extension opcode uses the
> first part of the payload data for its extension data, then the next
> opcode
> etc, until you get to a "data" opcode that uses what is left), then I
> think
> we are going to need to assign both RSV1 bits to the opcode to be able
> to accommodate extensions there (remember that we don't want to have to
> change the base framing protocol at all and this protocol will hopefully
> be
> used for a long time).  If we decide to have per-frame compression
> (certainly still up for debate), then we will likely need a single bit per
> frame to indicate compression efficiently (there are certainly other ways
> to
> do it as well).  If we decide to have separate extension metadata, we will
> need a bit to indicate an extension length and extension data is present.
>  And those are just the extensions we have been talking about the past few
> months -- who knows what things we may come up with in the future that
> might
> be more efficiently implemented if we assign one of the reserved bits
> rather
> than an extension data block or an opcode?
>
> --
> John A. Tamplin
> Software Engineer (GWT), Google
>