Re: [hybi] frame length encoding

["Shelby Moore" <[e-mail]>]

Just got back from 2-mile jog, nice way to process and focus thoughts...
(due the fact that the brain solves the N-dimension space via annealing,
so doing something entirely different can jolt your Newton gradient over a
hill into a deeper N-space global minima)

below for the key point...

>>>  The only thing that truly matters are what kinds of messages we want
>>> to
>>> handle with the protocol.  My point is that there's no reason for a
>>> websocket client or server to handle a single message >4GB.
>>
>>
>> You won't have to.  You will tell the other end your maximum frame size
>> on
>> the connection handshake, then the other end will re-fragment if
>> necessary.
>>
>>
>>>  For my chat
>>> server that I'm building, any incoming message larger than 64KB is
>>> likely
>>> to
>>> be ill-intended, and that's a very generous threshold.  Clearly the
>>> threshold would be different for someone else's application.
>>
>> Apologies I forgot to make the above point in my prior reply. I could
>> have
>> been more concensus driven with that.
>
> [snip]
>
>>
>> I do agree that I should not have made the point that you need to
>> process
>> 64-bit frames.  That would not be a free market. Rather we must have
>> re-fragmentation, so that no WS server has to be forced into any maximum
>> size.
>
> Given the above point, those who are arguing against the CPU load to test
> for value 126 and 127 to fork to 16-bit and 63-bit frame size of Option 2,
> we instead could consider adding a byte to the header and give 1 to 32766
> sizes without any test.  The end could set its maximum frame size to 32766
> on connection handshake, and then it never needs to perform that logic in
> CPU.  They just load 2 byte size field and use it.
>
> That eliminates the 16-bit size field of Option 2.  It is extends the size
> field of Option 1.
>
> That would cost us an extra byte on every frame, but the minimum frame
> size is 3 already, and those who were arguing that point, said that
> network speed will increase faster than CPU because there is no way to
> parallelize (use multi-cores) for a serial channel that has no rewind
> feature (unless you buffer and use pipelining perhaps?).


This could be referred to as Option 1/15-bit.

Note that although the re-fragmentation will still incur CPU overhead for
large frames, the point was that this new option eliminates the CPU
overhead when there wasn't going to be re-fragmentation (most frames' byte
size smaller than 15-bit).  And we must have the re-fragmentation any way,
because some servers simply won't be able to handle large frame sizes for
any number of possible reasons.

So if we place important priority on the possibility that network speeds
will outpace CPU speeds (due to multi-core point), then our choice boils
down to either Option 1/15-bit or Option 1/7-bit.  But 7-bits seems aweful
small as a useful maximum size.  So I think Option 1/15-bit is our only
choice that will bring concensus?

And the Option 1/15-bit is actually more efficient than Option 2, for
frame size between 126 to 32767.  It is equally efficient for > 32767. 
And it only costs 1 extra byte below 126 in size.

And what happens if we don't do Option 1/15-bit and indeed network speed
has a radical burst upward relative to single-core CPU speed?  See my
point about a possible radical shift in the realized market speed soon due
to global financial rebalancing via hyper-inflation (networks here may
have fast enough CPUs but overloaded pipes due to fact that pipes are not
regulatory subsidized here like they are in the socialized "rich"
countries):

http://www.ietf.org/mail-archive/web/hybi/current/msg03454.html

Btw, the above is an example of why insurance (surety about future) is a
fraud. Statistics can not see long-tail events:

http://goldwetrust.up-with.com/knowledge-f9/book-ultimate-truth-chapter-1-science-is-never-certain-t148.htm#3159

Some of that abstract discussion I did before this thread was relevant:

http://www.ietf.org/mail-archive/web/hybi/current/msg03348.html

>
> Also what are we using the 0 size value for? Couldn't we add 1 to the size
> field to get the size?
>
> Also for the special case where the maximum frame size is set on
> connection handshake to 32767 (instead of 32766), no need to escape that
> into an 8 byte size field.