Re: [hybi] frame length encoding

[gustav <[e-mail]>]

On 22 August 2010 13:31, Shelby Moore <[e-mail]> wrote:

> Just got back from 2-mile jog, nice way to process and focus thoughts...
> (due the fact that the brain solves the N-dimension space via annealing,
> so doing something entirely different can jolt your Newton gradient over a
> hill into a deeper N-space global minima)
>
> below for the key point...
>
> >>>  The only thing that truly matters are what kinds of messages we want
> >>> to
> >>> handle with the protocol.  My point is that there's no reason for a
> >>> websocket client or server to handle a single message >4GB.
> >>
> >>
> >> You won't have to.  You will tell the other end your maximum frame size
> >> on
> >> the connection handshake, then the other end will re-fragment if
> >> necessary.
> >>
> >>
> >>>  For my chat
> >>> server that I'm building, any incoming message larger than 64KB is
> >>> likely
> >>> to
> >>> be ill-intended, and that's a very generous threshold.  Clearly the
> >>> threshold would be different for someone else's application.
> >>
> >> Apologies I forgot to make the above point in my prior reply. I could
> >> have
> >> been more concensus driven with that.
> >
> > [snip]
> >
> >>
> >> I do agree that I should not have made the point that you need to
> >> process
> >> 64-bit frames.  That would not be a free market. Rather we must have
> >> re-fragmentation, so that no WS server has to be forced into any maximum
> >> size.
> >
> > Given the above point, those who are arguing against the CPU load to test
> > for value 126 and 127 to fork to 16-bit and 63-bit frame size of Option
> 2,
> > we instead could consider adding a byte to the header and give 1 to 32766
> > sizes without any test.  The end could set its maximum frame size to
> 32766
> > on connection handshake, and then it never needs to perform that logic in
> > CPU.  They just load 2 byte size field and use it.
> >
> > That eliminates the 16-bit size field of Option 2.  It is extends the
> size
> > field of Option 1.
> >
> > That would cost us an extra byte on every frame, but the minimum frame
> > size is 3 already, and those who were arguing that point, said that
> > network speed will increase faster than CPU because there is no way to
> > parallelize (use multi-cores) for a serial channel that has no rewind
> > feature (unless you buffer and use pipelining perhaps?).
>
>
> This could be referred to as Option 1/15-bit.
>
> Note that although the re-fragmentation will still incur CPU overhead for
> large frames, the point was that this new option eliminates the CPU
> overhead when there wasn't going to be re-fragmentation (most frames' byte
> size smaller than 15-bit).  And we must have the re-fragmentation any way,
> because some servers simply won't be able to handle large frame sizes for
> any number of possible reasons.
>
> So if we place important priority on the possibility that network speeds
> will outpace CPU speeds (due to multi-core point), then our choice boils
> down to either Option 1/15-bit or Option 1/7-bit.  But 7-bits seems aweful
> small as a useful maximum size.  So I think Option 1/15-bit is our only
> choice that will bring concensus?
>
> And the Option 1/15-bit is actually more efficient than Option 2, for
> frame size between 126 to 32767.  It is equally efficient for > 32767.
> And it only costs 1 extra byte below 126 in size.
>
> And what happens if we don't do Option 1/15-bit and indeed network speed
> has a radical burst upward relative to single-core CPU speed?  See my
> point about a possible radical shift in the realized market speed soon due
> to global financial rebalancing via hyper-inflation (networks here may
> have fast enough CPUs but overloaded pipes due to fact that pipes are not
> regulatory subsidized here like they are in the socialized "rich"
> countries):
>
> Where is the _proof_ of network speed is outgrowing cpu speed on the long
term ?.
Its imo more about some people having a hard time to design efficient
scalable code and hence even modern multi core CPUs becomes a rock wall  for
their throughput.

Should doomsday theories, or the presumed but unquantified incompetence of
the general programmer be the basis for a new protocol ?. I don't think so,
but its just me.

regards
 gustav trede