Re: [hybi] Frame size

[Mike Belshe <[e-mail]>]

On Sat, Apr 17, 2010 at 1:13 AM, Greg Wilkins <[e-mail]> wrote:

> Roberto Peon wrote:
> >
> >
> > On Fri, Apr 16, 2010 at 5:38 AM, Greg Wilkins <[e-mail]
> > <mailto:[e-mail]>> wrote:
> >
> >
> >
> >     I thought one of the key arguments for a limited frame size,
> >     is that we could avoid the need to consider end points that
> >     could not store an entire frame in a buffer at once.
> >
> > Why would you need to do that?
>
> At the 77 meeting, I raised the point that currently error
> handling in the protocol is not sufficient to deal with
> implementations that impose a maximum frame size.  The spec
> currently says the end point should just close the socket
> if it receives a frame that is too large.
>
> One option is to negotiate maximum frame size, but that
> appears to go against the no negotiation style of this
> protocol.
>
> The suggestion was then made that we pick a small fixed
> maximum frame size, that all implementations would be
> required to handle.
>
> Large messages would then be fragmented and assembled
> from 1 or more frames.    This does not solve the problem
> of end points that cannot handle large message, but it
> does make the problem not a transport problem.  Higher
> layers are better able to handle large messages, as
> they can stream to disk, reject with errors, parse and
> handle as a stream, etc etc.
>

It seems to me that we're over engineering here for what the websockets API
is trying to do.

I disagree that 32bit lengths means that you need to support chunking.  The
application can decide when it wants to chunk or not.  Also, why does a
buffer need to be contained in RAM?  Just because we have a long length
doesn't mean that the payload can't stream.  Lots of protocols work this way
today, like HTTP.  And what happens if you stream a 10GB message in a single
web page?  The browser just hangs up the phone at some point.  There is no
graceful error.  While this is a theoretical problem (which Greg raised), it
is not a practical one.

Unfortunately, once you decide that 32bit lengths are not good enough,
you're running into lots of subproblems:  fragmentation/reassembly, max
frame size, etc.  I think we're just overthinknig the needs to the
WebSockets API.

I propose we make lengths be 32bits and drop fragmentation/reassembly from
this protocol.

Mike





>
>
>
> >     So 2 byte's can still result in a 64k maximum frame size that
> >     some devices might baulk at... but not many.
> >
> >
> > "not many" probably isn't good enough. For the protocol to be a success
> > it needs to work reliably. For clients/servers, that implies that they
> > should be able to start processing before the entire frame has been
> > received. For proxies you just stream the data.. unes you're doing
> > multiplexing for people, in which case you have other concerns.
>
> That is why my suggestion is to go for a 16k max frame size
> that all implementation would have to be able to handle.
>

16KB is small today.  It seems incredibly small for the year 2020.


>
> This frees up 2 bits that can be used to coordinate
> the fragmentation and assembly.
>

*bits*?  Sure, we don't want to be wasteful, but with average internet
speeds of 3.9Mbps in the US *today*, are you sure you've got the right
optimization here? :-)

I'd take the simplicity of nixing fragmentation/reassembly and suck it up on
the extra bits for a longer length field.

Mike




>
>
>
> The alternative approach is to not have fragmentation,
> but to allow the max  frame size be of arbitrary (or very large)
> length so it can always handle a message or arbitrary (or very
> large) length.  Then have reasonable error handling to cope with
> too large messages/frames.
>
> The problem with this approach, is the sending of a too large
> to handle frame, breaks the ability to communicate bidirectionally.
>
> Imagine if both client and server both start sending too large
> to handle frames at the same time.   Neither will be able to
> send too-large errors because they are sending large frames
> and to their knowledge the other side is handling them.
>
> Fragmentation of large messages into frames gives an
> endpoint the ability to terminate the transmission of
> a too-large-message or even to interleave control frames
> containing errors.
>
>
>
> regards
>
>
> _______________________________________________
> hybi mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/hybi
>