Re: [hybi] Frame size

[Jamie Lokier <[e-mail]>]

Mike Belshe wrote:
>    On Sun, Apr 18, 2010 at 10:16 PM, Scott Ferguson <[1][e-mail]>
>    wrote:
>      I think chunking for many/most applications will be similar because
>      it's a pretty generic need, like HTTP's chunking is.  In the case
>      of HTTP, many developers don't need to be aware that their output
>      is getting chunked; it just works. I'd expect the same to be true
>      for the simple frame/message model. Assuming it does cover the
>      common cases, it's a useful abstraction to give users.
> 
>    So, given the WebSockets API, how would chunks be used?  The
>    application calls send(DOMString).  There is no need for chunking
>    here, because the full string (and it's length) is already available.
>     Or did I miss something?

The chunking is for the benefit of being able to splice in things like
graceful close, some kinds of error reponse, and any other control
messages as Greg described.

There are other uses too.  Let's use an analogy.  From HTTP (RFC2616):

    8.2.2 Monitoring Connections for Error Status Messages

	  An HTTP/1.1 (or later) client sending a message-body SHOULD monitor
	  the network connection for an error status while it is transmitting
	  the request. If the client sees an error status, it SHOULD
	  immediately cease transmitting the body. If the body is being sent
	  using a "chunked" encoding (section 3.6), a zero length chunk and
	  empty trailer MAY be used to prematurely mark the end of the message.
	  If the body was preceded by a Content-Length header, the client MUST
	  close the connection.

Note the part where a request can be aborted early and the connection
reused _only_ if it was encoded with chunks.  Analogously, it's not
hard to imagine adding an abortMessage() function to some future
version of the WebSocket API, so that applications can abort sending a
large message when their other side has signalled that the message is
not needed.

>      To tie this into the KISS requirement, it's a way to keep the
>      client/server APIs and application code simple for the simple
>      cases. Applications don't need to know about frames/chunks, just
>      messages.
> 
>    KISS, I think, would have solve the problem in the simplest way to
>    support the WebSockets API.
>    Are there really other protocols to be solved with the same websockets
>    framework?  (if so, let's list them!)  I think we're all in
>    disagreement because we haven't agreed on what problem we're solving.
>     If we agreed that the problem we're solving is providing a protocol
>    to support a WebSockets-like API to web developers, we'd likely be
>    done.  Another option would be to say we don't like the current
>    WebSockets API proposal, and offer up an alternative API which we're
>    trying to support (maybe one which does streaming or needs chunking?).
>    But I haven't seen that either.

You seem to keep mixing chunks with the API.  The point of chunks is
to be completely absent from the API, so that the WS implementation
can provide things that are better handled by the WS implementation
itself, so the application doesn't have to.

For example, reception of graceful close permits applications to
safely open a new connection and retry operations.

Without graceful close, applications cannot distinguish network errors
from remote end deciding to close an idle connection(*), which forces
applications to write *more* code to handle a normal situation.(**)

Without transport-level chunking, graceful close is not available in
some situations.

Greg's too-large-error scenario is also particularly interesting
because without handling it, some applications will continuously loop
retrying and failing.

(*) Not closing idle connections isn't an option on servers, because
they can't distinguish idle connections from dead ones, so will
consume all server resources over time if they don't do that.

(**) It could be application's own graceful close timeout messages,
keepalives, or sequence numbers with persistent history, or whatever.
Point is it forces the application to do something tricky, or be fragile.

>    My biggest fear is that a generic protocol will take a very high
>    burden of proof for implementation and deployment because it will
>    offer so many features and that this time will only delay solving the
>    one problem we do know of (fix hanging GET) while not really solving
>    any other specific problem either.  By contrast, if we stay focused on
>    the problem at hand, coming to agreement on the WebSockets protocol
>    seems quite achievable.

The subtext is that WebSockets doesn't solve the problem at hand.

See the recent post from Apache folks saying that WebSockets won't be
deployed by them in its current form because it creates more problems
than it solves.  Especially due to the large number of connections.

In its current form, WebSockets (after the handshake) seems to be a
fair match for the networks of the 80s and early 90s.  These days, you
cannot even rely on a TCP connection being reliable while you are
using it.  Remember that it's trying to go over port 80/443 because of
environments where all other ports are blocked by policy.  Those ports
often have proxies on them - and even on port 443, although they
cannot inspect the plaintext, proxies still expect a HTTP-like
communications pattern or they cut off the connection.

-- Jamie