Re: [hybi] Multiplexing in WebSocket

[Ian Hickson <[e-mail]>]

On Sun, 25 Oct 2009, Greg Wilkins wrote:
> >
> > The main difference is that most people won't manage UTF-8 encoding. 
> > In Perl, for instance, there's the "Encode" library that you can just 
> > pass byte strings to to have them converted into UTF-8, and 
> > vice-versa.
> 
> Forgive my total lack of knowledge of perl, but given the availability 
> of the encode library in perl, I see little difference in the complexity 
> of sentinel marked code which is:
> 
>   $octets = encode("utf-8", $message)
>   write ($octets)
>   write (oxff)
> 
> versus length encoding code which is
> 
>   $octets = encode("utf-8", $message)
>   write ($octets.length)
>   write ($octets)

Yup, in Perl, once you have encoded the string correctly, getting the 
lenght is relatively easy. However, I expect many people won't be 
explicitly encoding the string at all; they'll just output the string 
straight to the socket and have the socket encode the UTF-8. Many servers 
don't need to know what is going on, since they can just be opaque to most 
of the data and just use ASCII for the "command" part of their API.


> They will need a method to correctly encode the length as an unlimited 
> integer, but they will probably need that anyway as you currently 
> specify that length encoded framing is part of the standard (stacked 
> out).

They only need to support that if they will be using the binary part of 
the protocol.


> > We have binary frames already staked out in the protocol, waiting for 
> > support in JS for binary.
> 
> I don't see why an IETF protocol proposal needs to wait for the js 
> implementation?

The only reason this protocol exists is for the JS API. If it wasn't for 
the JS API's requirements, TCP itself already addresses the needs.


> Since the binary framing is about the same level of complexity, is more 
> capable and is already stacked out - they why not avoid the complexity 
> to two different framings and just use length?

Getting the length of strings is non-trivial in many environments 
(certainly compared to getting the length of binary data). Most servers 
will never have to deal with the binary stuff at all. And you're much more 
likely to have buffer overrun bugs if you use a length prefix followed by 
a variable-length encoding of text.


> JS is free to limit it's usage to just utf-8 as long as it likes.  But 
> JS is not the only client language that is going to use an IETF 
> sanctioned bidirectional protocol.

I sure hope nobody is hoping to use WebSocket with anything other than JS 
-- it's be a complete waste of time to use WebSocket for anything if we're 
just talking about an arbitrary client-server protocol. We already have 
TCP for people who want to write their own subprotocol, and we already 
have BEEP, Jabber, and a multitude of other transports for people who want 
a more structured protocol to tunnel over.

Now, granted, some people might want to speak to WebSocket services that 
were originally intended for JS scripts from command-line tools, just like 
people talk to JSON services that use HTTP cookies for authentication from 
command-line tools, and that is supported by the protocol, but that's 
certainly not the primary use case. There'd be no point having features 
that are _only_ usable by people _not_ using JS scripts in Web pages -- if 
you want to do something WebSocket can't do, then just use raw TCP.


> >> * A mobile phone that is restricted to a single outgoing
> >>   connection (not uncommon) so the browser wants to
> >>   transport HTTP over the websocket connection
> > 
> > Could you elaborate on this use case?
> 
> I unfortunately don't have all the details, but I have been a little 
> involved in the development of a comet system for a large mobile phone 
> company for global deployment.
> 
> One of the issues that we had to deal with was a single connection limit 
> that apparently affects many millions of phones.  This was a restriction 
> for a non browser application running on the phone with direct access to 
> the TCP/IP layer.
> 
> However I'm unclear if the limit was imposed by firmware in the phone or 
> carriers.

Well if there's a limit of a single TCP connection, WebSocket is a lost 
cause anyway (since you'd block all XHR, HTTP, etc, while the page was 
open). WebSocket is intended for systems capable of running a full Web 
browser that can run XHR-based apps today.


> >> * Something that none of us has thought of yet
> > 
> > Once we think of it, we can address it. There's no point trying to 
> > optimise for something we don't know about; we have no way to know if 
> > we are optimising in the right way.
> 
> I fundamentally disagree - and I think that this difference is probably 
> at the root of most/all our other differences on the protocol.

Possibly.


> I think the wait and see position you advocate is entirely correct for 
> the websocket API.
> 
> But i think it is entirely wrong for the websocket protocol.
> 
> A flexible and extensible protocol can be restricted to known uses by 
> the API.  You then do not need to update the entire network 
> infrastructure for each and every new idea.  Your attempt to "stack out" 
> the binary protocol is justified by avoiding exactly this type of 
> breakage.

I agree that if there are things we know we will need to support, that we 
should make sure we can support them. I agree that we should give 
ourselves forward- and backward-compatible extension points (e.g. the 
handshake is extensible and we have 253 frame types still available).

However, there's no point trying to design the protocol to support every 
possible extension mechanism, because I *guarantee* that you'll miss the 
one that you really need, however hard you try, and in the meantime you'll 
have a protocol with lots of extra complexity that never helps anyone.

Consider XML and XML Namespaces. It provides a variety of extension points 
-- you can add new elements in a new namespace, an element can have new 
attributes added, etc etc etc. However, XML doesn't support inline binary. 
It doesn't support an element having two element types. It doesn't support 
elements overlapping each other. It doesn't support attributes having 
multiple values. All of these are things that have been needed by various 
groups at one time or another, despite XML being branded as the ultimate 
extensible markup metalanguage.

So now we have XML and XML namespaces, together making a really expensive 
mechanism, and yet it doesn't do what people want, despite being designed 
to be extensible. I think that's not a good state to be in.

(PS. Stake out, not stack out. :-) )


> > To send a stream of images with WebSocket, you will need to wait until 
> > JavaScript gives you a way to get the image data in a usable form. I 
> > certainly wouldn't recommend sending it as text, that would be stupid.
> 
> I think this is another fundamental disagreement.
> 
> I think the IETF should be considering a bidirectional solution for all 
> web clients - not just for HTML5 JS clients

What's wrong with TCP, BEEP, or Jabber?


> If you are designing a non-extensible protocol that only supports HTML5 
> websocket API, then you may well end up with something like the 
> websocket protocol.

WebSocket is extensible, just not in the ways you want it to be 
extensible. But other than that, yes, that's what I'm designing.


> But if you want something that is extensible and supports 
> bidirectionality for any web client, then I think then it will look 
> substantially different.

Wouldn't it look pretty much exactly like TCP, BEEP, or Jabber?

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'