Re: [hybi] Multiplexing in WebSocket

[Greg Wilkins <[e-mail]>]

Ian Hickson wrote:
> On Fri, 23 Oct 2009, Greg Wilkins wrote:
>> Ian Hickson wrote:
>>> On Thu, 22 Oct 2009, Joshua Bell wrote:
>>>> * I seem to recall that one of the desires for sentinel-based frames 
>>>> was to allow octet streams for which the length was not known in 
>>>> advance.
>>> No; the only reason for sentinel-based frames was to not rely on 
>>> authors having to determine the length of their UTF-8-encoded strings, 
>>> which in many environments can be easy to get wrong.
>> Authors that can't determine the length of a UTF-8 string are not 
>> exactly the sort of developers that should be implementing network 
>> protocols.
> 
> Wow.
> 
> I cannot stand behind such a judgmental statement. Personally I would like 
> to make this kind of thing accessible to as many people as possible.

I don't understand your Wow.  Buffer overflows have historically
been one of the biggest security issues with any services exposed
to the internet.

Utf-8 will mostly have a 1 character to 1 byte mapping (at least
for english speakers), so many programmers will write code
that they think works by allocating byte buffers the same
size as their characters strings.  But then when giving
multi-byte characters they will suffer buffer overflows.

I've made this programming error myself many times! So perhaps
I too am one of the authors that should not be writing network
protocols.

The level of programming skill needed to manage meta data
or channels is entirely of the same order of magnitude as
managing utf-8 encoding.

So my point is probably better expressed by saying that
programmers who are able to write a websocket implementation
that correctly handles multi-byte utf-8 characters, will
be entirely capable of handling the additional "complexity"
of some of the additional capabilities being discussed here.



>> It seams an entirely reasonable simplification of websocket to use only 
>> length limited framing and to use the type byte to indicate such things 
>> as content charset
> 
> What's the use case for doing anything other than UTF-8?


Even limiting myself to js in the browser, I can think of
of reasons that a content type other than UTF-8 would be
desirable.

* Compressed UTF-8

* UTF-16 for those whose language happens to use a lot of >2 byte
  utf-8 characters

* UTF-16 for those that can't deal with the uncertainty and/or
  unpredictability of the length of a UTF-8 string



If we consider what a browser itself might like to do with
websocket, or a none- browser client might like, then we have:

* Mime encoded content for those that want to send other
  content types down a stream.  Images, sounds etc.

* Some other framed content for those that want to implement
  multiplexing on top of websocket (as you advocate) but don't
  want to have to do it with text based framing.

* Anybody that wants to send any content with 0xFF in it
  and does not want to base64 code their entire content
  as a result

* A mobile phone that is restricted to a single outgoing
  connection (not uncommon) so the browser wants to
  transport HTTP over the websocket connection

* Something that none of us has thought of yet


You also say that future multiplexing (or other complex
things), should be built on top of websocket. Yet  when
anybody asks for additional content types to help do that,
you say they are not needed and it all can be done in UTF-8.


I find this amusing, because Websocket has gone against the common
convention for web protocols being humanly readable ascii encoded.
Instead it is a byte squeezed binary protocol.

Yet when it comes to building protocols on top of websocket, then
all of a sudden you are an advocate of text encoded protocols.

To send a stream of images with websocket, you will need to
have a sentinel framed UTF-8 message that contains a JSON or
mime header to give the content type of the base64 or hex
encoded image.     That's 3 envelopes around each image!
Ouch!!!


regards