Re: [hybi] WS framing alternative

[Ian Hickson <[e-mail]>]

On Fri, 30 Oct 2009, Greg Wilkins wrote:
> Ian Hickson wrote:
> > 
> > The server will know what the clients will generally be sending -- 
> > e.g. a chat client is going to be expecting messages in the range of 
> > half a kilobyte or less, generally.
> 
> Why will the server know what the client will be sending?

It defines it.

Consider TCP. The server knows what kind of traffic the clients will be 
sending -- it depends on the kind of server. An IMAP server knows that the 
client will be trying to talk IMAP. An HTTP server knows that the client 
will be trying to talk HTTP. An SSH server knows that the client will be 
talking SSH.

The same applies to WebSocket. A Wave server knows that the client will be 
talking Wave. A VoidWars game server knows that the client will be talking 
the VoidWars game protocol.

If you try to talk IMAP to an SSH server, the server's just going to close 
the connection on you. If you try to talk the VoidWars game protocol to a 
Wave server, the Wave server is just going to close the connection on you.

The Wave server can therefore know that it's only expecting messages of a 
certain length, and that it can just close the connection if the messages 
are too long.


> The application built on top of the server might well know what it is 
> expecting, and maybe that will be of a size within a known range and 
> maybe that wont be able to be influenced by users doing silly things 
> etc.
> 
> but the application knowing and the code that implements the protocol 
> knowing are two entirely separate issues.

Only if you make them separate, which seems to just be inviting trouble, 
given how simple the protocol is.

You're proposing making changes to the protocol to make it easier to make 
dedicated frameworks for the protocol.

I'm proposing making the protocol so simple that dedicated frameworks are 
superfluous.


> The application could tell the server implementation what size to expect 
> - but insert all your arguments against buffer size negotiation on the 
> wire here as arguments against passing buffers sizes from app to server.

My argument against on-the-wire buffer size negotiation ("it's 
unnecessary, both sides already know the protocol") doesn't apply in the 
case you are talking about (since by definition, the information is being 
passed from the part of the server implementation that does know the 
protocol to the part that doesn't know the protocol).


> Besides, you keep saying that the people programming this don't know how 
> to calculate their own messages sizes - so they don't know the length 
> and couldn't tell you anyway.

I'm saying that _some_ people programming this don't know how to calculate 
their own messages sizes. That doesn't mean that none do. Making the 
assumption that we have to cater for inexperienced programmers is not the 
same as making the assumption that we should _only_ cater to inexperienced 
programmers. Experts are also part of the ecosystem.

When designing a protocol for the broad range of authors from 
inexperienced to expert, we have to make sure that the inexperienced can 
get simple results easily, and that complexity is handled by the experts. 
This means making the simple things simple and the complex things more 
complex, rather than making everything a middling level of complexity. 
Making everything equally complex is optimising simplicity for the 
experts, who are the group who (by definition) need it the least.


> > On Fri, 30 Oct 2009, Greg Wilkins wrote:
> >> However, the problem with making this kind of negotiation optional 
> >> (and this goes for my earlier examples of a load balancer 
> >> communicating SSL info and/or node stickyness), is that the current 
> >> WS protocol has no place for meta data to be added in an optional 
> >> manner - so that it can be ignored by implementations that don't 
> >> care.
> > 
> > Sure, you just put them in the frames, as part of the higher-level 
> > protocol.
> 
> But unless you allocate a standard flag for meta data, then 
> implementations that don't implement these optional features will crash 
> because they don't understand these injected frames.

Yes, if you define a protocol and people don't implement it right, you'll 
have bugs. The same applies to any protocol.


> If a ws client connects to a ws server, and neither knows if the other 
> supports some layered protocol - how do they discover that?

If a TCP client connects to a TCP server, and neither knows if the other 
supports some layered protocol - how do they discover that?

The answer is the same.


> >> This limitation of no standard meta-data paths, means that it is nigh 
> >> impossible to introduce features like multiplexing, load balancing, 
> >> fragmentations, HTTP transport etc. as optional additional 
> >> specifications built on top of ws.  Because there is no meta channel, 
> >> simple implementations will treat everything as a message and break 
> >> if there is any new protocol layered on top.
> > 
> > This is either completely false, or we have dramatically different 
> > goals.
> 
> Let's say that we have dramatically different goals.... sometimes that 
> happens and the sort of changes I'm minimally asking for are absolutely 
> trivial to the protocol and you can meet your goals and I can meet mine.

I do not see how to meet my goals of making the protocol the simplest 
possible layer on top of TCP if we make it more complicated than the 
simplest possible layer on top of TCP.

That is, I do not see how our goals could ever be met by the same 
protocol, if I understand your goals correctly.


> > Given the goal of just providing TCP for Web pages, it doesn't really 
> > make sense to have a more involved protocol.
> 
> I'm not asking for a more involved protocol.
> I'm asking for 1 bit to be allocated to indicate that a frame
> contains mime-encoded data.    That is mostly sufficient
> for layered protocols to be built on.

Do you mean a third framing mechanism? Or would it reuse the text or 
binary framing mechanism?

If the latter, then it's unnecessary -- you just have to define that all 
frames a MIME encoded, or all frames are MIME encoded if their first 
byte/character is 0x01/U+0001, or some such. It doesn't have to be at the 
WebSocket protocol level.

The WebSocket protocol level basically is just a reflection of the API. 
Just like the API doesn't have a sendDataMimeEncoded() method, and doesn't 
have a message-mime-encoded event, the protocol doesn't have anything 
special for sending or receiving MIME-encoded data. It's purely an 
application-layer concern. Why would we add a sendDataMimeEncoded() 
method? What problem would it solve for scripts?


> > It *would* make sense if your goal is something else, like tunneling 
> > Jabber over HTTP or something. But that's a different project than 
> > WebSockets. It's probably a perfectly reasonable thing for Hybi to 
> > work on, but it should be articulated as its own project.
> 
> The sensible way to proceed would be to develop a simple but extensible 
> protocol that provides bidirectionality over the web for all types of 
> clients.

I disagree that that would be sensible. I think overgeneralisation is one 
of the biggest mistakes one can make in protocol and language design. My 
experience has led me to be a firm believer in picking specific problems
and addressing them directly, rather than trying to solve everything at 
once (especially when one might not know all the problems one is trying to 
solve -- a common problem when discussing extensibility mechanisms).


> So browsers could add on their origin security model for example.

Security is not a feature you bolt on. It's an architecture that has to 
be designed in from the beginning.


> From a servers/intermediaries point of view, browser are just not that 
> special that they deserve their entirely their own protocol.

Browsers are the only reason for WebSocket existing. If it wasn't for the 
needs of scripts in Web pages, TCP would be a perfectly adequate solution 
for bidirectional communication for me, and I wouldn't be here. (Which 
isn't to say that there aren't perfectly valid other needs that deserve 
their own protocols, but those aren't the needs WebSocket was designed to 
address.)


> > I think you are misunderstanding the point of the frame types.
> 
> yes - kind of on purpose.
> 
> The WS Frame "byte" is really a frame bit, which is also used indicate 
> content-type (or at least content-encoding).
> 
> There are 7 unused bits in that byte.

That isn't an invitation to use them up. :-)


> Are you really saying you need to reserve space for 254 more types of 
> framing?

No, I'm saying that if it was possible, I would be just using one bit to 
carry the bit, and not a whole byte.


> It's not like you are suggesting alternative ways to solve the issues 
> that we are raising.

Could I ask you to more concretely describe your problem so that I could 
more concretely (maybe with actual code examples) describe the alternative 
solution that I am describing?

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'