Re: [hybi] asychronous HTTP [was:Apples and Orangutans]

[rektide <[e-mail]>]

On Wed, Apr 08, 2009 at 08:47:29AM -0700, Mark Lentczner wrote:
> From the point of view of our customers, the application level authors 
> (both client and server side), I think there are two major questions they 
> would ask to characterize any potential protocol:
>
> 1) What is the data model?
> 	Minimal (binary blob and/or UTF-8 string)
> 	Structured (JSON object or XML stanza)
> 	Message (data and meta-data)
>
> 2) What is interaction model?
> 	Event (datagram at the senders whim)
> 	Request/Response (request at senders whim, with response from the  
> receiver)

A messaging protocol ought be able to meet all of the interaction & data models above, 
ought be designed to convey any content, and potentially ails one of the greatest pitfalls 
of the web: the two/multiple connection limit[1].  

If the web ecosystem evolves towards w3c widgets and integrative software networks ala 
OpenSocial, the connection limit must be addressed.  And not by juvenile hacks to permit 
more connections (eg by creating subdomains); the real answer is that the web needs a 
multiplexed communication channel.  I can think of nothing more suitable for holding multiple 
conversations over a single wire than a messaging protocol.

Recently in #persevere (April 7), reflecting on HyBi:
<rektide> i really love greg wilkins constant chime ins that, http is a great model, why
break complete from it

HTTP already meets many of the critereon for being a suitable messaging protocol, and I
think this is of capital importance.  HTTP pipelining allows asychronous data sending, Kris
Zyp has developed HTTP Channels[2] to permit decoupled asychornous data recieving, and if
these are leveraged together, it leaves only the outstanding issue of proxies and multiplexing:
adapting HTTP to be a multichannel bidirectional messaging system supporting meta-data. 

Kris' model in HTTP Channels introduces a client-id and a sequence-id that can be used to
maintain conversations while clients are hurling a stream of pipelined HTTP requests and a
server is sending data to the clients in an out of order manner.  These tokens are sent as
HTTP headers, and although not presently a part of the server response in Kris' HTTP Channels 
spec, I believe ought be adapted as the primary multiplexing mechanism.

The one issue I think isnt particularly well addressed is broadcast messaging; can the
server send events to multiple or all client-id's, and how do these broadcast messages get
serialized?

Assuming the proxy issue can be adapted to be non terminal, assuming a persistent TCP
connection is not a deal breaker (i.e. a non connection oriented transport such as UDP is not
required), it ought be possible to fully extend HTTP to be a asychronous bidirectional
multiplexed communication channel, and further, I believe it would be possible to implement
a client side implementation on top of existing web browsers entirely client side (although
browser code for strictly enforcing multiplexing constraints would be "a good thing").

[1] http://www.openajax.org/runtime/wiki/The_Two_HTTP_Connection_Limit_Issue
[2] http://cometdaily.com/2008/05/13/http-channels-2/


> While issues of data framing, efficient use of TCP, and ease of  
> implementation, will influence their view, I don't think they will be  
> nearly as important as these first two questions[*].
>
> The protocols we're discussing can be characterized by these two  
> questions:
> 	WebSockets: Minimal, Event
> 	BOSH, Bayeaux: Structured, Event
> 	rHTTP:  Message, Req/Resp
>
> Other protocols can be fit into this framework, with perhaps some  
> extension:
> 	XMPP: Structured , Event
> 	TCP: Minimal, Stream
> 	SMTP: Message, Event
> 	HTTP: Message, Req/Resp

To emphasize, we already have the tooling to make HTTP sending and (most of the tools for
making) HTTP recieving asychronous.  This Event model HTTP still supports Req/Resp http
communication as well.


> Now, while clearly any of these be used to implement the others, there  
> is a cost. Where simpler models are implemented with more complex, then 
> there is overhead on the wire and processing. Where more complex models 
> are implemented on simpler, then the burden falls to the application 
> developer.

The cost savings I'd like to see most is being able to leverage HTTP, by simply removing
its fault of being so sychronous & undirectional.  If anyone believes a fully duplex multiplexed
communication channel is of itself insufficient or ill suited for your needs, please speak 
as to why.  If you find the HTTP framing unsuitable, eg a header & a payload, that too would
be intesting to hear.


Good day all, 
rektide