Re: [hybi] Call for interest: multiplexing dedicated for WebSocket

[Takeshi Yoshino <[e-mail]>]

On Mon, May 27, 2013 at 9:51 PM, Tobias Oberstein <
[e-mail]> wrote:

> >>Having those buffers in user space (in app) allows one to tune the
> buffer size _per stream_. Thus, we can have large buffers for streams that
> carry mass data, and small buffers for chatty streams. Thats a real
> >>advantage.
>
> >Exactly.
>
> In the meantime, I have read more about modern TCP/IP stacks.
>
> E.g. beginning with FreeBSD 7, the TCP/IP stack auto-tunes the TCP
> send/receive buffer sizes _per socket_. This is controlled via:
>
> net.inet.tcp.sendbuf_auto=1         # TCP send buffer size autotuning
> (default: on)
> net.inet.tcp.sendbuf_inc=16384      # TCP send buffer autotuning step size
> net.inet.tcp.recvbuf_auto=1         # TCP receive buffer size autotuning
> (default: on)
> net.inet.tcp.recvbuf_inc=524288     # TCP receive buffer autotuning step
> size
> net.inet.tcp.sendbuf_max=16777216   # TCP send buffer maximum size (tune
> up for Fat-Long-Pipes)
> net.inet.tcp.recvbuf_max=16777216   # TCP receive buffer maximum size
> (tune up for Fat-Long-Pipes)
>
> """
> TCP Autotuning
>
> Beginning with Linux 2.6, Mac OSX 10.5, Windows Vista, and FreeBSD 7.0,
> both sender and receiver autotuning became available, eliminating the need
> to set the TCP send and receive buffers by hand for each path. However the
> maximum buffer sizes are still too small for many high-speed network path,
> and must be increased as described on the pages for each operating system.
> """
> http://fasterdata.es.net/host-tuning/background/
>
> Apart from autotuning, there is also the option to control send/receive
> buffer sizes _per socket_ using setsockopt() with SO_SNDBUF / SO_RCVBUF.
>
> ===
>
> Given above, how would the following 2 approaches compare?
>
> a)
> - Client 1 <=> Server 1 connected with 2 TCP connections.
> - TCP autotuning on both client and server OR manually set buffer sizes:
> small for TCP-1, and large for TCP-2
> - First TCP _only_ carries "chatty" traffic, and second TCP carries "mass
> data" traffic.
>
> b)
> - Client 1 <=> Server 1 connected with 1 TCP connection
> - TCP autotuning OR large buffers set system wide
> - multiplexing done over the single TCP that carries both "chatty" and
> "mass data" traffic and
> - the multiplexing is further optimizing buffer size (app level) and does
> priorization of traffic (chatty = high-prio, mass = low-prio):
>
> "Compare": I am mostly interested in the end-user experience:
>
> i) Is the chatty traffic still low-latency, even in the presence of
> concurrent mass-data traffic?
>

a) it depends on process scheduler and network queue design of the server
and client, I think. I don't know much about per-socket buffer autotuning,
but it probably just improves memory usage and throughput but doesn't have
much effect on latency of chatty traffic?

b) the endpoints need to cut mass data into smaller chunks to reserve slots
for inserting chatty traffic. e.g. if acceptable delay for chatty traffic
is x ms, set the maximum send unit of mass traffic to (current through put)
* x.

To allow one to control the peer's message scheduling, both per-socket
buffer size and ws-mux flow control are insufficient, I think. Needs to
exchange ToS info.


> ii) Is the throughput of mass-data traffic concurrently to the chatty
> traffic still (nearly) as high as when only doing mass data transfer?
>

I think minimum delay requirement for the chatty traffic determines
fragmentation on mass-data traffic regardless it's done in OS's network
stack or ws-mux code.


> >>How about having a shared Web worker in the background that handles all
> messaging for the app over a single WS?
> >One restriction of Web Worker is that we cannot share a worker between
> webapps with different origins while WebSocket can connect to a host
> different from its origin.
>
> Ah, right. Same origin policy. Thus, with WS-MUX, there is _more_
> potentially TCP sharing than whats possible with shared WebWorkers.
>
> However, for a given single app (from a single origin) running in multiple
> browser tabs, the shared WebWorker seems not so bad. Works today. No need
> for WS-MUX or HTTP 2.0 MUX + WS over HTTP 2.0.
>

Yes. Thanks for reminding us about shared worker approach. We haven't
evaluated it well.


> >For load balancer vendors, it's better that multiplexing is standardized
> than each service provider develops its own.
>
> I don't have experience running G scale infrastructure. However, I'd like
> to understand why LBs need to have a look into WS at all .. why not Layer-4
> LB? Doing balancing based on hash of source IP?
>

Traffic for various services would be multiplexed into one TCP. We wanna
demultiplex it and forward to designated backends. This means that there're
again lots of TCP connections between LB-backends, but we can make any
optimization there as we want.


> >Basically multiplexing should be done as close to network as possible. If
> it's done in application layer, for example when compression is used, such
> a load balancer need to decode the compression to see the >multiplexing
> information. That doesn't make them happy.
>
> This seems to assume that
>
> - either the LB is done above layer 4 and/or
> - the logical WS connections contained in the physical WS would need to be
> balanced to _different_ backend nodes.
>

Yes. Sorry that we haven't told the story in our mind in detail.


> >As Google, our motivation of mux is to make sure that total meaningful
> traffic per server cost (memory, port, machine, etc.) small enough compared
> to HTTP/Comet approach.
>
> I see. For me, I am mostly interested in the use case: 1 app doing both
> chatty and mass-data coms.
>

OK