Re: [hybi] Multiplexing: Pre-AddChannelResponse quota

[Jamie Lokier <[e-mail]>]

Arman Djusupov wrote:
> 
>    Resending would be required when the allocation of a logical channel
>    has failed on the server or the intermediary due to any kind of mux
>    related error. For example: “Maximum number of logical channels per
>    physical connection has been reached “ or “Intermediary cannot reach
>    the destination endpoint”. The current draft requires that if an
>    AddChannelResponse is received with a failed flag, the client attempts
>    to open a new physical connection. Since the browser must keep the mux
>    extension transparent it cannot let the JS application handle the
>    recovery from mux related errors.

If there is a max_simultaneous_handshakes - why not recast that as
"initial new-channels window" - meaning a flow control grant, which is
updated by further grants from the server.  (Very similar to the flow
control per channel).

Then it means initially you can set a number, and subsequently, the
server will control the rate of admission of new channels.  Then it
can regulate it at a smaller or larger value, be conservative if
appropriate until a client has proven not to be an attack, etc.

There's basically zero message overhead, as the grant update would
follow AddChannelResponse if the server wants to maintain a constant
number in flight.

Having redefined max_simultaneous_handshakes that way....

It removes the need to re-send due to "maximum number of channels
reached" - because it can't happen.  The token is a promise.  (If the
server can't take it, there's a fault so it's a reason for a hard error).

"Intermediate cannot reach destination endpoint" ought to be a
non-retry error for the same reason that it would be for an HTTP
intermediary.  I guess this is debatable.

But assuming that debate was won :-) there would be no reason for re-sending.

That said, it doesn't really matter, the client is free to limit the
amount of data it sends before a channel is confirmed, to an amount it
considers reasonable to store.

A goal that I think is worthwhile are zero-latency-creation channels,

That is where a JS would be free to create channels, with no
round-trip overhead to start a request-response pattern over them.

It's currently paradoxical that HTTP (and especially SPDY) use one
_less_ round-trip to send short AJAX-style requests from independent
code (or independent pages), compared with multiplexed WebSocket.

Zero-latency-creation channels would make WS behave at least as well
as AJAX in the small-request-response pattern, which is after all
extremely common.

>    I like the idea of using the total pre-handshake quota for all logical
>    channels. This would allow high throughput channels to utilize a common
>    pre-handshake quota when other channels do not require to send any data
>    prior to receiving an AddChannelResponse. I believe that this solution
>    would be more optimal than having a pre-handshake quota per logical
>    channel.

It's more flexible for the client, which can choose to abuse it or
not.  I like the idea - but... does it make any problems for the
server?

The server may know it has enough memory for N bytes per channel
buffer, but it may not have a data structure which can efficiently
assign M*N bytes to a single channel (for example due to memory
fragmentation or pre-allocation issues, or simply how "malloc" works).
It might also want to prevent M*N for heuristic reasons such as
channel fairness - which it is already responsible for when channels
are up and running.

In my view, the server must have the ability to constrain the initial
parameters of each channel that the client can assume.  It would be
fine to allow the server to grant a total pre-handshake quota, as an
optional extra, but there should definitely be some constraint on
pre-handshake per-channel data.  But I do favour that being non-zero.

>    However, this solution would require the client to subtract the
>    required value from the common quota and specify this value in
>    the AddChannelRequest header, so that the server knows from which
>    initial value to start counting the remote side quota.

Isn't it clear without it?  The client sends pre-handshake data, the
server moves it from the common pool to the new channel buffer,
potentially freeing up space in the common pool (if it wants), and the
server simply treats the data, for flow control window purposes, as if
it came after the channel was set up.  Everyone agrees.

Anyway, the client shouldn't have to pause/delay sending data after
AddChannelRequest, until the response, if there is sufficient quota to
keep sending.  Specifying the amount used in the request would seem to
enforce that delay, which is unnecessary in principle.

>    There should also be a way to restore the common quota, e.g. in
>    AddChannelResponse the server can send a header which would
>    specify the value that should be added to the common
>    pre-handshake quota.

If there is a common pre-handshake data quota, I agree with this.
However, it should not be attached specifically to AddChannelResponse.
Rather, the server should be able to increase this amount granted at
any time, so have its own message.

It parallels what I suggested at the start of this mail, about making
the physical handshake quota be a window that is maintained by flow
control grants from the server, rather than a fixed in-flight
constant, in the same way as per-channel flow control does.

All the best,
-- Jamie