Re: [hybi] NAT reset recovery? Was: Extensibility mechanisms?

[Vladimir Katardjiev <[e-mail]>]

On 20 apr 2010, at 08.03, Pieter Hintjens wrote:

> On Tue, Apr 20, 2010 at 3:40 AM, Jamie Lokier <[e-mail]> wrote:
> 
>> Perhaps the missing thing is that request-response is more complicated
>> in this kind of protocol, because you also have to define a way to
>> _associated_ each response with the request.  That means sequence
>> numbers, or identifiers, or being careful with counting response
>> messages distinct from non-response messages.
> 
> This is pretty much the problem we solved in AMQP by using async KAs.

Is it really necessary to match responses to requests? The point is just to see if the connection is alive, no? All the keepalive would do is tell the server "hey, can you send me something". Logically, if the server already is sending data then it shouldn't need to send a keepalive back (because there's no need to establish liveness). So what interest would we have to be mapping these keepalive messages to each-other? What semantics would that derive? The best I can figure out is give the req/resp latency...

I mean, I can't really see why you'd need to match keepalive "responses" to which keepalive "request" that triggered them. Since we're mainly interested about ascertaining liveness, doesn't the fact that we receive a keepalive suffice? If there's traffic on the line, we can already deduce liveness, and if there's no traffic on the line, we can't send two keepalives in a row (because we'd timeout if we didn't receive some traffic in response to the first one). 

The reason I suggested "echo" though (at least for a basic case) is because it fulfils the "amateur programmer" requirement. The server is completely stateless, no timers, no nothing. This assumption even includes the server not keeping the state of if it's sending or not.

(To put it in RFC terms, if the server receives a KA, it MUST send some data to the client; if the server is already sending application data it MAY skip sending a keepalive back)
> 
>> There are also subtleties: If the other end sends a graceful close
>> message, and then receives a KA request, does it send a KA response,
>> or does the graceful close mean that KA response is among the message
>> types that won't be sent?
> 
> Indeed... having to respond to a KA request makes a graceful close
> much more complex.

I think this problem applies generally. Graceful close was supposed to let the remote side empty its buffer. But suppose it has 500gb in its sending buffer. That's going to take a while to gracefully close, so keepalives are still needed during this time period. And since the side that already sent graceful close won't send any more _data_ it needs to send keepalives.

Once both sides have sent a graceful close, what would be the point of replying to a keepalive? If the connection is alive, it would be closed first. If the connection is dead it's not going to make a world of difference.

(Of course, there is an argument for async KAs here, because if the server sent the graceful close, and the client is flushing a single 500gb frame, it can't send any KA frames, so the server would never echo back a KA in synchronous mode. Perhaps some sort of hybrid solution? It's a tricky question though, because my next point is also a concern...)
> 
>> However for packet efficiency, synchronising timing by using
>> request-response, or another synchronisation method, may still be
>> advantageous.  Even though it's more complicated than async keepalive.
> 
> Complexity is always worth removing unless the overhead is significant
> and since KAs can be entirely switched off when there is traffic at
> all, and then reduced to once per 5 or 10 or 30 seconds, packet
> efficiency seems irrelevant here.

Well, to be precise, my main concern isn't _packet_ efficiency so much as _power_ efficiency. Async keepalive keeps two separate timers on when to send, so in the worst case scenario (both intervals are equally long) some side will send a keepalive at a frequency that will be double that which synchronised keepalives would be. 

In a mobile scenario, what costs the most is radio uptime, not bandwidth. Whenever you need to set up a radio link to receive data, you need to spend a certain amount of time doing control signaling (during which time you're drawing power, but not getting any data). To minimise this, after a data link is set up it stays active for a couple of seconds, before going idle again.

For the purposes of keepalives, uncoordinated keepalives will almost always force the radio to wake up. This is bad enough for uncoordinated keepalives between different applications, but the problem would be doubled if each connection had two uncoordinated keepalives. All it takes is three uncoordinated keepalive timers to drain a modern smartphone's battery in the timespam of a single day -- and this is doing nothing but sending keepalives. If each connection has two uncoordinated timers, that means all it takes is two connections to empty the battery in a single day, and it only goes downhill from there.

This leaves us in the paradoxical scenario where sending keepalives from the client (slightly) more frequently than the keepalive timeout is actually better efficiency, because they can be sent when the radio is already up. But this is way overkill for a trivial WebSocket implementation. My interest is to have a base case of keepalives that is simple enough so that every server supports it, but flexible enough that I can have a smart client (smart phone? har har har) that can coerce the server into behaving well. Or that an even smarter client could cooperate with an intermediary on liveness and avoid many keepalives.

Vladimir

PS: Thanks Jamie for your in-depth replies! I'd like to look into more the per-hop keepalive option, especially if it can be done in an implicit manner.