Re: [hybi] NAT reset recovery? Was: Extensibility mechanisms?

[Jamie Lokier <[e-mail]>]

[e-mail] wrote:
> Martin Thomson wrote:
> >
> >Furthermore, you don't need a response, TCP provides ACKs.  A 
> >one-way exchange of as little as a single byte should be 
> >enough to keep any bindings alive in (TCP) intermediaries.
>
> This is true. 

It doesn't work.

It's fine for intermediaries: they must be willing to accept data in
either direction as sufficient to prevent disconnection, because it's
quite common that data flows in one direction only for long periods.

But the client won't be able to detect that the connection is down and
and open a new one in a reasonable time with that mechanism.  Two reasons:

1. There is usually no operating system interface for the client to
detect TCP ACKs.

2. Any intermediaries will send TCP ACKs.  These are not confirmation
that the next hops and server are responding.

Remember that keepalive isn't just to keep the intermediaries alive,
it's to inform the endpoints that the link is still alive and doesn't
need recreating too.

> I would say there are two options to do keepalives:
> i) Define a keepalive frame that the client sends. There is no
> websocket level response but the TCP ACK is still enough to keep
> NATs, firewalls etc. open.

Works for intermediaries, but fails in multiple ways as described above.

> Ii) Define a keepalive request frame that the client sends and
> keepalive response that the server uses to respond to it. The
> response is not really necessary for NATs etc., but might help the
> client to notice a broken connection faster. (I'm not sure if this
> is true but I remember this was an argument in another protocol,
> SIP, why keepalive over TCP was done in that way.) The response may
> make the protocol more complicated, though, as people have
> commented.

Yes, massively faster.  TCP will take about 20 minutes if I recall
correctly, to detect send failures.  That's too long for most
interactive applications to detect and recreate broken connections.

But that reponse is redundant, if the server has sent data within the
timeout interval, in most situations.  So why insist on a
request-response pattern if it's adding redundant messages?

> I would not want to make the keepalives peer-to-peer in a sense that
> also the server could send keepalives. I think the client should be
> in charge. The reason is that as a client developer for a mobile
> device I would like to be able to send keepalives as infrequently as
> possible. We have had some bad experiences with IMAP servers sending
> stuff frequently back to the client (presumably because not all
> clients are clever enough to do any keepalives themselves - this is
> a risk of course if we don't specify this at all in websocket spec),
> and the client has no way to affect their rate, which has been
> fixed.

No!  That's too 2010 and too application-specific.  What about when
the server is on your mobile phone and the client is in a data centre?
Think it won't happen?  What about that new Opera thingy where the
browser contains a web server for other browsers to use...

You can't assume the client side will always be the one with stricter
requirements, even in usual case of a user's web browser.  Servers
have resource limitations and need to manage idle connections.
1M+ connections to a single cluster is a fair implementation target;
then it matters.

> If there really is a requirement for the server to see keepalives at
> a certain minimum rate (for instance so that it can abandon stale
> connections), I would then have that negotiated so that the server
> can tell that back to the client who would still do the sending. But
> I'm not sure if that's needed.

This is not optional.

It is essential to abandon stale connections, otherwise servers run
out of resources over time - no matter how much memory they have.
Then they crash.

The only way to detect stale connections is send something or expect
to receive something.  TCP won't detect staleness if there's no activity.
(Useful word for blackholed connections, "stale", btw.  Thanks.)

Servers can send their own keepalive probes using the request-response
pattern, or by negotiated request to the client to send periodic
keepalives during idle periods, or by the packet-optimal synchronised
combination describe in an earlier mail from me.

All types have pros and cons, the same as the client side.

If both sides are sending request-response periodically, that's a high
overhead.  (Nb: In many applications, keepalives are the most expensive
bandwidth cost!)

Don't make the mistake of thinking a client-only regular ping is the
simplest keepalive to code...  It sounds simpler than it really is, when
you add what's needed for stale detection on both sides.

-- Jamie