Re: [hybi] Reliable message delivery (was Re: Technical feedback.)

[Maciej Stachowiak <[e-mail]>]

On Jan 29, 2010, at 11:33 PM, Justin Erenkrantz wrote:

> On Fri, Jan 29, 2010 at 11:25 PM, Maciej Stachowiak <[e-mail]> wrote:
>>> It depends upon what level of "reliability" you are looking for.  If
>>> you are aiming for the "common" case, the answer to both is "yes".
>>> 
>>> However, edge cases make the answer "no" - it is quite possible to
>>> have "lost" responses that a server actually sends, but the client
>>> will never see.
>> 
>> So you could lose messages, but can you tell in this case that you are not guaranteed yet that they have been delivered?
> 
> No, not really - the client simply thinks the server close()'d the
> connection but it has no way of knowing there were other data packets
> that the server really meant for the client to see.  Correspondingly,
> the server did everything in the right order - it wrote all the data
> it expected and then it close()'d the socket.  Yet...oops.

Presumably the server could know that at least all the packets ACK'd at the TCP level have been successfully delivered, right? So I assume the only problem is the remaining packets after that, if you don't do a lingering close.

In this case things are a bit more complicated because either the client or the server could be transmitting at any time, and either could choose to close the connection at any time, and either side may want to know if some of its messages are not guaranteed to be delivered.

> 
> I don't know how much code Jetty has to deal with lingering close, but
> httpd has an embarrassingly large amount of code to deal with this
> situation.

I would like to understand the lingering close issue better. Does it consist of waiting for TCP ACKs for all your packets before closing the TCP connection?

> 
>> It seems like a WebSocket-level close handshake would only solve part of the problem - you also need to be able to deal with an interruption of service that prematurely breaks the connection, and ideally you would have some better guarantee in that case than just assuming all messages are lost. Does that also need provision at the protocol layer, or could it just piggyback on TCP-level acks?
> 
> Like Greg, I think orderly close is about as good as you can do.
> Interruption of service is always going to be a possibility (power
> failure, router outages, etc.) - at least if orderly close is
> explicitly part of the protocol, then if it doesn't happen, then the
> client knows something went awry and then it can deal with it as best
> as it can.  Currently, in HTTP, you can't tell the difference between
> an orderly close and a "oh, no, something bad happened".  I think
> providing that type of hint would be a big step-forward - especially
> when async messages are involved.  -- justin

I think you can do better than just orderly close. Either from TCP-level acks or from WebSocket-protocol-level acks, you could tell that some number of your messages have definitely been delivered, even in the face of a service interruption. Right?

Maybe I'm thinking of reliable message delivery differently than you, but I assumed a major goal would be to know what might need to be retransmitted even if there is an unexpected disconnect.

Regards,
Maciej