Re: [hybi] Reliable message delivery (was Re: Technical feedback.)

[Jamie Lokier <[e-mail]>]

Maciej Stachowiak wrote:
> I'm curious about a couple of things:
> 

> 1) Do OS TCP stacks expose enough information to socket-level
> clients to determine what has definitely been sent when a TCP
> connection is closed?

In general, no they don't.  Linux does, but even then you have to poll
the statistics rather than waiting for an ack event.

There is a related issue, which is evident in Apache's
code and is a flaw in HTTP we'd do well not to repeat.
That is: It's not safe to close 


> 2) Is a TCP-level ack sufficient for this purpose? (Or would clients
> for reliable message delivery want a full end-to-end ack from the
> receiving application?)

No, and yes.  The reasons are numerous:

  - There are many kinds of TCP-level intermediaries
    which listen on a port, and for each incoming connection,
    create an outgoing connection elsewhere and simply relay
    octets both ways.  They do not interpret HTTP at all.

      + For example, SSH is sometimes used to establish the above
        type of proxy; in SSH it is called tunnelling.  There
        is other software which tunnels in this way.

      + Some routing/forwarding techniques, and some bandwidth
        management devices do it.

      + Ironically, sometimes it is used to avoid an intercepting HTTP
        proxy that is causing problems. :-)

      + Some mobile / wireless / satellite system relays do the
        same thing, because it allows different TCP algorithms to
        be used for the wireless link, which are better suited
        to the channel characteristics (e.g. high loss, fading).

  - Forwarding over HTTP proxies, for example using CONNECT on port
    443 is described in the WebSocket draft.

  - Forwarding over HTTP proxies that implement WebSocket
    detection and forwarding.  Not yet, but to be expected if
    it's deployed.

  - Forwarding by HTTP proxies that switch into "tunnelling mode"
    when they see something they cannot parse.  I am told these exist,
    because they are more reliable than strict proxies and it's needed
    due to a certain amount of badly non-compliant HTTP out there
    (e.g. header names containing spaces and quote marks).

    Some of these are intercepting "invisible"/"stealth" proxies and
    do not insert Via headers.  These proxies may end up forwarding
    WebSocket despite it's attempt to detect them.

  - Data that has been acknowledged by TCP ACK is not sent to the
    application under some circumstances:

      + The application calls close() or shutdown(SHUT_RD) between the
        data being sent and before it has read the data.  This
        window is unavoidable, because OSes generally don't allow
        applications to read any data that was buffered at that
        precise moment.

      + The network facing component crashes, is terminated,
        is taken offline etc.  For example, restarting cometd.
        Even though communication is nominally with another
        component *behind* the network facing one, which is still
        running.  This is just another kind of proxy.

-- Jamie