Re: [hybi] New port and Tunneling?

["Shelby Moore" <[e-mail]>]

>
> Just running this explanation (since I know it by heart already); I'll get
> to other points in due time.
>
> On 17 aug 2010, at 08.44, Shelby Moore wrote:
>>> I should also note that TCP doesn't guarantee delivery.
>>
>>
>> Thanks for clarification.  What I meant is that TCP guarantees that we
>> know when something wasn't delivered.
>
> No, it does not. You cannot prove the lack of existence of an event; you
> can only prove the existence of it. In other words, TCP can, in terms of
> the TCP stack itself, realise that a given packet has been delivered. It
> cannot realise that a given packet has been lost (it just assumes it is.
> This is why some packets are delivered twice).


You have just stated that TCP knows when a packet is delivered, combined
with timeout, TCP knows when something MAY have not been delivered.  Thus
TCP is able to guarantee that a packet was delivered (and it knows when it
MAY not have been). That is very different situation from UDP, or in
comparison between a TCP channel without application layer proxies and the
case where I am worried about a proxy silently eating a packet that TCP
thinks it was delivered.


> However, the TCP stack does
> not generally expose this functionality to upper layers.


I am not well versed on this, but I think you are saying that it is
possible (even if not generally done) for upper layers to distinquish
between whether if a TCP packet was delivered or MAYBE not delivered?

In any case, even when the upper layers can not read each TCP packet
deliver status, I think you are clarifying issues which do not apply the
context of the point I made. The point I am making is that TCP is much
more reliable channel (in terms of delivery) than for example UDP. We have
certain expectations/experience about how much more reliable it typically
is, and that is why we use is for data integrity applications such as
HTTP. My point being that if we lose that reliability in some cases due to
silent-data-eating HTTP proxies, then we no longer have the TCP
reliability across the entirety of the end-to-end client and server.


> Namely, it's
> generally not possible for us as an application writer to ascertain that
> any given byte has been delivered. Since this means that even your garden
> variety TCP connection is not a stable transfer option, every application
> needs to assume that messages can and will be lost, and thus will have
> their own ACKs layered on top.


Understood. Does not apply to the context of my point above. Thank you for
sharing your expertise.


>>> This is exactly
>>> the reason why WebSockets needs orderly close as part of the protocol
>>> itself (because otherwise TCP may lose data).
>>
>> I do not understand, because that is not my area of expertise (and my
>> brain is tired and I am in a rush to send this and go do other things).
>> If
>> you are willing to explain, I am willing to read. Maybe most other
>> readers
>> understand what you mean.
>
>
> Simply put, a client wishes to shut down. It needs to send a final piece
> of data to the server, and then close() the WebSocket connection. The
> client writes the bytes to be sent to the TCP output buffer, but then
> what? In the regular case, we can't ascertain when this buffer is empty.
> If we close the socket while the buffer is non-empty, we risk losing the
> data in it, depending on the implementation.
>
> However, let's assume that the implementation will first send the bytes,
> wait for ACKs, then send the FIN to close the connection. Since TCP will
> ACK on /delivered/ and not /read/ bytes, this does not imply the remote
> application received them yet (it may be processing a different request).
> When the remote end receives the TCP FIN, it will close down the socket,
> and elicit EOF on it (because the socket is closed). The remote
> application can now not retrieve the data that was sent.
>
> For this reason, all HTTP servers leave TCP connections open for a certain
> timeout after a successful request has been answered, because otherwise
> the client might not get the entire page delivered. And for this reason
> WebSockets has an orderly close mechanism, to ensure that the data has
> been delivered before the connection is closed.
>
> The reason I bring this up is because when you state that proxies break
> the guarantee of TCP, I state they don't.


Refer to my explanation above.


> TCP never guarantees that data
> will be delivered. TCP guarantees the following:
>
> Each byte read from the remote endpoint's socket will correspond to the
> same byte that was written in the local endpoint's socket. The bytes that
> arrive will arrive in the same order, and with no data corruption.
>
> TCP also provides flow and congestion control, but these are minor in
> comparison.
>
> What you likely refer to when you believe TCP guarantees delivery is the
> TCP retransmission mechanic. While a very good mechanic for a live stream,
> it does not guarantee you'll know where a transmission fails, only that it
> has failed. It's up to the application layer protocol to discover this
> failure and handle it.


Refer to my explanation above.


> With that in mind, I hold that intermediaries can very well provide a
> similar level of service that plain TCP can. History has also shown that
> we're more likely to have intermediaries than not to, so I do not buy the
> argument that we're better off doing our best to
> tunnel/bypass/protect/disrupt them. Is it a compromise? Yes. But I believe
> it is a necessary compromise between those who wish to provide a new type
> of service and those who wish to manage their networks.


So if I understand you correctly, you think that any transport (port+low
level protocol) and high level protocol we use or create will end up
having intermediaries on it, and that it is futile to try to assume that
we won't?

I would not disagree with that (because I believe in free markets), I am
just arguing that the protocol we use should be well formed.  What worries
me so far about WS/HTTP is that using HTTP Upgrade to send bi-directional
data is not well formed with respect to the historic use of HTTP. It is
another form of tunneling through a legacy market installed base. There is
no doubt that specification should allowed it to be well formed, but I am
just worried about the silent failure. Whereas if we start with a new
thing, there won't be that legacy that we are bumping against. And I
worried that we could break somethings that already work in HTTP. Until
further information, I am leaning that I would rather keep it orthogonal.

And note that is not the only benefit I am trying to point out for this
proposed WS/P+T (port+tunnel).

Some of this can be used to argue for WS/TLS as well.


>
> Cheers,
> Vladimir