[hybi] HTTP and round trips

[Willy Tarreau <[e-mail]>]

I wanted to draw some diagrams about what typical architectures are
commonly found on client and server sides in order to help quickly
spot pathological handshakes. But this takes time. At least I have
drawn various examples showing how the HTTP handshake is affected by
round trips.r

Comments welcome.
Regards,
Willy

------

Here are various cases of round trips for the HTTP handshake depending
on components the requests passes through on the client side, at the ISP
and on the server side. Note that the model can also be applied to other
types of handshakes. The purpose of this enumeration is only to point
the sensible parts in the handshake and to see how they can be optimised
(if required).

The example considers that the client is on a network which is 100 ms
away from the server (round trip time) and that the client's ISP is
in the middle, at 50 ms of each. This is a rough estimate as many
parameters may vary (client-side upload time when the link is loaded,
ISP-to-server RTT for very remote sites, download time for network
accesses over satellite links, etc...).

For the example below, it is assumed that we're using the handshake as
defined in draft-76, but with the server-side ordered correctly to send
the 101 before reading the data.


1) client to server

          +---------+
          | client  |
          +----+----+
               |               Send: GET+key3
               /               Recv: 101+hash
          ( internet ) 100ms
               /
               |
          +----+----+
          | server  |
          +---------+

The handshake here takes one RTT = 100 ms.


2) client via proxy(-ies) to server

          +---------+
          | client  |
          +----+----+
               |               Send: CONNECT+GET+key3
               |   1ms         Recv: 200+101+data
               |
          +----+----+
          |  proxy  |
          +----+----+
               |               Send: GET+key3
               /               Recv: 101+hash
          ( internet ) 100ms
               /
               |
          +----+----+
          | server  |
          +---------+

The handshake here takes 1 + 100 = 101 ms


3) client to server, via server-side gateway(s)

          +---------+
          | client  |
          +----+----+
               |               Send: GET+key3
               /               Recv: 101+hash
          ( internet ) 100ms
               /
               |
          +----+----+
          | gateway |
          +----+----+
               |               Send: GET
               |   1ms         Recv: 101
               |               Send: key3
               |               Recv: hash
          +----+----+
          | server  |
          +---------+

The handshake here takes 100 + 2*1 = 102 ms


4) client to server, client-side proxy(-ies) and server-side gateway(s)

          +---------+
          | client  |
          +----+----+
               |               Send: CONNECT+GET+key3
               |   1ms         Recv: 200+101+data
               |
          +----+----+
          |  proxy  |
          +----+----+
               |               Send: GET+key3
               /               Recv: 101+hash
          ( internet )   100ms
               /
               |
          +----+----+
          | gateway |
          +----+----+
               |               Send: GET
               |   1ms         Recv: 101
               |               Send: key3
               |               Recv: hash
          +----+----+
          | server  |
          +---------+

The handshake here takes 1 + 100 + 2*1 = 103 ms


5) client via gateway + proxy(-ies) to server

          +---------+
          | client  |
          +----+----+
               |               Send: CONNECT+GET+key3
               |   1ms         Recv: 200+101+data
               |
          +----+----+
          | gateway |
          +----+----+
               |               Send: CONNECT
               |   1ms         Recv: 200
               |               Send: GET+key3
               |               Recv: 101+hash
          +----+----+
          |  proxy  |
          +----+----+
               |               Send: GET+key3
               /               Recv: 101+hash
          ( internet ) 100ms
               /
               |
          +----+----+
          | server  |
          +---------+

The handshake here takes 1 + 2*1 + 100 = 103 ms


6) client to server, via client-side transparent proxy

          +---------+
          | client  |
          +----+----+
               |               Send: GET+key3
               /               Recv: 101+hash
           (  ISP  )   50ms
               /
               |
          +----+----+
          | t.proxy |
          +----+----+
               |               Send: GET
               /               Recv: 101     
          ( internet ) 50ms    Send: key3
               /               Recv: hash
               |
          +----+----+
          | server  |
          +---------+

The handshake here takes 50 + 2*50 = 150 ms


7) Preliminary conclusions

Since the protocol supports proxies, the round-trip caused by the /key3/
parameter is only on the server side when gateways or transparent proxies
are installed before it. Proxies do not add a round trip because the
protocol makes use of a CONNECT method in order to establish a tunnel to
make them ignore the protocol, so that they can send the /key3/ parameter
with the first packet. Gateways installed before the proxy do not add any
significant time either due to being close to the proxy.

Gateways (reverse proxies) are mostly installed on the server side, closer
to the server, resulting in a negligible additional round-trip.

The really slow case is for transparent proxies installed at the ISP's,
because the client performs a normal handshake (not one for a proxy),
and this transparent proxy has to wait for the 101 to send key3 and get
the hash. So there are two round trips between the ISP and the server,
but still one between the client and the ISP.

What this means is that the round trips to take into account are the
ones between the first gateway which receives a normal non-proxy
handshake, and the server. That basically means that only the server-side
time counts. Even if the client browses over a satellite link, the extra
round-trips on the server side will not change much to him.


8) Possible improvements

The extra round trip here is caused by the first equipement to have to
wait for the 101 in order to forward the /key3/ parameter.

One efficient solution against this extra round-trip would be to put
this parameter into a HTTP header, just like the other Sec-WebSocket-Key*.
That way, there would not be any additional data to send from client to
server after the 101 appears, thus eliminating the extra round trip.

Note that this data is used for three things :
  - detection of cross-protocol communication
  - ensure that no cache will return a cached content
  - ensure that intermediaries correctly forward data after the 101

The first point and second points are not changed with a header. The
third point changes slightly. With the key in the data, it was able
to detect some of the proxies which would reset the connection during
the handshake. With the key3 in a header, it will only detect the
connection reset when trying to send data for the first time. However,
neither method detects intermediaries which don't forward request body
and remain stuck on it, so on this point there is no change.

Also, since the hash currently concatenates key1, key2 and key3, it
would also be even easier to simply remove key3 and have the client
check that the server emits the correct hash for key1+key2.

----