Re: [hybi] Proposal for a clean way to detect non-HTTP compliant transparent proxies

[Willy Tarreau <[e-mail]>]

Hi Jamie,

first, thanks for taking the time to review the proposal in depth and
to match it with standards.

On Thu, Jul 22, 2010 at 01:38:01AM +0100, Jamie Lokier wrote:
> > That way, if a transparent proxy does not have explicit support for 101,
> > it will only forward the headers with an empty body (CL: 0) and close the
> > connection towards the client.
> 
> I think this won't work.  RFC2616 (HTTP 1.1):
> 
>    "Proxies MUST forward 1xx responses, unless the connection between the
>     proxy and its client has been closed, or unless the proxy itself
>     requested the generation of the 1xx response."
> 
> It's not spelled out explicitly, but the expectation is that proxies
> forward 1xx responses generically, and then forward the next response.
> 
> If the proxy doesn't know how to handle 101 responses, then it may
> simply forward the 101 as a generic 1xx, and then try to forward the
> *following* bytes as a HTTP response.

Yes but those will fail as the following bytes won't start with "HTTP/1.1",
and the proxy will close the connection.

> I would expect proxies to ignore Content-Length in 1xx:
> 
>    "1. Any response message which "MUST NOT" include a message-body (such
>        as the 1xx, 204, and 304 responses and any response to a HEAD
>        request) is always terminated by the first empty line after the
>        header fields, regardless of the entity-header fields present in
>        the message."

The content-length I proposed was precisely for those which are not
even aware of 1xx, nothing more.

> In other words, a relatively stupid proxy, but which is aware of the
> 1xx rule, will not close the connection, and will treat the hello
> text and subsequent bytes as a HTTP response to modify and forward.

Due to the next non-HTTP data, they must close with a "502 Bad Gateway"
upon receiving the first data following the 101 message that does not
look like "HTTP/1.1 XXX reason".

> There's some discussion on another list about using a 1xx response to
> indicate "request is being processed", for when the processing is slow
> to produce a response.

Nice idea, I was not aware of this discussion, I only initiated the one
about 101 not working like the general 1xx case ;-)

> The fact that is being taken seriously
> suggests that at least some deployments do forward 1xx messages in the
> way the standard requires.

Yes, at least the 100 are forwarded and heavily used with POST requests
("Expect: 100-continue").

But I agree with what you said in your other mail, the best way to know
is precisely to test !

As of now I have just tested via a Squid 2.6, Bluecoat proxy-sg v4 and
netcache (I don't remember the version), and all failed cleanly on the
101 by closing the connection. Apache 2.2.15 remains open after the 101
but immediately closes once it gets the first following byte. Also, none
of them sent the "Upgrade: WebSocket" in response, which is another good
reason to declare a failure ;-)

I'm not pretending the scheme covers every case (and I've not tested them),
but I think it addresses various generations of simplified implementations.

Regards,
Willy