Re: [hybi] Proposal for a clean way to detect non-HTTP compliant transparent proxies

I don't like to reply to myself, but I'm just realizing that using a
POST instead of a GET as was already suggested in the past would most
likely match most of the already in-place cache-bypass rules in front
of transparent proxies, precisely because those proxies have no business
processing such POST requests. I have a feeling that doing just that
should already improve the success rate.  It may also unlock the
client-to-server direction on very low-level devices that would otherwise
fail, allowing them to work.

Willy

On Thu, Jul 22, 2010 at 02:58:45AM +0200, Willy Tarreau wrote:
> Hi Jamie,
> 
> first, thanks for taking the time to review the proposal in depth and
> to match it with standards.
> 
> On Thu, Jul 22, 2010 at 01:38:01AM +0100, Jamie Lokier wrote:
> > > That way, if a transparent proxy does not have explicit support for 101,
> > > it will only forward the headers with an empty body (CL: 0) and close the
> > > connection towards the client.
> > 
> > I think this won't work.  RFC2616 (HTTP 1.1):
> > 
> >    "Proxies MUST forward 1xx responses, unless the connection between the
> >     proxy and its client has been closed, or unless the proxy itself
> >     requested the generation of the 1xx response."
> > 
> > It's not spelled out explicitly, but the expectation is that proxies
> > forward 1xx responses generically, and then forward the next response.
> > 
> > If the proxy doesn't know how to handle 101 responses, then it may
> > simply forward the 101 as a generic 1xx, and then try to forward the
> > *following* bytes as a HTTP response.
> 
> Yes but those will fail as the following bytes won't start with "HTTP/1.1",
> and the proxy will close the connection.
> 
> > I would expect proxies to ignore Content-Length in 1xx:
> > 
> >    "1. Any response message which "MUST NOT" include a message-body (such
> >        as the 1xx, 204, and 304 responses and any response to a HEAD
> >        request) is always terminated by the first empty line after the
> >        header fields, regardless of the entity-header fields present in
> >        the message."
> 
> The content-length I proposed was precisely for those which are not
> even aware of 1xx, nothing more.
> 
> > In other words, a relatively stupid proxy, but which is aware of the
> > 1xx rule, will not close the connection, and will treat the hello
> > text and subsequent bytes as a HTTP response to modify and forward.
> 
> Due to the next non-HTTP data, they must close with a "502 Bad Gateway"
> upon receiving the first data following the 101 message that does not
> look like "HTTP/1.1 XXX reason".
> 
> > There's some discussion on another list about using a 1xx response to
> > indicate "request is being processed", for when the processing is slow
> > to produce a response.
> 
> Nice idea, I was not aware of this discussion, I only initiated the one
> about 101 not working like the general 1xx case ;-)
> 
> > The fact that is being taken seriously
> > suggests that at least some deployments do forward 1xx messages in the
> > way the standard requires.
> 
> Yes, at least the 100 are forwarded and heavily used with POST requests
> ("Expect: 100-continue").
> 
> But I agree with what you said in your other mail, the best way to know
> is precisely to test !
> 
> As of now I have just tested via a Squid 2.6, Bluecoat proxy-sg v4 and
> netcache (I don't remember the version), and all failed cleanly on the
> 101 by closing the connection. Apache 2.2.15 remains open after the 101
> but immediately closes once it gets the first following byte. Also, none
> of them sent the "Upgrade: WebSocket" in response, which is another good
> reason to declare a failure ;-)
> 
> I'm not pretending the scheme covers every case (and I've not tested them),
> but I think it addresses various generations of simplified implementations.
> 
> Regards,
> Willy
> 
> _______________________________________________
> hybi mailing list
> hybi@ietf.org
> https://www.ietf.org/mailman/listinfo/hybi