Re: what constitutes an "invalid" content-length

[Willy Tarreau <[e-mail]>]

On Wed, Jul 13, 2016 at 08:40:42AM -0400, Patrick McManus wrote:
> My biggest frustration in this space is actually around the unreliability
> of truncation detection. *lots* of non persistent h1 transactions that seem
> to be strongly framed come up short by some criterion - c-l, missing a
> zero-chunk terminator, or unclean close termiations like RST or TLS Alerts.

I actually second this observation. In haproxy we have to process socket
errors *after* processing the message so that we get a chance to pass a
complete message in case it ends with a reset. And since haproxy is most
often installed very close to the server, we cannot even blame some
random blackboxes, it's the server which does nasty stuff. I found that
one cause of such issues is when the max_orphans setting on the system
is low and applications immediately close() after the last send(), and
haproxy itself used to be caught doing this from time to time as well.
As a workaround for this specific case in haproxy, we do a shutdown(SHUT_WR)
before the close(), which manages to send the last FIN before trying to
turn the socket into an orphan, so that if there's no more orphans and
an RST has to be sent, at least a FIN was sent before.

But what it means is that quite a number of resets are actually caused
by this and also indicate an intentional close() by the application,
meaning that you likely got the last chunk... unless it was killed in
the TCP stack upon receipt of the reset of course.

Willy