Re: Question on HTTP 408

[Sébastien Barnoud <[e-mail]>]

Hi,

I have several years ago used customized Curl and GSoap to automaticaly retry requests on such a condition. 
As this is used for money transfer, it is very important to distinguish retryable errors (connect timeout, ...) from others errors (request timeout, ...).
On a close, no retry is possible except if the server replies with this 408.
So i agree with you, and i know that some application uses this feature. 

> Le 4 juin 2014 à 14:49, Willy Tarreau <[e-mail]> a écrit :
> 
> Hi Michael,
> 
>> On Wed, Jun 04, 2014 at 07:52:27AM -0400, Michael Sweet wrote:
>> Willy,
>> 
>>> On Jun 4, 2014, at 12:42 AM, Willy Tarreau <[e-mail]> wrote:
>>> Hi Michael,
>>> 
>>>> On Tue, Jun 03, 2014 at 07:54:18AM -0400, Michael Sweet wrote:
>>>> William,
>>>> 
>>>> This sounds like a pretty obvious bug - a HTTP server only responds when it
>>>> has received something.  The usual keep-alive timeout of connections is
>>>> silent (server just closes the connection, with a preceding TLS shutdown for
>>>> HTTPS).
>>> 
>>> Please could you check my other response to Roy ? In short I do instead
>>> think that responding 408 is the only way to gracefully shut down *and*
>>> inform the client it can safely retry and I really think it was designed
>>> for this exact purpose.
>> 
>> I don't think it was designed for being returned when no request had started
>> coming in, but rather when a partial request was received and then timed out.
> 
> It's hard to navigate through the W3C archives to find the original discussion
> for this status code introduced in HTTP/1.1. Some minutes talk about request
> timeout :
> 
>  http://lists.w3.org/Archives/Public/ietf-http-wg-old/1994SepDec/0257.html
> 
> On the WG, 408 was already mentionned in a few exchanges here :
> 
>  http://lists.w3.org/Archives/Public/ietf-http-wg-old/1995MayAug/0340.html
>  http://lists.w3.org/Archives/Public/ietf-http-wg-old/1995SepDec/0158.html
> 
> 
> The first mention of it I could find in a published draft was in draft-00
> dated 1995/11/22 :
> 
>    http://www.w3.org/Protocols/HTTP/1.1/draft-ietf-http-v11-spec-00.txt
> 
> And the text says :
> 
>  "
>   408 Request Timeout
> 
>   The client did not produce a request within the time that the 
>   server was prepared to wait. The client may repeat the request 
>   without modifications at any later time.
>  "
> 
> At least I'm not seeing any mention about starting to send anything, there
> is a client and it did not produce a request, that's how I've always read
> it. Also, please note this specific point in -p2 :
> 
>   http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-26#section-6.5.7
>   6.5.7. 408 Request Timeout
>   The 408 (Request Timeout) status code indicates that the server did
>   not receive a complete request message within the time that it was
>   prepared to wait.  A server SHOULD send the close connection option
>   (Section 6.1 of [Part1]) in the response, since 408 implies that the
>   server has decided to close the connection rather than continue
>   waiting.  If the client has an outstanding request in transit, the
>   client MAY repeat that request on a new connection.
> 
> Note the last sentence : "*If* the client has an outstanding request ...".
> For me that clearly means that the client might very well receive a 408
> while it did not have an outstanding request, which implies that no byte
> was sent over the wire yet.
> 
>> There is nothing in any of the HTTP RFCs that talks about returning a
>> response line, headers, and message body to the client without first
>> receiving something from the client.
> 
> Of course, but the same is true for the reverse proposition, nothing
> says it's not possible. And in fact, 400 bad request is a perfect
> example. If the client sends unparsable crap (so nothing matching
> even the beginning of a valid request), a 400 bad req will be
> returned. We're talking about protocol errors here leading to a
> connection being closed as a result, I'm not seeing anything wrong
> with that.
> 
>> What CUPS does (and this was largely based on asking the question, "What does
>> Apache HTTP Server do?" many years ago) is to silently close the connection
>> after the configured timeout, which is large enough (300 seconds for CUPS and
>> Apache) to account for any reasonable traffic delays (on the Earth, at
>> least).
> 
> But as I said in the other thread, the spec prevents the client from retrying
> a non-idempotent request if the server does this, while 408 explicitly allows
> it to. So I think that the intent behind the 408 was pretty clearly to inform
> the client that no damage was made and that it had a chance to retry. On silent
> close, you must not resend a POST for example, since it might very well have
> already been processed :
> 
>   http://tools.ietf.org/html/draft-ietf-httpbis-p1-messaging-26#section-6.3.1
>   6.3.1. Retrying Requests
>   Connections can be closed at any time, with or without intention.
>   Implementations ought to anticipate the need to recover from
>   asynchronous close events.
> 
>   When an inbound connection is closed prematurely, a client MAY open a
>   new connection and automatically retransmit an aborted sequence of
>   requests if all of those requests have idempotent methods (Section
>   4.2.2 of [Part2]).  A proxy MUST NOT automatically retry non-
>   idempotent requests.
> 
>   A user agent MUST NOT automatically retry a request with a non-
>   idempotent method unless it has some means to know that the request
>   semantics are actually idempotent, regardless of the method, or some
>   means to detect that the original request was never applied.
> 
>> I can understand if you don't want to have as large of a timeout for a proxy
>> that is shared by thousands of clients, but you can definitely make this
>> configurable and default to something reasonable (30-60 seconds).  If the
>> network connectivity is so bad that the request line and headers don't start
>> flowing within that timeframe, then there is probably no point in retrying
>> automatically anyways.
> 
> I'm seeing some sites running with smaller timeouts to limit the risk of DoS.
> When you know that you can accept 300k connections per second, with a 5s
> timeout you already have 1.5 million connections. That's not unreasonable
> to use aggressive timeouts in such conditions (especially for sites which
> run older pre-forked servers which cannot accept such large numbers).
> 
> And you can clearly have a few seconds pause on 3G during a handover. I've
> observed multi-second delays which cumulate with TCP retransmits to result
> in something in the 10s range. The handover is an interesting case because
> the network condition is temporarily bad and is good again once you get the
> response, so there's a real incentive to retry instead of telling the user
> "I'm sorry, I tried to validate your order but something has cut in the
> middle so I don't know if you'll get it or not, please check your cart".
> 
> In the case of haproxy, sending the 408 only if something was started on
> the connection is a trivial change, and I even delayed its final release
> by a few days so that we have the time to discuss this use case and decide
> what the proper cut-off mode is. But I think it would be a wrong change
> that degrades the client's ability to seamlessly recover. For me the most
> likely explanation for a close is "the server crashed after receiving the
> request" (FIN) or "before receiving the request" (RST). 408 makes it pretty
> clear that it's neither.
> 
> What would be interesting in fact is to know how various clients act on
> silent close an on 408, with idempotent and non-idempotent requests. And
> based on that we should likely clarify the text on what should be done.
> Pat said that Firefox handles 408 fairly well (I found a bug report for
> it being fixed in 2004). I don't know however how a POST over a silently
> closed connection is handled there. Maybe non-browser clients like Curl
> would help us determine the best behaviour as well. Daniel ?
> 
> Regards,
> Willy
> 
>