Re: Fwd: New Version Notification for draft-nottingham-httpbis-retry-01.txt

Willy Tarreau <> Thu, 02 February 2017 08:18 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id D561E129416 for <>; Thu, 2 Feb 2017 00:18:33 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -10.12
X-Spam-Status: No, score=-10.12 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-3.199, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id cWflstt4fSux for <>; Thu, 2 Feb 2017 00:18:31 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 4E94C120727 for <>; Thu, 2 Feb 2017 00:18:30 -0800 (PST)
Received: from lists by with local (Exim 4.80) (envelope-from <>) id 1cZCXs-0007AZ-HT for; Thu, 02 Feb 2017 08:15:08 +0000
Resent-Date: Thu, 02 Feb 2017 08:15:08 +0000
Resent-Message-Id: <>
Received: from ([]) by with esmtps (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from <>) id 1cZCXn-0005BM-6O for; Thu, 02 Feb 2017 08:15:03 +0000
Received: from ([] by with esmtp (Exim 4.84_2) (envelope-from <>) id 1cZCXg-0007A5-T6 for; Thu, 02 Feb 2017 08:14:57 +0000
Received: (from willy@localhost) by pcw.home.local (8.15.2/8.15.2/Submit) id v128EU2P020306; Thu, 2 Feb 2017 09:14:30 +0100
Date: Thu, 02 Feb 2017 09:14:30 +0100
From: Willy Tarreau <>
To: Mark Nottingham <>
Cc: HTTP Working Group <>
Message-ID: <>
Reply-To: HTTP Working Group <>
References: <> <>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <>
User-Agent: Mutt/1.6.1 (2016-04-27)
Received-SPF: pass client-ip=;;
X-W3C-Hub-Spam-Status: No, score=-7.0
X-W3C-Hub-Spam-Report: AWL=0.929, BAYES_00=-1.9, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, W3C_AA=-1, W3C_IRA=-1, W3C_IRR=-3, W3C_WL=-1
X-W3C-Scan-Sig: 1cZCXg-0007A5-T6 3c90237ae550f56ba7c96de2f4c016cf
Subject: Re: Fwd: New Version Notification for draft-nottingham-httpbis-retry-01.txt
Archived-At: <>
X-Mailing-List: <> archive/latest/33424
Precedence: list
List-Id: <>
List-Help: <>
List-Post: <>
List-Unsubscribe: <>

Hi Mark,

On Wed, Feb 01, 2017 at 07:26:29PM +1100, Mark Nottingham wrote:
> FYI; fairly minor update. Would love to hear what people think about the various suggested paths forward.

Quite an interesting document. The questions left open are even more
difficult for an intermediary because as you've mentionned, methods are
not sufficient to guess application's idempotency and everyone seems to
expect that transient network issues should be hidden from the end user.

I'm even wondering if applications should pass some information to the
client to indicate a guaranteed idempotency that could be indicated back
along the chain by a header. We could thus imagine to replay such POSTs
when the application uses some anti-replay request IDs. Regarding GET,
there's probably no point asking applications to pass such information
because changing them to do this is as difficult as expecting them to
use POST instead, so we should probably continue to consider that GETs
are idempotent.

I found something which can be slightly improved in 4.1 regarding the
detectable conditions for a retry. "Connection closes" and TCP RST
could be cut into to very distinct categories :
  - those which happen without ACKing the request
  - those which happen after ACKing the request

In the first case it's mostly a race on a idle timeout causing the server
(or an intermediary) to close at the same time the client sends the request.
It is safe to retry because it is guaranteed at the TCP layer that the data
were not consumed.

In the second case you don't know. The data may simply have been delivered
to TCP socket buffers (and ACKed) at the moment the application closed the
listening socket (eg during an application reload). Or the request may have
been delivered to the application, causing it to crash, explaining the close
or reset.

I'd argue that the first ones SHOULD be retried (with reasonable efforts)
while the second ones SHOULD NOT unless the request's idempotency has been
figured out.

One difficulty is that current TCP stacks don't make it easy to find if
data was ACKed and can be dropped. In fact the problem is not the moment
the reset/close event is detected, but the moment before. For an
intermediary it's not possible to buffer infinite amounts of data, so
it's needed to drop transmitted data to make buffer room for more data
(eg for a POST request). And data being ACKed doesn't translate into an
event that can suddenly cause a paused connection to be woken up
(SNDLOWAT 1) usually is ignored. For a client (browser) the issue is
the same except that all elements having lead to the request should
probably still be present to build a completely new request from scratch
so the network buffering issue should not exist.

So in the end we do have some reliable transport-level information to
detect the conditions for a safe retry of non-idempotent requests but
these information are not easy to pick and exploit. It's a particular
issue for TCP Fast Open implementation as a client (I have implemented
TCP Fast Open to servers in haproxy but not merged it yet since I have
not sorted this out yet).