"Timeout" request header to tell server to wait for resource to become available

Brendan Long <self@brendanlong.com> Sat, 28 March 2015 00:35 UTC

Return-Path: <ietf-http-wg-request+bounce-httpbisa-archive-bis2juki=lists.ie@listhub.w3.org>
X-Original-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Delivered-To: ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com []) by ietfa.amsl.com (Postfix) with ESMTP id 0E7AA1A079D for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Fri, 27 Mar 2015 17:35:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -6.288
X-Spam-Status: No, score=-6.288 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, MIME_HTML_ONLY=0.723, RCVD_IN_DNSWL_HI=-5, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id yfuBm2Iekl9c for <ietfarch-httpbisa-archive-bis2Juki@ietfa.amsl.com>; Fri, 27 Mar 2015 17:35:45 -0700 (PDT)
Received: from frink.w3.org (frink.w3.org []) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 456C61A0277 for <httpbisa-archive-bis2Juki@lists.ietf.org>; Fri, 27 Mar 2015 17:35:45 -0700 (PDT)
Received: from lists by frink.w3.org with local (Exim 4.80) (envelope-from <ietf-http-wg-request@listhub.w3.org>) id 1YbefG-0003FS-I1 for ietf-http-wg-dist@listhub.w3.org; Sat, 28 Mar 2015 00:31:50 +0000
Resent-Date: Sat, 28 Mar 2015 00:31:50 +0000
Resent-Message-Id: <E1YbefG-0003FS-I1@frink.w3.org>
Received: from maggie.w3.org ([]) by frink.w3.org with esmtp (Exim 4.80) (envelope-from <self@brendanlong.com>) id 1Ybef6-0003Ek-Kz for ietf-http-wg@listhub.w3.org; Sat, 28 Mar 2015 00:31:40 +0000
Received: from li799-54.members.linode.com ([] helo=brendanlong.com) by maggie.w3.org with esmtp (Exim 4.72) (envelope-from <self@brendanlong.com>) id 1Ybef5-00069M-Ru for ietf-http-wg@w3.org; Sat, 28 Mar 2015 00:31:40 +0000
Received: by brendanlong.com (Postfix, from userid 1002) id 0D18850037; Fri, 27 Mar 2015 19:31:18 -0500 (CDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=brendanlong.com; s=default; t=1427502678; bh=6YTtXk29Rmz2bA0l5UXBfWyVDymEhwogDbu5neFdtmg=; h=Date:From:To:Subject; b=hlQAjPFDro3tCMdrJelpk1RQP2AqMKPD8hxYUoog28Su/MwosRvZbYxX1X30KxEgH CpXOW+RvhTn/8AgqIZmC5E5V3wuC+CLahH3/rPneLxnjT4Tj4y7QqI96cJ0vnXbKnE /j1v9cM9OJqqv8o1PxPwqDECru1yPBhtoZ4vgQuQ=
Received: from [] (96-42-255-30.dhcp.roch.mn.charter.com []) by brendanlong.com (Postfix) with ESMTPSA id 8F0B350036 for <ietf-http-wg@w3.org>; Fri, 27 Mar 2015 19:31:16 -0500 (CDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=brendanlong.com; s=default; t=1427502676; bh=6YTtXk29Rmz2bA0l5UXBfWyVDymEhwogDbu5neFdtmg=; h=Date:From:To:Subject; b=H4H89e1WGxtSB3chnBSZZMuzlsC0pL4HI17r8RMfWwHUoGdn9z0VgfDTI41xC0JcW 24HR59IZnXzdbbiX4GT/NyrNtQCcijb431zHQ+63MQoH0ICq4Js5+R9WKF/1oEbyRV bXHcsjOwFzXowYtM0DXia3RO9Ocx1YzO/naj34xw=
Message-ID: <5515F653.6080506@brendanlong.com>
Date: Fri, 27 Mar 2015 19:31:15 -0500
From: Brendan Long <self@brendanlong.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0
MIME-Version: 1.0
To: HTTP Working Group <ietf-http-wg@w3.org>
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: 7bit
Received-SPF: pass client-ip=; envelope-from=self@brendanlong.com; helo=brendanlong.com
X-W3C-Hub-Spam-Status: No, score=-4.0
X-W3C-Hub-Spam-Report: AWL=-1.999, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, MIME_HTML_ONLY=1.105, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, W3C_AA=-1, W3C_DB=-1, W3C_WL=-1
X-W3C-Scan-Sig: maggie.w3.org 1Ybef5-00069M-Ru ebd6aed24cb9425e140f1cfb9f4ae030
X-Original-To: ietf-http-wg@w3.org
Subject: "Timeout" request header to tell server to wait for resource to become available
Archived-At: <http://www.w3.org/mid/5515F653.6080506@brendanlong.com>
Resent-From: ietf-http-wg@w3.org
X-Mailing-List: <ietf-http-wg@w3.org> archive/latest/29041
X-Loop: ietf-http-wg@w3.org
Resent-Sender: ietf-http-wg-request@w3.org
Precedence: list
List-Id: <ietf-http-wg.w3.org>
List-Help: <http://www.w3.org/Mail/>
List-Post: <mailto:ietf-http-wg@w3.org>
List-Unsubscribe: <mailto:ietf-http-wg-request@w3.org?subject=unsubscribe>

Resending this as an unsigned email since someone had trouble reading it.


I'm hoping this is the right group to address this to. Please let me know if I should take it somewhere else, or if I should do an independent submission instead. The use-cases I have in mind are MPEG-DASH and HLS, but I feel like this header isn't really specific to either of them, so an RFC would be a good place to define it.


When live streaming using MPEG-DASH, clients need to poll the server for manifest changes and new media segments. Due to fluctuations in encoding time and network latency, the client can't know exactly when a segment will be available, so it tries to request the new resource a short time after should become available (added latency). If the client requests the new resource too soon, it will get a 404 response and need to retry (introducing an RTT of latency).

It would be nice if we could just request a new media segment or updated MPD and have the server send it when it's ready.


Add a header telling the server to wait for a resource to become available before giving up. I chose to name this header "Timeout" with a value as a timeout in milliseconds. The actual timeout would be the value of the smaller of the value of the timeout header or the server's own configured timeout. If the client also sends an "If-None-Match" header, the server should also calculate etags (however it chooses to do so) and wait to send to a 200 or 304 response until the etag of the resource changes.

Header Examples

GET /rep1/segment-1.mp4 HTTP/1.0
Timeout: 5000
Meaning: Give me /rep1/segment-1.mp4 as soon as you see it, but you can give up after 5000 ms.

With ETag:

GET /manifest.mpd HTTP/1.0
Timeout: 5000
If-None-Match: W/"5f0-1193253510"

Meaning: Give me /manifest.mpd as soon as you see it and the etag doesn't match W/"5f0-1193253510", but you can give up after 5000 ms.

Software Examples

I wrote some Express middleware to implement this behavior here:

https://github.com/brendanlong/express-timeout-header" rel="nofollow">https://github.com/brendanlong/express-timeout-header

And an example demonstrating the usage with media segments MPEG-DASH in Google Chrome:

https://github.com/brendanlong/dash-http-timeout" rel="nofollow">https://github.com/brendanlong/dash-http-timeout
https://www.youtube.com/watch?v=YUcfNzPaqf0" rel="nofollow">https://www.youtube.com/watch?v=YUcfNzPaqf0


Does this seem like a good idea? Are there other use-cases? Should I write some sort of RFC?

PS -

As I write this, I'm starting to wonder if we just need headers like "Wait-Until: available" or "Wait-Until: etag-changes", since you can always close the HTTP connection, but the timeout could be useful for HTTP/1.1 pipelining. Either way works for me though.

See here if that's your preference:

https://github.com/brendanlong/express-timeout-header/tree/wait-until" rel="nofollow">https://github.com/brendanlong/express-timeout-header/tree/wait-until