Re: [Moderator Action] Range Responses of Indeterminate Length: Draft

Rodger Combs <> Tue, 07 April 2015 12:33 UTC

Return-Path: <>
Received: from localhost ( []) by (Postfix) with ESMTP id 639631A87A0 for <>; Tue, 7 Apr 2015 05:33:50 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -6.912
X-Spam-Status: No, score=-6.912 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id wUSc9nEVuoEP for <>; Tue, 7 Apr 2015 05:33:45 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 81AA11A879D for <>; Tue, 7 Apr 2015 05:33:45 -0700 (PDT)
Received: from lists by with local (Exim 4.80) (envelope-from <>) id 1YfSdD-0006pv-4a for; Tue, 07 Apr 2015 12:29:27 +0000
Resent-Message-Id: <>
Received: from ([]) by with esmtp (Exim 4.80) (envelope-from <>) id 1YfSd9-0006ox-2O for; Tue, 07 Apr 2015 12:29:23 +0000
Received: from ([]) by with esmtps (TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from <>) id 1YfSd7-0006rE-Ej for; Tue, 07 Apr 2015 12:29:22 +0000
Received: from ([] helo=[]) by with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.80) (envelope-from <>) id 1YfSd6-0000OZ-Px for; Tue, 07 Apr 2015 12:29:21 +0000
Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\))
Content-Type: text/plain; charset="utf-8"
To: Michael Sweet <>
From: Rodger Combs <>
In-Reply-To: <>
Resent-From: Yves Lafon <>
Date: Tue, 07 Apr 2015 11:39:48 +0000
Cc: Mark Nottingham <>, HTTP Working Group <>
Content-Transfer-Encoding: quoted-printable
Resent-Date: Tue, 07 Apr 2015 14:29:21 +0200
Message-Id: <>
X-Name-Md5: efe3dad792d606410c9cc49cedaffc94
References: <> <> <>
X-Mailer: Apple Mail (2.2070.6)
X-W3C-Hub-Spam-Status: No, score=-0.0
X-W3C-Hub-Spam-Report: ALL_TRUSTED=-1, AWL=0.001, T_RP_MATCHES_RCVD=-0.01, W3C_NW=1
X-W3C-Scan-Sig: 1YfSd7-0006rE-Ej 047911e0fa50a96655f7ce320741eb85
Subject: Re: [Moderator Action] Range Responses of Indeterminate Length: Draft
Archived-At: <>
X-Mailing-List: <> archive/latest/29287
Precedence: list
List-Id: <>
List-Help: <>
List-Post: <>
List-Unsubscribe: <>

Yup; we use that for some platforms, but not all devices support it, and it limits us to the codecs supported by MPEG-TS (or that the platform supports in it). We use MPEG-DASH on some platforms, but it has similar issues. Both also require either knowledge of the segment durations beforehand (here, before they're generated), or the ability to force them to be consistent (which is impractical if remuxing streams a user's existing video files), though HLS implementations tend to be a fair bit more forgiving on this point than DASH ones. When it's possible, we've found that a single live-generated MKV file streamed over chunked HTTP is both compatible with a very wide variety of codecs, and very flexible about the sizes of a stream's GoPs and the like.

One thing I've had success in some experiments with lately is writing the MKV header to a single file, writing individual segments to separate files, and keeping track of their start timestamps. Then, when a request comes in for time T, I can return the header concatenated to the segment with the greatest start time <= T, and all segments after it. This approach is somewhat similar to how MPEG-DASH works, but with the work of preparing a fully-valid MKV file handled by the server instead of the client. The downside is that implementations often refuse to make any kind of seek in a resource that doesn't support range requests (and you can't properly support them when the content length is unknown), so I end up limited to seeks to the nearest on-server chunk boundary.

Sorry for going off-topic, but I hope this provides some insight into the complex issues involved in my use-case.

(Oh, and yes, you can use HLS without foreknowledge of segment durations for live streams, but not if you intend to present the media as a fully-seekable VOD stream... which we like doing when possible, as it means less work for the client to handle when seeking)

> On Apr 7, 2015, at 06:14, Michael Sweet <> wrote:
> Have you looked at:
>> On Apr 7, 2015, at 1:24 AM, Mark Nottingham <> wrote:
>> Hi Roger,
>> There are a bunch of folks talking about how to do streaming (especially video) over HTTP, and byte ranges often come up as a mechanism. This reminds me of a discussion we briefly had at the end of the Dallas meeting, where some folks wanted to use byte ranges for sparse content in MPEG DASH.
>> I think the concerns I had there apply here too; by overloading/changing the partial content mechanism, you’re risking interoperability problems with deployed infrastructure — especially proxy/caches, which can and do cache partial responses, generate partial requests, etc.
>> For example, if there’s a caching proxy in the middle that doesn’t understand Accept-Indefinite-Ranges, it’ll pass it through unmodified, and the server will then potentially generate 206 responses that are malformed.
>> This and many similar discussions I’ve had recently makes me wonder whether it’d be interesting to define a genuinely new partial content mechanism that’s tailored for media streaming; while it would depend on intermediaries rolling out support for the new mechanism to get any benefit from them, at least we could design it so that it doesn’t interact badly with deployed intermediaries.
>> What do people (especially intermediary implementers) think?
>> Cheers,
>>> On 6 Apr 2015, at 10:17 pm, Rodger Combs <> wrote:
>>> Hello!
>>> I've written up a draft standard for an HTTP extension that allows range requests to work more effectively with resources of indeterminate length. I've submitted it as an Internet-Draft, and wondered if anyone had any suggestions as to how to improve it, or move it towards standardization?
>>> Thanks,
>>> --Rodger
>> --
>> Mark Nottingham
> _________________________________________________________
> Michael Sweet, Senior Printing System Engineer, PWG Chair