Re: [Moderator Action] Range Responses of Indeterminate Length: Draft

Amos Jeffries <> Tue, 13 October 2015 09:40 UTC

Return-Path: <>
Received: from localhost ( []) by (Postfix) with ESMTP id 917ED1A8776 for <>; Tue, 13 Oct 2015 02:40:18 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -6.912
X-Spam-Status: No, score=-6.912 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 3m3MU4ZDRrQi for <>; Tue, 13 Oct 2015 02:40:15 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 1FC0F1A7020 for <>; Tue, 13 Oct 2015 02:40:15 -0700 (PDT)
Received: from lists by with local (Exim 4.80) (envelope-from <>) id 1Zlw1L-0007Rj-8m for; Tue, 13 Oct 2015 09:37:23 +0000
Resent-Date: Tue, 13 Oct 2015 09:37:23 +0000
Resent-Message-Id: <>
Received: from ([]) by with esmtps (TLS1.2:DHE_RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from <>) id 1Zlw1I-0007Qz-MI for; Tue, 13 Oct 2015 09:37:20 +0000
Received: from ([] by with esmtp (Exim 4.80) (envelope-from <>) id 1Zlw1G-0006Ba-W0 for; Tue, 13 Oct 2015 09:37:20 +0000
Received: from [] (unknown []) by (Postfix) with ESMTP id 68AF1E6DA0 for <>; Tue, 13 Oct 2015 22:36:47 +1300 (NZDT)
References: <> <> <> <>
From: Amos Jeffries <>
X-Enigmail-Draft-Status: N1110
Message-ID: <>
Date: Tue, 13 Oct 2015 22:36:12 +1300
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Received-SPF: pass client-ip=;;
X-W3C-Hub-Spam-Status: No, score=-4.8
X-W3C-Hub-Spam-Report: AWL=-0.153, BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, TVD_RCVD_IP=0.001, W3C_AA=-1, W3C_WL=-1
X-W3C-Scan-Sig: 1Zlw1G-0006Ba-W0 d1cb4eb655f93e02097109675fb40174
Subject: Re: [Moderator Action] Range Responses of Indeterminate Length: Draft
Archived-At: <>
X-Mailing-List: <> archive/latest/30360
Precedence: list
List-Id: <>
List-Help: <>
List-Post: <>
List-Unsubscribe: <>

On 7/04/2015 11:39 p.m., Rodger Combs wrote:
> Yup; we use that for some platforms, but not all devices support it,
> and it limits us to the codecs supported by MPEG-TS (or that the
> platform supports in it). We use MPEG-DASH on some platforms, but it
> has similar issues. Both also require either knowledge of the segment
> durations beforehand (here, before they're generated), or the ability
> to force them to be consistent (which is impractical if remuxing
> streams a user's existing video files), though HLS implementations
> tend to be a fair bit more forgiving on this point than DASH ones.
> When it's possible, we've found that a single live-generated MKV file
> streamed over chunked HTTP is both compatible with a very wide
> variety of codecs, and very flexible about the sizes of a stream's
> GoPs and the like.
> One thing I've had success in some experiments with lately is writing
> the MKV header to a single file, writing individual segments to
> separate files, and keeping track of their start timestamps. Then,
> when a request comes in for time T, I can return the header
> concatenated to the segment with the greatest start time <= T, and
> all segments after it. This approach is somewhat similar to how
> MPEG-DASH works, but with the work of preparing a fully-valid MKV
> file handled by the server instead of the client. The downside is
> that implementations often refuse to make any kind of seek in a
> resource that doesn't support range requests (and you can't properly
> support them when the content length is unknown), so I end up limited
> to seeks to the nearest on-server chunk boundary.

Please do not do that to generate payload streams without some
indicators in the headers or request-URL to distinguish the different
responses. It screws cacheability both in proxies and browsers which are
unaware of how the body is generated.

YouTube partial .swf responses used to do this when the user selected a
custom offset to start playing. There were a lot of problems and
complaints about truncated videos and even wrong codecs being delivered
to users, where one user selecting a late start would make other users
receive truncated videos for a while.

Nasty as they are multipart payloads would be better for HTTP as a whole
to deliver this type of service.

Or dare say it; chunked extensions would have been a perfect solution to
present a kind of multipart payload where the chunk extension headers
are teh multipart prefix. But that ship has sailed for 1.1. We might be
able to get something defined with HTTP/2 extensions or perhapse
PUSH_PROMISE that serves the use case nicer.