Re: Issue with "bytes" Range Unit and live streaming

[Craig Pratt <[e-mail]>]

Hey Roy,

On 4/20/16 5:36 PM, Roy T. Fielding wrote:
>> On Apr 20, 2016, at 2:08 PM, Craig Pratt <[e-mail]> wrote:
>>
>> Reply in-line.
>>
>> cp
>>
>> On 4/20/16 1:14 PM, Matthew Kerwin wrote:
>>>
>>> On 21/04/2016 5:37 AM, "Thorsten Lohmar" <[e-mail] <mailto:[e-mail]>> wrote:
>>>> Hi Craig,
>>>>
>>>>> What *is* missing is the ability to get a continuous byte range on live content
>>>>> that starts at an arbitrary offset and the ability to directly jump to the live
>>>>> point.
>>>> Yes, and there are two solutions to the issue:
>>>> A. Enable it on HTTP Layer through the definition of a new range request method
>>>> B. Enable working with existing HTTP procedures, i.e. client can workout the precise byte offsets.
>>> C. Use a different URL (e.g. a ?start= query parameter). It's less optimal for caching, but it works today without any new specs.
>>>
>> [cp] We can't use a different URI in our application (the UPnP CDS and DLNA RUIH guidelines only one URI per resource).
>>
>> [cp] We can define a custom header. But (a) it appears that the feature we're describing has been requested many times over, (b) at least having an RFC enables caches to be implemented. Using a custom header or URI will require caching to be disabled by the origin server.
>>
>> [cp] One point I haven't mentioned about caching with bytes-live: Since the bytes-live Range Unit is designed to work in concert with the bytes Range Unit (not superceded it), caching can still be performed by proxies that don't understand bytes-live. The read-through and hit ratio will be reduced. But there's still opportunities to cache.
> This doesn't make any sense.  The live stream URI (whatever it is) always identifies
> the current live stream head.  An HTTP cache cannot save that because each new request
> expects the latest live stream.  What it can do is parallel cache the live stream's
> representation under a different "past stream" URI (or an array of such) and include
> instructions on how to use that within the live stream metadata.  Regardless, the live
> stream HTTP response must be marked as non-cacheable to avoid filling up normal caches.
[cp] I think you need to go back and read the rest of the thread - or 
even just my introduction. This is not the use case described. But I'll 
describe it again another way - in the hopes of avoiding misunderstandings.

The URI in this case refers to a random-accessible representation which 
is periodically appended to - not a live-only stream. The representation 
supports the "bytes" Range Unit, so a client can access bytes 0-1000, 
40000-90000, etc. of the growing resource at its discretion. And at some 
point a client wants to be able to get to the live point - just 
receiving the newly-appended (live) data just like you're describing. 
And to be consistent with HTTP semantics (and enable cache hits btw), a 
Range-less GET will get you the beginning of the representation, not the 
end. So there's no good way to get to the live point currently (polling 
being "non-good").

Re: Representation caching

Whether a representation is considered cacheable in this use case is at 
the discretion of the origin server and specific to the use 
case/application - as it should be (imho). There's no *necessity* in 
having a periodically-appended resource marked non-cachable, correct? If 
the resource mutates, it's not cacheable. If it's just being appended 
to, it is cacheable. And if an appended resource stops being appended 
to, it doesn't invalidate the cached representation.

And with the high compression rates of audio/video streams, and the 
larger capacity of proxy caches, I think video resources are more 
cache-worthy than they have ever been (esp live ones that can put high 
demands on a proxy). But they're still free to implement policy based on 
mime type, expiration, etc. - again, as it should be.
>
> On balance, I think this is a poor use of HTTP.  The Apple live streaming protocol,
> which is just a dynamically updated representation of a queue by reference, is
> able to perform the same task without twisting backwards.  By indirectly referencing
> the data streams, the primary queue can remain non-cacheable (until the stream dies
> and is archived) without impacting (partial) requests on the referenced data streams.
>
> ....Roy
>
[cp] I'm going to assume this conclusion is in question since there's 
misunderstanding of the scenario. If not, I'll describe why both basic 
single-representation content and segmented content have their places. 
And please enlighten me if any of the proposed semantics are not 
consistent with the HTTP representation model.

Happy Wednesday,

cp

-- 

craig pratt

Caspia Consulting

[e-mail]

503.746.8008