Re: Issue with "bytes" Range Unit and live streaming

[Craig Pratt <[e-mail]>]

Barbara did a good job providing some needed context. But I want to respond to the "jump to live" aspect (below).

cp

On 4/14/16 6:20 PM, Remy Lebeau wrote:

AFAICT, the existing ABNF for byte ranges *does* already support what you are looking for, without having to introduce a new unit type:

    byte-range-spec = first-byte-pos "-" [ last-byte-pos ]

Simply omit the * and leave the second value empty:

    Range: bytes=1234567-

RFC 2616 Section 14.35.1 says:

"If the last-byte-pos value is absent, or if the value is greater than or equal to the current length of the entity-body, last-byte-pos is taken to be equal to one less than the current length of the entity- body in bytes."

RFC 7233 Section 2.1 says:

"If the last-byte-pos value is absent, or if the value is greater than or equal to the current length of the representation data, the byte range is interpreted as the remainder of the representation (i.e., the server replaces the value of last-byte-pos with a value that is one less than the current length of the selected representation)."

Since the media is being recorded while streaming, the client can randomly jump around within the recorded portion of the media all it wants. Once jumped, it would simply be receiving media from the recording, not the live feed.  To jump back to the live feed, simply omit the "Range" header.

Remy Lebeau
Lebeau Software

[cp] There are a couple of issues with the above usage model (which we definitely considered and discussed)

(1) The GET request (without a Range header) is typically expected to transfer the entire representation. In the kind of in-progress recording example, this means the transfer would start from the first recorded byte (and not the live point). While there might be some room for various interpretations, our use case is constrained by the DLNA specifications here (and implementations based on it). The spec and endpoints expect Range-less transfers to start at the beginning of the representation. And really, if there is ambiguity on the GET response, I'd rather not make assumptions one way or the other.

(2) Even if a server returned live data from a Range-less GET, you can end up with data discontinuity. For instance, say you want to jump ahead in the stream - through the random access portion and into the live data. Under this model, I might perform a Range request with "bytes=1234567-", process the data, and then jump to the live point once I'd consumed that response. Well, no matter how small I make that Range request, data could have been added between the time I did the "bytes=1234567-" request and I jump to the live point. Since the client can't know/detect this discontinuity, it has to be implemented assuming the worst - flushing the framebuffer, resetting the rendering state, and waiting for the next keyframe - which can be seconds away.

In fact, and in general, jumping to live on a video stream like this is best accomplished by performing a request that precedes the end of the stream. This is due to this dependance on a key frame to initialize rendering. Being able to capture the most recent key frame can make a difference of seconds in initial presentation of a live stream. But it definitely requires continuity - which is what the bytes-live range unit is intended to provide.

I hadn't made much of this "discontinuity" aspect in the RFC. Please let me know if you think it deserves more attention/explanation. And thanks much for the thoughts.

Happy Friday,

cp

On 4/14/2016 4:26 PM, Craig Pratt wrote:

Hello all,

We have a use case where we are "live streaming" media over HTTP. As with other (non-segmented) HTTP streaming applications, our live streaming solution has the HTTP client perform a GET on a URI representing the live audio or video content stream and with the server progressively returning data using chunked transfer coding. The data returned to the server is "live" so long as the client reads data as quickly as it's generated - with TCP flow control allowing the client's reading of the response data to be paced to the live chunk generation of the HTTP server.

More recently, use cases have appeared that require a combination of random access and live streaming. For example, an HTTP media server may want to support streaming of an in-progress recording - with the HTTP client using HTTP byte Range requests to access any point within the already-recorded content. But we've run into issues supporting both live content transfer and HTTP byte Range requests on the same resource. There's no good way that we've found to get to this "live point" and also support random access. The use case requires a byte range that ends at an indeterminate point, e.g.

    Range: bytes=1234567-*

However, the ABNF for the bytes Range Unit doesn't allow for this form of request.

Rather than coming up with a proprietary solution, we are proposing a new Range Unit called "bytes-live" as a potential solution to this issue. Support for this new "bytes-live" Range Unit could be exposed by servers (using the Accept-Ranges header) for content that support both byte-wise random access and live streaming. e.g. A server that supports both the standard HTTP/1.1 bytes Range Unit and bytes-live would include the header:

    Accept-Ranges: bytes bytes-live

An HTTP client can use Range requests with the standard "bytes" Range Unit to transfer stored content from a representation. And when the bytes-live Range Unit is supported, an HTTP client can request a byte range that starts or ends at the live point of the content representation. For example, a client could request the transfer of bytes starting at byte 1234567 and transitioning to the live point once all stored content has been transferred using:

    Range: bytes-live=1234567-*

We believe the bytes-live Range Unit could have multiple applications outside of our immediate use cases. For instance, internet-connected video cameras with on-board storage could use bytes-live to support streaming of continuously recording content with lower latencies than can be achieved using HLS, MPEG-DASH, or other segmented video streaming techniques. And there are also applications with on-demand transcoding.

The draft RFC can be found here https://tools.ietf.org/html/draft-pratt-httpbis-bytes-live-range-unit-00" rel="nofollow">https://tools.ietf.org/html/draft-pratt-httpbis-bytes-live-range-unit-00. Please let me know what you think - and if there are other solutions we haven't considered.

Regards,

craig

--

craig pratt Caspia Consulting [e-mail] 503.746.8008

--

craig pratt Caspia Consulting [e-mail] 503.746.8008