Re: [Hls-interest] LL-HLS: status-code expected in the response to the PRELOAD request

["Law, Will" <[e-mail]>]

@Mark<mailto:[e-mail]> @Lucas – thank you both for your quick responses, both pointing at RFC8673 https://www.rfc-editor.org/rfc/rfc8673.html. This does address the exact problem. The solution is outlined below for those who want a primer. It revolves around the client signaling a magic number last-byte-pos which is “much larger than the last-byte-pos returned in response to an open-ended byte-range HEAD request, as described above, and much larger than the expected maximum size of the representation.” . This leads to the obvious problem of what threshold value should an origin or CDN use? A stateless edge server has no idea what is “the expected maximum size of the representation”.

All CDNs should use the same threshold value, for consistency of client responses. Additionally, all LL-HLS origins should also apply the same threshold, for the same reason. The RFC makes a recommendation to use 2^^53-1 (9007199254740991) as the threshold. It is Javascript safe and in fact corresponds to Number.MAX_SAFE_INTEGER. Akamai is implementing this as we speak, and our current intent is to use 9007199254740991. However some flexibility to align with the status quo and we are willing to align with other CDNs for industry consistency in support.

@Mark – what threshold value does Fastly use?

@Lucas – what threshold value does Cloudflare use?

@Other CDN implementers on this list – what value do you use?

Equally, all LL-HLS clients should use the same threshold value. For those building LL-HLS clients that support byte range addressing, any objections to using 9007199254740991?



----------------------------------------------------------------

A client can indicate support for handling indeterminate-length byte-range responses by providing a very large value for the last-byte-pos in the byte-range request. For example, a client can perform a byte-range GET request of the form:

GET /resource HTTP/1.1
Host: example.com
Range: bytes=1230000-999999999999

where the last-byte-pos in the request is much larger than the last- byte-pos returned in response to an open-ended byte-range HEAD request, as described above, and much larger than the expected maximum size of the representation. See Section 6 for range value considerations.

In response, a server may indicate that it is supplying a continuously aggregating/live response by supplying the client request's last-byte-pos in the Content-Range response header field.

For example:

GET /resource HTTP/1.1
Host: example.com
Range: bytes=1230000-999999999999

returns

HTTP/1.1 206 Partial Content
Content-Range: bytes 1230000-999999999999/*

from the server to indicate that the response will start at byte
1230000 and continue indefinitely to include all aggregated content,
as it becomes available.

Cheers
Will


From: Pieter-Jan Speelmans <[e-mail]>
Date: Monday, August 17, 2020 at 9:53 PM
To: "Law, Will" <[e-mail]>
Cc: "[e-mail]" <[e-mail]>
Subject: Re: [Hls-interest] LL-HLS: status-code expected in the response to the PRELOAD request

Hi all

Related to that:

To start-up, the client would issue 6 GET range-requests , starting at the independent part The server would respond with a 206 response for each and a content-range response header indicating the range being returned.
An optimisation we have considered from a client side would be to identify those 6 range requests would actually fold together in one larger range request (and potentially the request for the preload hint as well). Folding the first six requests together should be OK according to normal HTTP flows.

What would be the expected behaviour however if the preload-hint request also gets collapsed into this or simply the asset v1_1-7728.m4s would be requested without a range request at all?

Best regards,
Pieter-Jan


Pieter-Jan Speelmans
CTO

THEO Technologies NV
w | www.theoplayer..com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.theoplayer.com_&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=3V-L-7irWJaIKqd70_fh4S30y0kfh-uHlFMaZnMQJfo&s=zRH5_eqUYPpnZnmTuXadhCfqsPz4Kf1T4_9-KmoNRpk&e=>
Philipssite 5, Bus 1. 3001 Leuven, Belgium
[Image removed by sender.]<https://urldefense.proofpoint.com/v2/url?u=https-3A__d2oj23ymxjbl3l.cloudfront.net_&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=3V-L-7irWJaIKqd70_fh4S30y0kfh-uHlFMaZnMQJfo&s=Q6XrRZNHy3KpCquvR83tEmqOc-klGILP6gPYJegRx7s&e=>
Leuven (Belgium) - New York (US) - San Francisco (US) - Singapore (SG) - Barcelona (ES)
BTW: BE 0847.829.290, RPR: Leuven


On Tue, Aug 18, 2020 at 1:30 AM Law, Will <wilaw=[e-mail]<mailto:[e-mail]>> wrote:
Hello

I am requesting a clarification on the expected server response status code when a client makes an open-ended range request against a media segment when playing back LL-HLS.  Consider the following media playlist snippet:

#EXTINF:4..000,
v1_1-7727..m4s
#EXT-X-PROGRAM-DATE-TIME:2020-07-28T17:37:48.771Z
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="64267@0",INDEPENDENT=YES
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="63033@64267"
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="57810@127300"
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="60558@185110"
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="68575@245668"
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="60880@314243"
#EXT-X-PRELOAD-HINT:TYPE=PART,URI="v1_1-7728.m4s",BYTERANGE-START=375123

To start-up, the client would issue 6 GET range-requests , starting at the independent part The server would respond with a 206 response for each and a content-range response header indicating the range being returned.

The next request would be for the PRELOAD-HINT part. This would be a GET request with a “range: 375123-“. The client is indicating it wants to receive this object starting at offset 375123 and continuing to the end of the segment.

How should the origin (or proxy server) respond to this PRELOAD request? Three possible options

  1.  It holds back any response until the end of segment, returning at that time a 206 response with a content-range of 375123 – T/T+1 (where represents total size of segment). This would ruin the low latency behavior for the client.
  2.  It starts an immediate response, signaling 206 with content-range: 375123 - */*. This is actually forbidden by the RFC’s, which indicate that the last-byte-pos cannot hold a value of “*”.
  3.  It starts an immediate response, signaling a 200 response and if serving H1 to a proxy server which then serves H2 to the client, a "Transfer-Encoding: chunked" response header.

Option [3] looks to be the most correct however it raises the question of  whether a 200 response (chunked-transfer or not) implies to the client that is has received the complete object i.e starting at offset 0 instead of offset 375123. As a CDN, we need to build a behavior that is robustly supported by all HTTP clients and not just a particular class of application. The proxy-server cannot tell if is serving a LL-HLS client or some other client, therefore we need a consistent behavior when it is asked for an open-ended range request against an object of unknown size.

So the questions are:

  1.  Is a 200 status-code expected in the response to the PRELOAD request?
  2.  If so, are there other clients or applications on the internet that would break if we did so for all open-ended range requests against objects of unknown size (outside of the application space of LL-HLS)?

Cheers
Will



--
Hls-interest mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/hls-interest<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_hls-2Dinterest&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=3V-L-7irWJaIKqd70_fh4S30y0kfh-uHlFMaZnMQJfo&s=lAcAl0Wr7tRdhayCf3crO76tH1jgoJAQ8ant2PDXpCA&e=>

Legal Notice
You can find the latest terms and policies of THEO Technologies under www.theoplayer.com/terms<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.theoplayer.com_terms&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=3V-L-7irWJaIKqd70_fh4S30y0kfh-uHlFMaZnMQJfo&s=raivU9TwEr7KoP4i_LeGjwPIRM5Tn_6GWXUoLm5wiN4&e=> including our "GENERAL TERMS AND CONDITIONS". In the absence of a signed agreement between you and THEO Technologies, by replying to this email these terms apply to the relevant transaction between us.
The contents of this email message and any attachments are intended solely for the addressee(s) and contain CONFIDENTIAL and/or privileged information and are legally protected from disclosure. If you are not the intended recipient of this message, please immediately alert the sender by reply email and then delete this message including any attachments and you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.
Full security of emails over the internet cannot be ensured. Despite our efforts it is your responsibility to provide for your protection.
Global HQ: THEO Technologies NV, Philipssite 5 bus 1, 3001 Heverlee, Belgium - BE 0847.829.290 - CEO: Steven Tielemans