Re: Issue with "bytes" Range Unit and live streaming

[Craig Pratt <[e-mail]>]

On 5/11/16 7:55 PM, Adrien de Croy wrote:

------ Original Message ------
From: "Craig Pratt" <[e-mail]>
To: "[e-mail]" <[e-mail]>
Sent: 12/05/2016 2:01:39 p.m.
Subject: Re: Issue with "bytes" Range Unit and live streaming

Reply in-line.

cp

On 5/11/16 6:16 PM, Adrien de Croy wrote:

unless we allow Content-Length to also use the new unit (which I'd be dead against), I'd suggest we steer clear of minting any new range units.

[cp] I don't see how a Range Unit would apply to Content-Length.

yes. That's the problem.

And that's why suggestions for alternative range units keep cropping up.

If you're an intermediate that has no knowledge of pages or frames, it's impossible to comply with any other range than bytes.

It's not hard to ask for

GET something?page=1

Which is cachable, vs making life impossible for intermediaries.

Adrien

[cp] This is not a good solution for large representations.

There's no way for a cache to know that "GET something?pages=1-10" has some of the same content that "GET something?pages=2-11" since it doesn't (can't) have any understanding of the URI params. So you'll end up with two copies of pages 2-10 in the cache.

And at the same time, the cache can't service a request of the form "GET something?page=3" for the same reason - even though you would have 2 copies of page 3 in the cache. The request will go to the origin and you'll now have 3 copies of page 3 in the cache.

Obviously, an application can define it's own request header and mark the response data non-cacheable. And this would still be better than the above. But what I would rather do is ensure that any Range Units that are defined include a bytes range that would at least *allow* for caching. A Vary can probably take care of preventing the kind of cache problems like those described above - given a little thought... I'll try to write up something concrete.

Regardless of how a range is requested, the server is always transferring bytes in the response body - whether indicated by Content-Length, chunks, or whatever. The Content-Range response header just lets the server indicate the returned range associated with the response body *in terms of the indicated range unit* and *in the domain of the representation*.

e.g. If a Range Unit called "blobs" is used, and a request is made with "Range: blobs=5-11", the response could have header "Content-Range: blobs=5-11/50" indicating that 7 blobs are being returned (out of 50) in the response body and a "Content-Length: 123456" could also be supplied designating that the response body (associated with blobs 5-11) is 123456 bytes.

Am I missing something?

All the proposed use cases for new range units that I've seen are application specific, and could arguably better be dealt with using another header.

Adrien

[cp] Doesn't the fact that there's a Range Unit Registry mean that Range Units can be added that are application-specific - assuming there's sufficient demand? It's just a different mechanism than adding custom headers, IMHO - with well-defined semantics.

------ Original Message ------
From: "Mark Nottingham" <[e-mail]>
To: "Darshak Thakore" <[e-mail]>
Cc: "[e-mail]" <[e-mail]>
Sent: 12/05/2016 12:59:25 p.m.
Subject: Re: Issue with "bytes" Range Unit and live streaming

Hi Darshak,

I don't think that's where we're at. Based on the discussion so far, it seems like there are two possible paths forward:

1. Changing the 'bytes' range-unit to allow this use case
2. Minting a new range-unit

As discussed, both have downsides. What we need is data about how current implementations -- especially caching intermediaries -- behave when faced with a) 'bytes' used in the desired way, and b) a new range-unit.

Cheers,

 On 11 May 2016, at 5:57 AM, Darshak Thakore <[e-mail]> wrote:

 Hi all,

 Based on feedback on this thread, it seems like the need for being able to send an open ended (read as unknown-last-byte-pos) Range response has been discussed a couple of times (with different use cases). Also there seems to be somewhat general agreement that the Content-Range ABNF in RFC 7233 is deficient in providing this. The initial argument has been, “is there a compelling enough need” and i think with different use cases popping up (log files, media streaming, gzip… others ??) there seems to be some value in defining a non-application specific Range unit that plugs this gap. Clearly fixing RFC 7233 is invasive and will result in thing breaking in unknown ways so that’s a no-go. With that, we can:
  • Ensure that the scope of this work item is narrow and restricted only to fixing the gap in RFC 7233
  • Define a new range unit (anything with “bytes” in it seems like a bad idea, so maybe call it “live-octets”, “blive”, “b-add" - suggestions welcome….)
  • Decide if there is enough interest/reason to do this as a WG item
 Any objects/suggestions to any of the above ?

 Regards,
 Darshak


 From: "[e-mail]" <[e-mail]>
 Date: Thursday, April 21, 2016 at 4:25 AM
 To: 'Craig Pratt' <[e-mail]>, "[e-mail]" <[e-mail]>
 Cc: "[e-mail]" <[e-mail]>
 Subject: RE: Issue with "bytes" Range Unit and live streaming
 Resent-From: <[e-mail]>
 Resent-Date: Thursday, April 21, 2016 at 4:25 AM

 On Thursday,21 April 2016 05:18 [e-mail] wrote:
 >
 > Re: Representation caching
 >
 > Whether a representation is considered cacheable in this use case is at
 > the discretion of the origin server and specific to the use
 > case/application - as it should be (imho). There's no *necessity* in
 > having a periodically-appended resource marked non-cachable, correct? If
 > the resource mutates, it's not cacheable. If it's just being appended
 > to, it is cacheable. And if an appended resource stops being appended
 > to, it doesn't invalidate the cached representation.
 >

 I couldn't agree more. However, it seemed the prevailing sentiment when we tried to resolve the related issue of ranges before content codings, with a new bbcc unit (bytes-before-content-coding), was that the use cases for append-only growth represent an insignificant portion of HTTP traffic. “We live by app-specific protocols to handle these cases. What is so special ... that it must be addressed by http (in a very ugly way)?” [1]

 [1] https://lists.w3.org/Archives/Public/ietf-http-wg/2014AprJun/1383.html" rel="nofollow">https://lists.w3.org/Archives/Public/ietf-http-wg/2014AprJun/1383.html



 This email message is intended only for the use of the named recipient. Information contained in this email message and its attachments may be privileged, confidential and protected from disclosure. If you are not the intended recipient, please do not read, copy, use or disclose this communication to others. Also please notify the sender by replying to this message and then delete it from your system.

-- Mark Nottingham   https://www.mnot.net/" rel="nofollow">https://www.mnot.net/

--
craig pratt

Caspia Consulting

[e-mail]

503.746.8008

--

craig pratt Caspia Consulting [e-mail] 503.746.8008