Re: [Hls-interest] LL-HLS Amendment Proposal: Optional part response headers for CDN efficiency.

[Jan Van Doorn <[e-mail]>]

One of the original LL-HLS design goals was to scale with regular CDNs and not require anything other than HTTP/2 on the caches. I think CDNs work best when they just implement the HTTP spec. I think that is a strong reason to not implement edge stitching as well.

Rgds,
JvD


> On Feb 12, 2021, at 6:11 PM, Weil, Nicolas <[e-mail]> wrote:
> 
> From the origin perspective, I would add that we’d have a strong reason to not implementing it: the edge stitching dilutes responsibility and makes it impossible to debug efficiently when things start to go wrong.
>  
> I would rather propose that origins add a specific header to the last part of a full segment, with the name of it, so that CDNs can prefetch the full segment and provide the best delivery performance possible – something like hls-fullsegment-prefetch: vid720_segment_1521.m4v
>  
> This would work well with ad insertion discontinuities and preserve a clear split of the responsibilities between the origin and the CDN.
>  
> Thanks,
> Nicolas
> ----------------
> Nicolas Weil | Senior Product Manager – Media Services
> AWS Elemental
>  
> From: Hls-interest <[e-mail] <mailto:[e-mail]>> On Behalf Of Law, Will
> Sent: Friday, February 12, 2021 2:28 PM
> To: [e-mail] <mailto:[e-mail]>
> Subject: RE: [Hls-interest] LL-HLS Amendment Proposal: Optional part response headers for CDN efficiency.
>  
> Hi Roger
>  
> From an Akamai perspective, we acknowledge the issue raised by Andrew, in that duplicate parts and segments reduce cache efficiency at the edge. This is something the HLS community should strive to reduce over time. I have four main responses to this proposal.
>  
> Firstly, we feel that the existing HLS spec <https://tools.ietf.org/html/draft-pantos-hls-rfc8216bis-08> already provides a solution to this problem, namely the use of byte-range addressing for parts. Under this addressing mode, the only objects the origin produces are segments. These are the single objects that are cached at the edge. The use of ranges to retrieve parts affords the clients the ability to lower their latency below the segment duration and to start and switch quickly. This approach has the following advantages over edge-stitching :
>  
> Segment caching and byte-range delivery is natively supported by any CDN which supports Http2 delivery. It does not need to be taught new behaviors (RFC8673 edge case aside). One of the secrets to HLS success has been the simplicity of scaling it out. Any http server has sufficed in the past. In introducing edge stitching, we are moving to more of a Smooth streaming model, which places a dependency on logic at the edge. This logic has to be implemented consistently across multiple CDNs and introduces critical path complexity which has to be managed.
> Edge stitching has an overhead cost, mainly in directory searches to discover aggregate parts. Searches are always more expensive than lookups since you must span the whole directory tree. This compute cost would have to be absorbed by the CDN. Edge stitching is basically trading cache efficiency for compute. Byte-range does not force you to make this trade-off.
>  
> Secondly, the period time in which we have duplicate cache content is not actually that long. Per the HLS spec, blocking media objects requests (such as parts) should be cached for 6 target durations (basically a 100% safety factor since they are only described for the last 3 target durations of the streams). For 4s segments, this means we can evict our stored parts after 24s. The media segments, which can be requested by standard latency clients and as well as low latency clients scrubbing behind live, need to be cached longer. 
>  
> Consider the case of a live stream with 4s segments and 1s part durations.
>  
> After 40s of streaming, we have
> 10 media segments holding 40s of data in the cache
> 24s of duplicate part data
> The overall cache duplication is 24/40  = 60%
>  
> After 5mins (300s) of streaming, we have
> 300s of media segments in the cache
> 24s of duplicate part data
> The overall cache duplication is 24/300  = 8%
>  
> After 30mins (1800s) of streaming, we have
> 1800s of media segments in the cache
> 24s of duplicate part data
> The overall cache duplication is 24/1800  = 1.3%
>  
> So streams with realistic durations in the minutes actually have quite a low percentage of duplicate data, as long as the CDN is aggressive about cache eviction and the origin does a good job in setting cache-control headers.
>  
> Thirdly, at Akamai we would have a complex time in implementing the edge stitching as proposed and the same may be true for other CDNs. The reason is that while the origin header information gets written in to our cache store entry table, the store tables are architected to very efficiently tell you if an object exists and if so, to return it. They are not databases optimized for horizontal searching. We cannot search across the cache, for example asking for all objects whose X-HLS-Part-Root-Segment header matches a certain string. It would very difficult to implement the edge stitch proposed here. We would need to externalize the header information in to some sort of parallel database which we could query. While we have such structures (via EdgeWorkers and EdgeKV), their use would raise the cost and complexity of delivering LL-HLS. At that point we would probably choose to suffer the low duplication rates and instead focus on efficiently evicting parts.
>  
> Fourthly, if the community opinion is to still proceed with this edge-stitch plan, then I would offer the following suggestions:
> 1.       To avoid header bloat, the sequence and offset headers could be collapsed into a single header, for example HLS-Part-Info:<current-part>,<total-part-count>,<byte-offset. This would look like HLS-Part-Info:2,8,623751. Due to HPACK and QPACK header compression, we would not want to place the root-segment in the same bundle, as it will be invariant over the parts from the same segment and hence can be compressed more efficiently if it is separate.
> 2.       The IETF strongly discourages the use of X- in header prefixes <https://tools.ietf.org/html/rfc6648>. A simple header name such as ‘HLS-part-info’ would be preferable.
> 3.       Do you need both byte offset and sequence? Once you know the sequence, you can read the byte-lengths from the individually stored parts.
> 4.       Segments get truncated without warning, often for ad insertion discontinuities and always at the end when the encoder is turned off.  Say you are making 8 parts per 4s segment and have sent off the first two parts to the CDN before reaching a sudden discontinuity. You have labelled these as 1/8 and 2/8 respectively. Since 3-8 are never produced, the edge routine would waste some time and resources looking for 3-8, before giving up and going to the origin to fetch the segment. Performance – especially TTFB and apparent throughput – would suffer.
> 5.       You may have an edge server which is only serving legacy clients pulling segments and no low-latency clients seeding the edge with part requests. In this case, the edge would waste time searching for constituent parts before giving up and going to the origin to fetch the segment. Performance again would suffer. 
>  
> I appreciate Limelight raising these issues and look forward to debating a mutually efficient solution which benefits content distributors, CDNs and players.
>  
> Have a good long weekend!
>  
> Cheers
> Will
>  
>  
> --------------------------------------------------------
> Chief Architect – Edge Technology Group
> Akamai Technologies
> San Francisco
> Cell: +1.415.420.0881
>  
>  
>  
>  
>  
>  
>  
> From: Roger Pantos <rpantos=[e-mail] <mailto:rpantos=[e-mail]>>
> Date: Friday, February 12, 2021 at 10:40 AM
> To: Andrew Crowe <acrowe=[e-mail] <mailto:acrowe=[e-mail]>>
> Cc: "[e-mail] <mailto:[e-mail]>" <[e-mail] <mailto:[e-mail]>>
> Subject: Re: [Hls-interest] LL-HLS Amendment Proposal: Optional part response headers for CDN efficiency.
>  
>  
>  
> 
> On Feb 9, 2021, at 7:54 AM, Andrew Crowe <acrowe=[e-mail] <mailto:acrowe=[e-mail]>> wrote:
>  
> Hello,
> 
> CMAF content packaged and delivered using LL-DASH and range-based LL-HLS are easily managed as duplicate content by CDNs as they specify only segment files. In fact, once the segment is complete, it can then be served out of CDN cache for players that are not Low Latency capable - effectively reducing latency for them as well. Part-based LL-HLS introduces individually named part files that then collapse to the separately named segment file upon final part completion. This then means that on first request for the whole collapsed segment file the CDN will have to go back to origin to request bytes that it likely already has in the individually named part files. CDNs can improve cache efficiency, origin hit rate, and whole segment delivery times with a little bit of additional information from origin.
> 
> 
> On request for a named part file an origin may provide a set of response headers:
> 
> *X-HLS-Part-Sequence*
> A multi value header that represents the current part sequence (index=1) and the total number of parts for the segment. The values will be separated by a forward slash ("/"). For example a 2 second segment with 8 parts per segment will respond to the 2nd part request (vid720_segment_1521.part2.m4v) like
> X-HLS-Part-Sequence: 2/8
> 
> 
> *X-HLS-Part-Offset*
> A single value header that represents the byte offset of the part in the segment. The first part of a segment will always be 0 while, for example the second .25s part of a 2mpbs stream (vid720_segment_1521.part2.m4v) may have a value like 623751
> 
> 
> *X-HLS-Part-Root-Segment*
> A single value header that provides the name of the root segment of the current part. This lets the CDN/proxy know which root file to concatenate the parts into. vid720_segment_1521.part2.m4v would have a value of vid720_segment_1521.m4v
> 
> 
> With the information from these three headers the CDN can recognize the individually named part files as ranges of a larger file, store them effectively and deliver a better experience to viewers across all formats. 
>  
> Hello Andrew. I’m interested in this proposal, but I’d also like to hear some feedback from others in the CDN and packager spaces. Specifically, I’d like to know if other folks:
>  
> - Agree that it’s a good way to solve the problem
>  
> - Can spot any problems or limitations in this proposal that might make it difficult to produce (or consume) these headers
>  
> - Can see themselves implementing it
>  
>  
> thanks,
>  
> Roger Pantos
> Apple Inc.
>  
> 
> 
> Regards,
> -Andrew
> -- 
>  <https://urldefense.com/v3/__https:/www.limelight.com/__;!!GjvTz_vk!Gcp-s1jYCIAYpNsmuK09dLU1cDo5FUINbvdFY1ZXSct8lPTh9xqsUiZS1ril$>
> Andrew Crowe Architect
> EXPERIENCE FIRST.
> [phone] <tel:+1+859+583+3301>
> www.limelight.com <https://urldefense.com/v3/__https:/www.limelight.com/__;!!GjvTz_vk!Gcp-s1jYCIAYpNsmuK09dLU1cDo5FUINbvdFY1ZXSct8lPTh9xqsUiZS1ril$>
>  <https://urldefense.com/v3/__https:/www.facebook.com/LimelightNetworks__;!!GjvTz_vk!Gcp-s1jYCIAYpNsmuK09dLU1cDo5FUINbvdFY1ZXSct8lPTh9xqsUqon8WvN$> <https://urldefense.com/v3/__https:/www.linkedin.com/company/limelight-networks__;!!GjvTz_vk!Gcp-s1jYCIAYpNsmuK09dLU1cDo5FUINbvdFY1ZXSct8lPTh9xqsUgrrsizx$> <https://urldefense.com/v3/__https:/twitter.com/llnw__;!!GjvTz_vk!Gcp-s1jYCIAYpNsmuK09dLU1cDo5FUINbvdFY1ZXSct8lPTh9xqsUvWN2UYO$>
>  
> -- 
> Hls-interest mailing list
> [e-mail] <mailto:[e-mail]>
> https://www.ietf.org/mailman/listinfo/hls-interest <https://www.ietf.org/mailman/listinfo/hls-interest>
>  
> 
> -- 
> Hls-interest mailing list
> [e-mail] <mailto:[e-mail]>
> https://www.ietf.org/mailman/listinfo/hls-interest <https://www.ietf.org/mailman/listinfo/hls-interest>