Re: [Hls-interest] Survey: signaling HLS client capability

[Roger Pantos <[e-mail]>]

Great discussion, thank you to everyone for contributing insights. This is a single reply with responses to several of the major points; please let me know if it gets too hard to follow:

> On May 21, 2021, at 1:48 PM, Law, Will <wilaw=[e-mail] <mailto:wilaw=[e-mail]>> wrote:
> Architecturally, I see concerns in having servers making decisions for clients. The reasons for this are two-fold:
> 1. There is a clear and strong movement towards client privacy. This includes the deprecation and freezing of User-Agent and other client identifiers, which is well under way. If not already, it will become increasingly harder for servers to make any sort of reliable inference about a client's capabilities based purely upon the request.

Agreed that privacy is a concern. I expect that this kind of a signal would typically follow in lockstep with either the OS release (Apple native clients) or app generation (HLS.js et al), so the anonymity set (per https://w3c.github.io/fingerprinting-guidance/ <https://w3c.github.io/fingerprinting-guidance/>) would remain fairly large.

To give everyone a sense of where we're leaning at the moment, I'm thinking of a single _HLS_capabilities query parameter of the form _HLS_capabilities="CS-1,INT,LL,V-9" (alpha-ordered).
  CS-1 : client supports Content Steering protocol version 1
  INT : support Interstitials
  LL : supports low-latency
  V-9 : understands HLS protocol version 9

Haven't decided on the exact set yet; I'm interested in hearing about requirements.

> Two schools of thought then as to how clients can handle diversity of content & features:
> Clients should ask explicitly for what they want in a way that is consistent and the response cacheable . You have already established a convention of reserved _HLS_ query args  on which the newer features of HLS are quite dependent. It would make sense to extend this to allow players to add additional filters as they request content. Examples of filters:
> _HLS_codec=hevc, avc
> _HLS_bitrate=500000-4000000
> _HLS_lang=fr
> _HLS_admode=interstitial
> Another approach is that the main playlist is a menu of all content available, and that the client is capable of extracting the pieces from it that it wants. Real-time feature-based detection, rather than brittle device categorization, is the sustainable way forward. We already require smart clients to play back HLS. Those same clients are aware of which features (low latency, steering, interstitial support) they support as well as are capable of discovering the capabilities (codec, HDCP, HDR etc) of the host device. This approach gives the same main playlist to all clients, simplifying distribution.

Right. [2] is what we have today, and in general I still prefer having the server vend everything and having the client be smart enough to choose what to use. 

We're considering [1] for cases in which providing everything in a single master playlist places too much of a burden on the server. Gaetan gave an example where offering both (new) interstitials and (backward-compatible) segment stitching ad mode would be a big lift.

>  
> One problem with approach [1] is that you would want to use it when the client requests the main playlist, however it doesn’t know if the origin supports such filtering. It could therefore request AVC as the only codec, but receive a static playlist back from an origin that did not support filtering. There could be a playlist attribute added to indicate that the origin acknowledges the requests and has implemented the filters. If the client does not see such an acknowledgement echoed back, then it would know that it can’t trust the origin and would have to implement some decisions of its own. One could then argue that if it can make those decisions on its own, then why entertain the complexity of the filters? If a client is modern enough to add these filters to its requests, then couldn’t it simply apply them itself? 

My advice to clients would be not to rely on any kind of playlist filtering in response to the capabilities offered, and to be prepared to deal with anything the server throws its way. 

Related, _HLS_capabilities is clearly optional, on both sides.

> On May 21, 2021, at 2:24 PM, Alex Giladi <[e-mail] <mailto:[e-mail]>> wrote:
> 
> Regardng [1] in Will's e-mail, I'm concerned about nuclear
> proliferation of query parameters and special snowflake server-side
> logic.

Agreed. I think that if we restrict it to a single query parameter, it becomes manageable for CDNs to include it or exclude it from their cache keys as appropriate.

Regarding a proliferation of special snowflake server behavior, I agree that this feature is somewhat prone to what I'd consider misuse (or at least overuse). I don't have an immediate solution for that.

> On May 25, 2021, at 12:30 PM, Gaëtan Hervouet <gaetanhervouet=[e-mail] <mailto:gaetanhervouet=[e-mail]>> wrote:
> I do agree with Will’s response. Relying on user-agent [1] is indeed becoming harder and we have to move away from that. Cacheability and debuggability are of course valid points [2] which is why the query parameters do seem like a good solution (as long as there is no explosion of combinations). I do see the problem with having the query parameter sent on master manifest request impact the content of the variant playlists; it would break cacheability. However, if that parameter were to be set for all requests of the playback session (including variants) then the problem goes away and the response is now deterministic and cacheable.

I'm not sure if I agree with this last bit. If a client indicates that it does (or doesn't) support interstitials, the server can hand it a master playlist with either interstitial-laden URLs or stitched URLs. There is no reason that I can see for them to be the same URLs, so they ought to be independently cacheable. 

> We may also want to consider an approach where the filters are not "features" but rather an HLS spec/draft version that the player is compatible with. 


My thinking is to provide both. Note that solely providing a spec/draft version is insufficient because a client might implement aspect of the spec (interstitials) but not another (low-latency).


Roger.