Re: [Moq] Flow to join 15 seconds before live

["Law, Will" <[e-mail]>]

Thanks Cullen for initiating this thread and Cullen and Luke for proposing solutions.

I find the proposed worklow to start N seconds behind to be unnecessarily convoluted, As proposed, it requires the player to make SUBSCRIBE request which is “frozen” , followed by FETCH request, followed by a SUBSCRIBE “unfreeze”. In comparison, our workflow for joining a real-time track is clean – a single SUBSCRIBE and all received objects can be piped to a decode buffer and rendered. We should aspire to an equally clean workflow for non-real-time and VOD playback. These are our two majority join use-cases.

I am not of fan of frozen=true. This is overloading SUBSCRIBE to return state information about the track. This would be better off being moved to an explicit control message, perhaps (as Luke just proposed) TRACK_INFO.

I also support the suggestion to remove relative start and end groups from SUSBCRIBE. This is causing more problems than it is solving. Real-time players don’t care about relative offsets because they always want the latest group. Non-real time players have the luxury of time . The key difference with starting N seconds behind live is that we can afford to spend one RTT discovering things (assuming RTT << N). So a clean workflow would be:


  1.  Client makes the MOQT connection.
  2.  Subscribe to the catalog. The catalog tells it the track names and also details of their numbering schemes and group duration.
  3.  For the track it wants to starts with, request TRACKINFO(track) (or a timeline track, if this is offered by the streaming format).
  4.  If the relay has an active subscription, it can immediately return the latest group number. Otherwise it makes the same request upstream and then relays the answer to the client. The client now knows the latest group number.
  5.  The player uses its knowledge of the latest group, the numbering scheme, the group duration and the desired starting buffer to calculate the starting group G.
  6.  It then makes a single FETCH(startgroup=G endgroup=undefined priority=X). The response from this can be piped directly into its decode buffer. (note, this is a changed API from what we currently have)

We have two options for step 6.

  1.  We can extend SUBSCRIBE to allow a reliable ASCending delivery mode, or
  2.  We can extend FETCH to allow an undefined end group i.e keep delivering into the future until end-of-track is signaled (as I showed above).

I don’t have religion on either of these options, although separating the reliable and ordered nature of FETCHed delivery into its own API seems reasonable and preferable.

Cheers
Will


From: Moq <[e-mail]> on behalf of Luke Curley <[e-mail]>
Date: Monday, March 25, 2024 at 6:58 AM
To: Cullen Jennings <[e-mail]>
Cc: MOQ Mailing List <[e-mail]>
Subject: Re: [Moq] Flow to join 15 seconds before live

Hey Cullen, thank you very much for writing down the proposed flow.

The problem with this approach is that while there is head-of-line blocking on startup (via FETCH), there is no head-of-line blocking during the steady state (via SUBSCRIBE). Any congestion will cause the player buffer to shrink, and unfortunately it will be filled in reverse group order. This makes sense for low latency playback as it enables skipping content, but it will cause excessive buffering for high latency playback that does not skip content.

The other big issue in #421<https://urldefense.com/v3/__https:/github.com/moq-wg/moq-transport/pull/421__;!!GjvTz_vk!ShhE-HViGjsrzsZesMScVrIVVAcs8AfyrZ5jX9EAl3puqYDj97MAshlITS2bwutnaV46CztKMoJk_V0$> is that we still need the ability to SUBSCRIBE at absolute group IDs, otherwise ABR won't work because of numerous race conditions. Even a back-to-back UNSUBSCRIBE 720p + SUBSCRIBE 240p are evaluated at different times based on different cache states (including potentially empty). I can't think of any algorithm that works using only relative SUBSCRIBEs, especially if groups are not aligned between tracks.


As I see it, the main problem with SUBSCRIBE today is the RelativeNext and RelativePrev start/end ID. These are both hard to reason about and implement; what if we removed them altogether? I propose that a SUBSCRIBE could only start/end at an absolute ID or the latest group.

And then we add SUBSCRIBE order=ASC|DESC (#411<https://urldefense.com/v3/__https:/github.com/moq-wg/moq-transport/issues/411__;!!GjvTz_vk!ShhE-HViGjsrzsZesMScVrIVVAcs8AfyrZ5jX9EAl3puqYDj97MAshlITS2bwutnaV46CztKgyJTgA0$>) to indicate if head-of-line blocking is desired, supporting both VOD (old content) and reliable live (new content). I do think we also need SUBSCRIBE priority=N (also #411<https://urldefense.com/v3/__https:/github.com/moq-wg/moq-transport/issues/411__;!!GjvTz_vk!ShhE-HViGjsrzsZesMScVrIVVAcs8AfyrZ5jX9EAl3puqYDj97MAshlITS2bwutnaV46CztKgyJTgA0$>), as otherwise it's undefined how multiple subscriptions/fetches interact once we can conditionally ignore send_order, but that's a bigger discussion.


Reliable Live (Fixed Groups)
-> SUBSCRIBE    range=live..   order=ASC priority=1
<- SUBSCRIBE_OK range=823..
-> SUBSCRIBE    range=818..823 order=ASC priority=0
<- SUBSCRIBE_OK range=818..823

Optional: The first subscribe is lower priority than the follow-up, allowing it to both learn the latest group (823) and use the first RTT to warm the cache/connection with eventually useful data. Your proposed frozen=true is okay but conceptually it's behaving as a HEAD request, which I think we should add anyway (TRACK_INFO?).

Reliable Live (Dynamic Groups)
-> SUBSCRIBE    track=timeline range=live.. order=DESC
<- SUBSCRIBE_OK track=timeline range=823..
-> SUBSCRIBE    track=video    range=818..  order=ASC
<- SUBSCRIBE_OK track=video    range=818..

Optional: The timeline track has aligned groups in this example but you should parse the received OBJECTs instead. It's still going to be 1RTT and the timeline track needs to be fetched anyway for DVR support.

On Sun, Mar 24, 2024, 3:15 PM Cullen Jennings <[e-mail]<mailto:[e-mail]>> wrote:

At the IETF meeting I said I would write something up about how a MoQ client could join and get the video for 15 seconds behind the live edge.


The goal here as I understand it is let the the client get say 15 seconds of video before the the current live edge then start playing roughly that far behind. The reason it would be 15 seconds behind would be e just to create a play out buffer and be able to do things like shift to a lower bitrate subscription (client side ABR style)  if the network was getting crappy without having a stall in the play out. This is not a use case I do so I might be missing something in this but I do want to make sure this use case works if it is important for other people.


Here is how I am thinking about client could do this:

Step 1:

Discover a relay and setup the TLS/ QUIC / MoQ connection to it.

Step 2:

Subscribe to the catalog and get the catalog information. From this learn the track name but also learn that each group is 5 seconds long ( so 3 groups for 15 seconds ). For things that use variable groups sizes, subscribe to the track that gives the mapping from time to group numbers and get the latest data from that.

Step 3:

Subscribe to Track with start=next, and frozen=true.

This will cause the relay to get the information from the upstream if it does not already have it and return information about the live head. Note that if the relay already has a subscription for this track, it does not need to do anything, just retun OK with this relays view of the current live edge.

Relay will return a Subscribe OK with the live edge object - for example, lets say it is group=1234 object=5,

Step 4:

At this point the client knows it needs to go 3 groups back from 1234 so it needs groups 1231 to 1234.

Client sends a Fetch of Group 1231 to 1234 with the direction set to normal not reverse. If the relay. is missing some of this in the relays cache it will request it upstream. The relay will send an OK and start sending all the objects in those three groups. Note that if the relay got several clients joining at same time, and the first requested 1231-1234 and the second client requested 1230-1233, the relay MAY do the optimization book keeping to send then  1231-1234 upstream then for the second just send 1230-1230 upstream as it already has requested the others.

The client will receive all the objects for the three groups in the order of the group id / object id.

Step 5:

When the last object in group 1234 arrives, the client sends and subscribe Update that changes the freeze to false in the original subscribe and causes objects from the subscription to start going to the client.

( and yes we need a way to know what the last object in a group is but that is a separate issue. We agree we will have some way of doing this even though we are not sure exactly what that way is yet )

At this point the client will start receiving objects from group 1235 and future groups.


>From a processing point of view, the client does pretty much the same thing when it gets an object regardless of if it came from Fetch or Subscribe.

Few questions on this:

A. Before we get into if this is the optimal solution for this, am I understand the problem and use case correctly ?

B. Does this explanation make sense and does this solution work ?

C. What is uggly or unfortunate about this solution and how big a deal is that ?