Re: [AVTCORE] Some (SFrame RTP encapsulation) questions to ponder

Sergio,

I think you had done this analysis for h264, vp8 and vp9 two years ago. There are some existing  slides about it.

If Apple/Intel add a slide for h265 since they support it in safari, we add av1, Stefan bring in h266, we have more or less most of the codecs of interest and enough data to judge.

We could have three sections:
1 - what do you need to use to go into a generic packetizer and how much is codec specific. 
2 - what do You need to duplicate in an rtp header extension for an SFU to work.
I see three different things:
- base for encrypted non-SVC payload (frame marking)
- scalability structure (payload encrypted or not).
a/ I have not had the time to look at Stefan a proposal for the new MPEG codecs but his messages on the list were going in the same direction (higher complexity with full SVC than with temporal scalability only). 
b/ I do not know how far appart the AV1 structure and the vvc or ecc would be, if there is a way to make it generic, and if it a even a goal we would want to have.
c/ I understand there was originally a framemarking part to the vvc rtp payload proposal, I m wondering if there was a gap when trying to represent all SVC structures the vvc bistream otherwise provides.
- sfu optimisation structures, eg for av1 the Instant decidability decision structure (payload encrypted or not), whic is not critical for sfu operation but allow much faster adaptation and smarter CC.

If we go down that path that could mean a codec specific version of each of:
- Payload format
- Sframe rtp packetizer
- rtp extension header

With the following triplet already more or less working well together already:
[h264, generic, framemarking]
[VP8, generic, framemarking]
[VP9, generic, DD]
[AV1, generic, DD]

Now in terms of security for the encrypted case, that means the presence or absence of a given rtp header extension leaks info about e.g which codec is being used. I m not sure this is something w can completely avoid anyway. 

Sent from my iPhone

> On 11 Mar 2021, at 15:15, Sergio Garcia Murillo <sergio.garcia.murillo@gmail.com> wrote:
> 
> 
> 
> 
>> El jue, 11 mar 2021 a las 7:22, Bernard Aboba (<bernard.aboba@gmail.com>) escribió:
>> 
>> I ask this because there appear to be codecs that are widely deployed (e.g. VP8) that require "hacks" for SFUs to forward, as noted in the presentation at IETF 109.  In the case of VP8, the "hack" was required because of a deficiency in the VP8 decoder specification (e.g. the requirement that PictureIDs be consecutive, not just monotonically increasing).  This means that the "hacks" described can't be eliminated just by coming up with a better RTP forwarding extension.  You'd actually need to change the VP8 specification and upgrade VP8 decoders. 
>> 
> 
> The PictureID and the TL0PICIDX  are defined in rfc7741 RTP Payload Format for VP8 Video and not on the VP8 spec. So, if using an agnostic packetizer instead of the rfc7741 one, the issue just doesn't exist.
> 
> Best regards
> Sergio
> _______________________________________________
> Audio/Video Transport Core Maintenance
> avt@ietf.org
> https://www.ietf.org/mailman/listinfo/avt