Re: [Sframe] [dispatch] Magnus Westerlund's Block on charter-ietf-sframe-00-00: (with BLOCK and COMMENT)

[Magnus Westerlund <[e-mail]>]

Hi,

> > So do I assume correctly that the idea is that the application layer
> > using SFRAME if it haves multiple independent or sets of dependent
> > streams there will be no support for that aspect in SFRAME layer?
> > Instead it is up to the application to put such information to either
> > put it inside the encapsualted part of the SFRAME or map it to the
> > lower transport layer, like to RTP SSRCs and extension information.
> 
> 
> If i understood you correctly, SFrame is kind of agnostic to the media
> streams. Currently it has a single global frame counter for all the
transport, so
> the number of streams/dependency between them is not known/needed
> by SFrame.

Yes, and when using a multi-stream application with layered encoding being
protected by SFRAME and transported over RTP with repair functions the high
layer application will need to have a model for how it maps individual
SFRAMEs to RTP layer functions to enable them to do their work. You get a
very limited functionality if in an SFU system tries to send all over a
single RTP SSRC. So this becomes a discussion in the RTP Payload format
context when carrying SFRAMEs. 

> 
> 
> >
> >> However, if the encrypted output is going to be transmitted over RTP
> >> using a new packetization format, we need to address how to negotiate
> >> that in the SDP.
> > Good
> >
> >> Note that this new packetization format will not only be alid for
> >> transporting encrypted frames (SFrame or not) but even any other
> >> audio/video frames.
> > I understand the potential exists. However, the recommendation for RTP
> > has been to try to consider Application Level Framing. In the case of
> > SFRAME that consideration moves up above SFRAME in the choice of how
> > data is split into SFRAMEs. However, having an RTP paylaod format for
> > SFRAME, then that should carry SFRAMEs. If you starts putting in other
> > binary objects into it, then you create a new demultiplexing point for
> > format between the RTP payload format and the SFRAME processing. That
> appears unmotivated.
> 
> 
> Note that SFRAME is not the only encryption possible with w3c insertable
> streams, it is up to the application to define its own crypto if they
want. So
> the payload must be able to transport non-SFRAME opaque blobs.

So I would consider this a misuse of the RTP payload format. The media type
will be for SFRAMEs. I understand that you might not be able to prevent this
in WebRTC. However, from an interoperability point of view you will be
stating that this is SFRAMEs. RTP will not be able to tell a difference. But
I don't see that this is will explicitly support carrying other things than
SFRAME. Because that means bringing in other consideration including the
security model for the E2E usage. 

> >
> > Should the RTP payload format aspect of the charter be more explicit
> > in that it needs to disucss the general model of how to use SFRAME and
> > how an application can use the facilities of RTP to get good
> > performance from RTP mechanism like FEC and retransmission?
> 
> 
> I don't think so, RTX and FEC frames MUST not be modified/affected by
> SFRAME/the new packetization format, so they work out of the box. We can
> be explicit about that in the chapter as a requirement.
> 

So I am not saying that RTX or FEC shall be modified. What I am saying is
that the SFRAME payload format should discuss the impact on the performance
of these functions depending on how you structure the SFRAMES across SSRCs.
The most simple example is the one where you put all layers for a video
encoding in a single SSRC. In such a stream you loose a single RTP packet
between the transmitting end-point and the SFU. Now the SFU is only
forwarding the base layer and not the enhancement layers. If base layer and
enhancement layer share RTP stream, the SFU can't determine if the missing
packet contains an base layer data or enhancement layer data. If the base
layer and enhancement layer would be on different SSRC, the fact that there
is a missing packet on the enhancement layer RTP stream means that the SFU
can ignore that as it anyway are not forwarding it. This same argument can
be applied between media sources. So how you map your application level
structures onto RTP do matter for the resulting performance of RTP. Putting
all on a single SSRC is not a path to good transport performance. 

Cheers

Magnus Westerlund