Re: [Sframe] using sframe with existing media networks

[Justin Uberti <[e-mail]>]

On Fri, Jun 12, 2020 at 1:03 AM Pascal Buhler (pabuhler) <pabuhler=
[e-mail]> wrote:

> Hi,
>
>
>
> I had a couple of questions relating to the draft posted by Emad and
> sframe usage in general, not sure if it is best to post here, on github or
> a different forum.
>
>
>
> First I make the assumption that although sframe is a valid
> concept/standard on its own and that webrtc is not explicitly mentioned, I
> take it that it is in the context of webrtc and the insertable streams api
> that this was originally developed.
>
>
>
> I am looking at this from the prospective of a webrtc gateway, so there is
> no guarantee that there is a webrtc client at both ends.
>
>
>
> I see that the encrypted frame will be packetized with a generic RTP
> packetizer and use a generic RTP header extension, but no references to
> what these are or how one would signal/configure a client to use the
> generic packetize instead of a codec specific one. This is maybe not
> directly relayed to the sframe standard but relevant when using sframe in
> conjunction with other standards, for example SDP.
>

Yes, some form of signaling is definitely needed, perhaps some new option
in a m= section negotiating the use of a generic packetizer.

>
>
> Secondly I am interested in the Partial Decoding section, this seams to
> require that an entire frame is encoded as a single blob but there are many
> codecs that can encode parts of a frame as independent units, for example a
> single NAL that covers only a slice of a frame in H.264 or a single OBU
> that contains a tile in AV1. In these cases the units could still span many
> packets. Is it possible for sframe to work in these cases if one considers
> each of these unit to be a frame in the sframe concept? I am not familiar
> enough with the insertable streams api in webrtc to know how that works in
> the case of slices and tiles.
>

Yes, to enable this each independently decodable unit would be a 'frame'
for SFrame. This would include H.264 slices/AV1 OBUs, as well as individual
layers in a scalable coding context.