Re: [avtext] [MMUSIC] framemarking: add frame size info

Thanks for the pointer Mo,
I couldn't assist to the IETF in Chicago,so I appreciate so much your
response ;).

Best!!

2017-03-27 17:10 GMT+02:00 Mo Zanaty (mzanaty) <mzanaty@cisco.com>:

> Hi Miguel,
>
> This was discussed in IETF 97 during the AVTEXT session on Frame Marking.
> See the slides and minutes.
> https://datatracker.ietf.org/meeting/97/session/avtext
>
> The recommended and agreed solution was to use RID rather than add frame
> size
> in the Frame Marking header extension.
>
> Thanks,
> Mo
>
>
> From: mmusic <mmusic-bounces@ietf.org> on behalf of Miguel París Díaz <
> mparisdiaz@gmail.com>
> Date: Monday, March 27, 2017 at 3:43 AM
> To: Jonathan Lennox <jonathan@vidyo.com>
> Cc: "avtext@ietf.org" <avtext@ietf.org>, "mmusic@ietf.org" <
> mmusic@ietf.org>
> Subject: Re: [MMUSIC] [avtext] framemarking: add frame size info
>
> Hello again,
> is there anybody considering this proposal, or nobody see the benefits?
>
> Kind regards!!
>
> 2016-11-10 15:18 GMT+01:00 Miguel París Díaz <mparisdiaz@gmail.com>:
>
>> Hello,
>> in the new draft of sdp-simulcast an "RTP Aspect" section [1] has been
>> added, which explains how the media is handled on RTP level.
>>
>> Specifically, In the Media-Switching Mixer section [2] the same thoughts
>> I exposed are said:
>>
>>    This section discusses the behavior in cases where the RTP middlebox
>>    behaves like the Media-Switching Mixer (Section 3.6.2 <https://tools.ietf.org/html/draft-ietf-mmusic-sdp-simulcast-06#section-3.6.2>) in RTP
>>    Topologies [RFC7667 <https://tools.ietf.org/html/rfc7667>].  The fundamental aspect here is that the media
>>    sources delivered from the middlebox will be the mixer's conceptual
>>    or functional ones.  For example, one media source may be the main
>>    speaker in high resolution video, while a number of other media
>>    sources are thumbnails of each participant.
>>
>>    The above results in that the RTP stream produced by the mixer is one
>>    that switches between a number of received incoming RTP streams for
>>    different media sources and in different simulcast versions.  The
>>    mixer selects the media source to be sent as one of the RTP streams,
>>    and then selects among the available simulcast streams for the most
>>    appropriate one.  The selection criteria include available bandwidth
>>    on the mixer to receiver path and restrictions based on the
>>    functional usage of the RTP stream delivered to the receiver.  An
>>    example of the latter, is that it is unnecessary to forward a full HD
>>    video to a receiver if the display area is just a thumbnail.  Thus,
>>    restrictions may exist to not allow some simulcast streams to be
>>    forwarded for some of the mixer's media sources.
>>
>>
>> In our case to provide this feature, currently we have to depay the RTP
>> packets, and apply different types of parses (depending on the codec) to
>> read the frame size, which reduces the scalability of the system and hinder
>> the implementation a lot.
>> Because of that, I think that having frame size (width and height) info
>> in the Frame Marking RTP header extension is quite interesting to implement
>> this kind of use cases in a easy and efficient way (the same that an
>> audio-level extension header is provided to avoid analysing it in the
>> middlebox side).
>>
>> I am adding MMUSIC group in the thread, because I think that this also
>> should be discussed in the context of the simulcast case.
>>
>> Best!!
>>
>> Refs
>> [1] https://tools.ietf.org/html/draft-ietf-mmusic-sdp-simulcast-
>> 06#section-7.2
>> [2] https://tools.ietf.org/html/draft-ietf-mmusic-sdp-simulcast-
>> 06#section-7.2.1
>>
>>
>> 2016-08-30 12:37 GMT+02:00 Miguel París Díaz <mparisdiaz@gmail.com>:
>>
>>> I assume that the media distributor has the information from the SDP (it
>>> performs the SDP negotiation which each "client"), but the point is that
>>> encoders may change the video size depending on the available bandwidth,
>>> the complexivity of the video source, etc., unless the sender forces the
>>> encoders' configuration with a fix frame size...
>>>
>>>
>>> 2016-08-26 19:07 GMT+02:00 Jonathan Lennox <jonathan@vidyo.com>:
>>>
>>>> (As an individual.)
>>>>
>>>> In the latest version of simulcast the media distributor would need the
>>>> RID values, not the PT values, but the idea is the same — it needs the SDP.
>>>>
>>>> Note that if the media distributor doesn’t have information from the
>>>> SDP it can’t reliably identify the frame marking header extension at all,
>>>> since header extension IDs are negotiated. So I’m not sure how much benefit
>>>> there is to putting the size in the header extension.
>>>>
>>>> That said, if we envision a scenario where encoders might be frequently
>>>> changing their video size (in response to available network bandwidth, or
>>>> the like), it might be useful for encoders to be able to indicate the
>>>> current size they’re encoding without needing to send updated SDP all the
>>>> time.
>>>>
>>>> On Aug 26, 2016, at 12:52 PM, Paul E. Jones <paulej@packetizer.com>
>>>> wrote:
>>>>
>>>> Miguel,
>>>>
>>>> You make the assumption that the media distributor will not see the
>>>> SDP, I suppose.  While certainly a valid model, I'll admit that I had
>>>> personally assumed any media forwarding function would see the SDP (or at
>>>> least be told the PT values and any relevant flow information similar to
>>>> what RFC 6236 provides) and would thus know which PT values correspond to
>>>> what video resolutions if simulcast is employed.
>>>>
>>>> Paul
>>>>
>>>> ------ Original Message ------
>>>> From: "Miguel París Díaz" <mparisdiaz@gmail.com>
>>>> To: avtext@ietf.org
>>>> Sent: 8/25/2016 10:12:48 AM
>>>> Subject: [avtext] framemarking: add frame size info
>>>>
>>>> Hello,
>>>> it would be great having frame size (width and height) info in the
>>>> Frame Marking RTP header extension [1].
>>>>
>>>> Why?
>>>> For example, in the case of using simulcast in an SFU, selecting the
>>>> stream by the size would ease the application development and improve the
>>>> experience of the users.
>>>> Application developers don't usually have deep knowledge about media
>>>> like bitrate, etc., but they know which video size has to be rendered in
>>>> the GUI, which may depend on the client where the app is running: a mobile,
>>>> a PC with a 13"· screen, a PC with 27" screen, etc.
>>>>
>>>> In this way and taking a videoconference app as example, if a
>>>> participant select another participant to be rendered as main video, the
>>>> app could ask the SFU to select the video quality that better matches to
>>>> 800x600 size.
>>>>
>>>> What do you think about this idea?
>>>>
>>>> Thanks and best regards!!
>>>>
>>>> Refs
>>>> [1] https://tools.ietf.org/html/draft-ietf-avtext-framemarking-02
>>>>
>>>> --
>>>> Miguel París Díaz
>>>> ------------------------------------------------------------
>>>> ------------
>>>> Computer/Software engineer.
>>>> Researcher and architect in http://www.kurento.org
>>>> http://twitter.com/mparisdiaz
>>>> ------------------------------------------------------------
>>>> ------------
>>>>
>>>> _______________________________________________
>>>> avtext mailing list
>>>> avtext@ietf.org
>>>> https://www.ietf.org/mailman/listinfo/avtext
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Miguel París Díaz
>>> ------------------------------------------------------------------------
>>> Computer/Software engineer.
>>> Researcher and architect in http://www.kurento.org
>>> http://twitter.com/mparisdiaz
>>> ------------------------------------------------------------------------
>>>
>>
>>
>>
>> --
>> Miguel París Díaz
>> ------------------------------------------------------------------------
>> Computer/Software engineer.
>> Researcher and architect in http://www.kurento.org
>> http://twitter.com/mparisdiaz
>> ------------------------------------------------------------------------
>>
>
>
>
> --
> Miguel París Díaz
> ------------------------------------------------------------------------
> Computer/Software engineer.
> Researcher and architect in http://www.kurento.org
> http://twitter.com/mparisdiaz
> ------------------------------------------------------------------------
>

-- 
Miguel París Díaz
------------------------------------------------------------------------
Computer/Software engineer.
Researcher and architect in http://www.kurento.org
http://twitter.com/mparisdiaz
------------------------------------------------------------------------