Re: [MMUSIC] [avtext] framemarking: add frame size info

Miguel París Díaz <mparisdiaz@gmail.com> Mon, 27 March 2017 07:43 UTC

Return-Path: <mparisdiaz@gmail.com>
X-Original-To: mmusic@ietfa.amsl.com
Delivered-To: mmusic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 612C212949B; Mon, 27 Mar 2017 00:43:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.698
X-Spam-Level:
X-Spam-Status: No, score=-2.698 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hrF-n98k-5Il; Mon, 27 Mar 2017 00:43:08 -0700 (PDT)
Received: from mail-yw0-x22a.google.com (mail-yw0-x22a.google.com [IPv6:2607:f8b0:4002:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4B86C129463; Mon, 27 Mar 2017 00:43:08 -0700 (PDT)
Received: by mail-yw0-x22a.google.com with SMTP id d191so25405478ywe.2; Mon, 27 Mar 2017 00:43:08 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=mPS8lcSH7pHRAn+YUDZ+BvYETb2lGky+LgtesjZqq6Y=; b=W7zoochEFOQ5gg2kS98hxxbFm9YjOXSX8HwElq/Jt9AanZUhxdibPws9yL+aqUpqWB ydH3K0weuQGuXBzN9tTwodDn4qUu+X0Kj7e8gkQ9QRqimkYaREnZaNDA1l4RMy9svkrU jLyDzFWUgP4qUioO8HonagZpHUVXKqUp8Gow0C3Ceg2gcj2POIqvIFKQ6r1r+KHdahhb HiiVXrcBz/EgTDlrXNUDvgrm6DmPbybWBuWOF4FgIH+UTLmugCWg+IvX75Z+ITfIcQb5 vBGIRVeC64dyyTIjgmi2adUZVSvqjFFFbIDNKnRI0qSqQrKoUBkwLuAu9p9Wb+HI4eZH 66sg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=mPS8lcSH7pHRAn+YUDZ+BvYETb2lGky+LgtesjZqq6Y=; b=aoadpdLHBres+1QaomgmdSUHKBPso+d+VtaZpZv7XU9tgkecfQjWY07vnZSgyuJSdM Xr0cUfzaif4+6HSBgx6pjOenmesz/2qh730/guKtlPpqry910bZEw4hhAS2RtxyBgrHL ajS5k39aNZ+bigO5guDHaM4Y2NmW54L/qtkzFnd4vjlfNa7K+0yi1n8jVkWqws57FEEE MwC4kl5C/UluhPoCeYazaYGicCyrtZZfvxmDy0gchssj8XDteUcgMYB6f5WfAJzGNpAC pHPiuH3Qg+TLiRCVHBSBSSlwZCA9/sq9RMjNWkVuc5yvn5jmTaOHctFmJeVwLP0F4je6 lp+Q==
X-Gm-Message-State: AFeK/H3+k7WdZxzmI1+9VPibNWG4pmQRDrnZ2c/lYH74S+HNFVULJzPks5GfBhhjhqr3vgoYSIshJ3KrvpHPhw==
X-Received: by 10.129.125.5 with SMTP id y5mr15437149ywc.120.1490600587208; Mon, 27 Mar 2017 00:43:07 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.13.230.73 with HTTP; Mon, 27 Mar 2017 00:43:06 -0700 (PDT)
In-Reply-To: <CAEn+E3iqskKLDidPnw2Y3DGMP_x-rWD_tnuC7K3vT=EU5gb7cw@mail.gmail.com>
References: <CAEn+E3h-b=8VEkhZ56Z9Ww+mTCA2H1B93UAkgbfmySyi2CnvnA@mail.gmail.com> <em8de2860d-9b70-44ce-87e5-3c6ecb1fb1ee@sydney> <B4BD5FDA-FB39-4714-92A3-EE647A8D06D9@vidyo.com> <CAEn+E3jt9gzKU748uJrxAsu6eY-c5G23_=u6SHLRAv=oD4Z-ow@mail.gmail.com> <CAEn+E3iqskKLDidPnw2Y3DGMP_x-rWD_tnuC7K3vT=EU5gb7cw@mail.gmail.com>
From: =?UTF-8?Q?Miguel_Par=C3=ADs_D=C3=ADaz?= <mparisdiaz@gmail.com>
Date: Mon, 27 Mar 2017 09:43:06 +0200
Message-ID: <CAEn+E3gK4CQ3WEXJvitUePb4N4au2uEEQZDagVBfPSKjinkxXg@mail.gmail.com>
To: Jonathan Lennox <jonathan@vidyo.com>
Cc: "Paul E. Jones" <paulej@packetizer.com>, "avtext@ietf.org" <avtext@ietf.org>, mmusic <mmusic@ietf.org>
Content-Type: multipart/alternative; boundary=001a1149364480cfa5054bb17ee0
Archived-At: <https://mailarchive.ietf.org/arch/msg/mmusic/y47ZdOK_YzSQ_bwanN6mzhfKMjs>
Subject: Re: [MMUSIC] [avtext] framemarking: add frame size info
X-BeenThere: mmusic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Multiparty Multimedia Session Control Working Group <mmusic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/mmusic>, <mailto:mmusic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/mmusic/>
List-Post: <mailto:mmusic@ietf.org>
List-Help: <mailto:mmusic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/mmusic>, <mailto:mmusic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 27 Mar 2017 07:43:11 -0000

Hello again,
is there anybody considering this proposal, or nobody see the benefits?

Kind regards!!

2016-11-10 15:18 GMT+01:00 Miguel París Díaz <mparisdiaz@gmail.com>om>:

> Hello,
> in the new draft of sdp-simulcast an "RTP Aspect" section [1] has been
> added, which explains how the media is handled on RTP level.
>
> Specifically, In the Media-Switching Mixer section [2] the same thoughts I
> exposed are said:
>
>    This section discusses the behavior in cases where the RTP middlebox
>    behaves like the Media-Switching Mixer (Section 3.6.2 <https://tools.ietf.org/html/draft-ietf-mmusic-sdp-simulcast-06#section-3.6.2>) in RTP
>    Topologies [RFC7667 <https://tools.ietf.org/html/rfc7667>].  The fundamental aspect here is that the media
>    sources delivered from the middlebox will be the mixer's conceptual
>    or functional ones.  For example, one media source may be the main
>    speaker in high resolution video, while a number of other media
>    sources are thumbnails of each participant.
>
>    The above results in that the RTP stream produced by the mixer is one
>    that switches between a number of received incoming RTP streams for
>    different media sources and in different simulcast versions.  The
>    mixer selects the media source to be sent as one of the RTP streams,
>    and then selects among the available simulcast streams for the most
>    appropriate one.  The selection criteria include available bandwidth
>    on the mixer to receiver path and restrictions based on the
>    functional usage of the RTP stream delivered to the receiver.  An
>    example of the latter, is that it is unnecessary to forward a full HD
>    video to a receiver if the display area is just a thumbnail.  Thus,
>    restrictions may exist to not allow some simulcast streams to be
>    forwarded for some of the mixer's media sources.
>
>
> In our case to provide this feature, currently we have to depay the RTP
> packets, and apply different types of parses (depending on the codec) to
> read the frame size, which reduces the scalability of the system and hinder
> the implementation a lot.
> Because of that, I think that having frame size (width and height) info in
> the Frame Marking RTP header extension is quite interesting to implement
> this kind of use cases in a easy and efficient way (the same that an
> audio-level extension header is provided to avoid analysing it in the
> middlebox side).
>
> I am adding MMUSIC group in the thread, because I think that this also
> should be discussed in the context of the simulcast case.
>
> Best!!
>
> Refs
> [1] https://tools.ietf.org/html/draft-ietf-mmusic-sdp-
> simulcast-06#section-7.2
> [2] https://tools.ietf.org/html/draft-ietf-mmusic-sdp-
> simulcast-06#section-7.2.1
>
>
> 2016-08-30 12:37 GMT+02:00 Miguel París Díaz <mparisdiaz@gmail.com>om>:
>
>> I assume that the media distributor has the information from the SDP (it
>> performs the SDP negotiation which each "client"), but the point is that
>> encoders may change the video size depending on the available bandwidth,
>> the complexivity of the video source, etc., unless the sender forces the
>> encoders' configuration with a fix frame size...
>>
>>
>> 2016-08-26 19:07 GMT+02:00 Jonathan Lennox <jonathan@vidyo.com>om>:
>>
>>> (As an individual.)
>>>
>>> In the latest version of simulcast the media distributor would need the
>>> RID values, not the PT values, but the idea is the same — it needs the SDP.
>>>
>>> Note that if the media distributor doesn’t have information from the SDP
>>> it can’t reliably identify the frame marking header extension at all, since
>>> header extension IDs are negotiated. So I’m not sure how much benefit there
>>> is to putting the size in the header extension.
>>>
>>> That said, if we envision a scenario where encoders might be frequently
>>> changing their video size (in response to available network bandwidth, or
>>> the like), it might be useful for encoders to be able to indicate the
>>> current size they’re encoding without needing to send updated SDP all the
>>> time.
>>>
>>> On Aug 26, 2016, at 12:52 PM, Paul E. Jones <paulej@packetizer.com>
>>> wrote:
>>>
>>> Miguel,
>>>
>>> You make the assumption that the media distributor will not see the SDP,
>>> I suppose.  While certainly a valid model, I'll admit that I had personally
>>> assumed any media forwarding function would see the SDP (or at least be
>>> told the PT values and any relevant flow information similar to what RFC
>>> 6236 provides) and would thus know which PT values correspond to what video
>>> resolutions if simulcast is employed.
>>>
>>> Paul
>>>
>>> ------ Original Message ------
>>> From: "Miguel París Díaz" <mparisdiaz@gmail.com>
>>> To: avtext@ietf.org
>>> Sent: 8/25/2016 10:12:48 AM
>>> Subject: [avtext] framemarking: add frame size info
>>>
>>> Hello,
>>> it would be great having frame size (width and height) info in the Frame
>>> Marking RTP header extension [1].
>>>
>>> Why?
>>> For example, in the case of using simulcast in an SFU, selecting the
>>> stream by the size would ease the application development and improve the
>>> experience of the users.
>>> Application developers don't usually have deep knowledge about media
>>> like bitrate, etc., but they know which video size has to be rendered in
>>> the GUI, which may depend on the client where the app is running: a mobile,
>>> a PC with a 13"· screen, a PC with 27" screen, etc.
>>>
>>> In this way and taking a videoconference app as example, if a
>>> participant select another participant to be rendered as main video, the
>>> app could ask the SFU to select the video quality that better matches to
>>> 800x600 size.
>>>
>>> What do you think about this idea?
>>>
>>> Thanks and best regards!!
>>>
>>> Refs
>>> [1] https://tools.ietf.org/html/draft-ietf-avtext-framemarking-02
>>>
>>> --
>>> Miguel París Díaz
>>> ------------------------------------------------------------------------
>>> Computer/Software engineer.
>>> Researcher and architect in http://www.kurento.org
>>> http://twitter.com/mparisdiaz
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> avtext mailing list
>>> avtext@ietf.org
>>> https://www.ietf.org/mailman/listinfo/avtext
>>>
>>>
>>>
>>
>>
>> --
>> Miguel París Díaz
>> ------------------------------------------------------------------------
>> Computer/Software engineer.
>> Researcher and architect in http://www.kurento.org
>> http://twitter.com/mparisdiaz
>> ------------------------------------------------------------------------
>>
>
>
>
> --
> Miguel París Díaz
> ------------------------------------------------------------------------
> Computer/Software engineer.
> Researcher and architect in http://www.kurento.org
> http://twitter.com/mparisdiaz
> ------------------------------------------------------------------------
>



-- 
Miguel París Díaz
------------------------------------------------------------------------
Computer/Software engineer.
Researcher and architect in http://www.kurento.org
http://twitter.com/mparisdiaz
------------------------------------------------------------------------