Re: [rtcweb] [payload] VP8 payload, decoder processing capabilities (was Re: Resolution negotiation - a contribution)

["Yuepeiyu (Roy)" <[e-mail]>]

Hi Harald and Stephen,

First of all, the "SamplePerSecond" in the option 2 should be the maximal sample per second of the stream, because of the variable frame rates.

Second, if we are talking about signaling of decoder complexity point of view, 
	1) I don't see much difference between option 1 and 2, if option 2 is to signal the real number of "sample per second". 
	2) I believe a derivative of "SamplePerSecond" (still option 2) would make more sense, since to some extent, less values would be seen as a guideline for the encoder to produce the media stream. This is meaningful especially for the declarative context where the encoder has fewer information of the decoder's capability.

Just my 2 cents.
Roy
-----邮件原件-----
发件人: [e-mail] [mailto:[e-mail]] 代表 Harald Alvestrand
发送时间: 2012年4月17日 8:32
收件人: Stephan Wenger
抄送: [e-mail]
主题: Re: [payload] VP8 payload, decoder processing capabilities (was Re: [rtcweb] Resolution negotiation - a contribution)

Responding strictly on the payload list, and regarding the VP8 payload 
format only:

The a=framerate attribute is a maximum framerate only. Since the maximum 
framerate is the one that places the most stress on the decoder, the 
ability to use lower framerates is irrelevant to the question of 
restricting the decoder load.

Apart from that, I believe we agree on what we disagree about; the input 
of other participants would be valuable.

                 Harald

On 04/16/2012 11:40 PM, Stephan Wenger wrote:
> Hi all,
>
> For context: Harald and myself have been at odds for a while now about the
> lack of support for a code point in the VP8 payload that can be used to
> negotiate a maximum decoder/bitstream complexity.  Specifically, Harald
> (and other VP8 payload folks) suggested that generic mechanisms, such as
> the a=framerate attribute of RFC4566 in conjunction with the picture size
> aspect of the imageattr of RFC 6236 can be used, at least in the rtcweb
> context.  However, as far as I understood our argument, these two
> mechanisms in combination are not meant as a limit for decoder complexity
> (in terms of samples/sec processing rate), but rather as an indication,
> from receiver to sender, of an upper bound of what is "useful to send".
> See the email below.  To me, it's quite obvious that an indication of
> "useful to send" includes "my decoder can handle this"; however, it could
> be more restrictive in that factors other than decoder horsepower could
> also be at play, such as screen size, user interface settings, and so on.
>
> I believe that the combination of what can be signaled using the above
> mechanisms should be sufficient for rtcweb.  However, I also believe that
> it is insufficient for general purpose use, mostly because it requires the
> support of RFC 6236, which is not exactly a widely deployed technology.
> Further, the a=framerate attribute is not a particularly useful attribute
> these days anymore, because variable frame rates, at least for software
> encoding/decoding, are the norm.
>
> In previous posts on the payload list (in response to the VP8 payload
> WGLC), I have commented on the practical shortcomings of the (lack of)
> complexity negotiation, and suggested that this needs to be fixed.
>
> Two options:
>
> 1) codify Harald's mechanism (based on a=framerate and imageattr in the
> VP8 payload draft, at MUST strength.  "In a declarative context, a
> prospective media sender supporting this (VP8 payload) specification MUST
> support RFC 4566 a=framerate and RFC6236 imageattr, and MUST include code
> points according to both mechanisms to identify the properties of the
> media stream.  In an offer-answer context, both offerer and answerer
> receiver supporting this VP8 payload specification MUST support
> a=framertate and imageattr, and MUST include both in their respective
> offer/answer messages, so to identify an operation point that will not
> overload the media decoder's capabilities.
>
> The issue with this approach, IMO, is that we are dealing here with three
> individual code points (framerate, horizontal and vertical picture size),
> where a single code point ought to be sufficient for determining whether a
> décor is capable of decoding a stream, at least from a complexity
> viewpoint).
>
> 2) include, in the V8 payload, a negotiable SDP code point indicating the
> complexity of a stream, in units of samples per second processing
> requirements or a derivative thereof (such as: levels as used in the MPEG
> world).  For example, the VP8 payload could include a single, optional,
> negotiable parameter "SamplePerSecond".  If SamplePerSecond were absent in
> the SDP, a value of xxxxx must be inferred.  (a sensible value for xxxxx
> could be, for example 9216000, which is the number of samples per second
> for VGA resolution at 30 Hz).  If SamplePerSecond is present in a
> declarative context, it indicates the minimum processing requirements a
> decoder must support in order to successfully decode the stream.  In a
> symmetric offer-answer context, SamplePerSecond can be used to "dial down"
> the complexity of the stream to a value that both encoder and decoder can
> support.
>
> My preference is obviously the second proposal, but I'm willing to help
> fleshing out either or both of them, just not today :-)
>
> Regards,
> Stephan
>
>
>
> On 4.13.2012 00:45 , "Harald Alvestrand"<[e-mail]>  wrote:
>
>> On 04/12/2012 11:13 PM, Stephan Wenger wrote:
>>> On 4.12.2012 12:08 , "Harald Alvestrand"<[e-mail]>   wrote:
>>>
>>>> On 04/12/2012 08:19 PM, Stephan Wenger wrote:
>>>>> Hi Harald,
>>>>> Thanks for this strawman.  I believe it should work, but I fail to see
>>>>> how
>>>>> a two dimensional negotiation requirement (negotiating max values for
>>>>> framerate and image size--which, in turn, also has two-dimensional
>>>>> properties) leads to better interop than a one dimensional negotiation
>>>>> (pixels per second processing requirement).
>>>> Stephan,
>>>>
>>>> I do not see this (or the requirement from the use-cases document)
>>>> first
>>>> and foremost a decoder complexity negotiation; it is a negotiation of
>>>> how much data it is useful to send, given the recipient's intended use
>>>> of that data.
>>> Then such a negotiation should be executed in addition.  Decoder cycle
>>> requirement do matter in practical implementations.
>> Feel free to propose language that captures this requirement. As noted,
>> my I-D fragment doesn't.
>>
>>
>
>

_______________________________________________
payload mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/payload