Re: [rtcweb] [payload] VP8 payload, decoder processing capabilities (was Re: Resolution negotiation - a contribution)

["Cullen Jennings (fluffy)" <[e-mail]>]

I'd would find it very helpful to see the parameterization of Google hardware implementations. It been said that anyone can get the CHDL/Verilog for this but I can't get it. Looking at that would help understand what are limitations of at least one hardware implementation. I think that would shed a bunch of light on what might be needed. 


On Apr 16, 2012, at 14:40 , Stephan Wenger wrote:

> Hi all,
> 
> For context: Harald and myself have been at odds for a while now about the
> lack of support for a code point in the VP8 payload that can be used to
> negotiate a maximum decoder/bitstream complexity.  Specifically, Harald
> (and other VP8 payload folks) suggested that generic mechanisms, such as
> the a=framerate attribute of RFC4566 in conjunction with the picture size
> aspect of the imageattr of RFC 6236 can be used, at least in the rtcweb
> context.  However, as far as I understood our argument, these two
> mechanisms in combination are not meant as a limit for decoder complexity
> (in terms of samples/sec processing rate), but rather as an indication,
> from receiver to sender, of an upper bound of what is "useful to send".
> See the email below.  To me, it's quite obvious that an indication of
> "useful to send" includes "my decoder can handle this"; however, it could
> be more restrictive in that factors other than decoder horsepower could
> also be at play, such as screen size, user interface settings, and so on.
> 
> I believe that the combination of what can be signaled using the above
> mechanisms should be sufficient for rtcweb.  However, I also believe that
> it is insufficient for general purpose use, mostly because it requires the
> support of RFC 6236, which is not exactly a widely deployed technology.
> Further, the a=framerate attribute is not a particularly useful attribute
> these days anymore, because variable frame rates, at least for software
> encoding/decoding, are the norm.
> 
> In previous posts on the payload list (in response to the VP8 payload
> WGLC), I have commented on the practical shortcomings of the (lack of)
> complexity negotiation, and suggested that this needs to be fixed.
> 
> Two options:
> 
> 1) codify Harald's mechanism (based on a=framerate and imageattr in the
> VP8 payload draft, at MUST strength.  "In a declarative context, a
> prospective media sender supporting this (VP8 payload) specification MUST
> support RFC 4566 a=framerate and RFC6236 imageattr, and MUST include code
> points according to both mechanisms to identify the properties of the
> media stream.  In an offer-answer context, both offerer and answerer
> receiver supporting this VP8 payload specification MUST support
> a=framertate and imageattr, and MUST include both in their respective
> offer/answer messages, so to identify an operation point that will not
> overload the media decoder's capabilities.
> 
> The issue with this approach, IMO, is that we are dealing here with three
> individual code points (framerate, horizontal and vertical picture size),
> where a single code point ought to be sufficient for determining whether a
> décor is capable of decoding a stream, at least from a complexity
> viewpoint).
> 
> 2) include, in the V8 payload, a negotiable SDP code point indicating the
> complexity of a stream, in units of samples per second processing
> requirements or a derivative thereof (such as: levels as used in the MPEG
> world).  For example, the VP8 payload could include a single, optional,
> negotiable parameter "SamplePerSecond".  If SamplePerSecond were absent in
> the SDP, a value of xxxxx must be inferred.  (a sensible value for xxxxx
> could be, for example 9216000, which is the number of samples per second
> for VGA resolution at 30 Hz).  If SamplePerSecond is present in a
> declarative context, it indicates the minimum processing requirements a
> decoder must support in order to successfully decode the stream.  In a
> symmetric offer-answer context, SamplePerSecond can be used to "dial down"
> the complexity of the stream to a value that both encoder and decoder can
> support.
> 
> My preference is obviously the second proposal, but I'm willing to help
> fleshing out either or both of them, just not today :-)
> 
> Regards,
> Stephan
> 
> 
> 
> On 4.13.2012 00:45 , "Harald Alvestrand" <[e-mail]> wrote:
> 
>> On 04/12/2012 11:13 PM, Stephan Wenger wrote:
>>> 
>>> On 4.12.2012 12:08 , "Harald Alvestrand"<[e-mail]>  wrote:
>>> 
>>>> On 04/12/2012 08:19 PM, Stephan Wenger wrote:
>>>>> Hi Harald,
>>>>> Thanks for this strawman.  I believe it should work, but I fail to see
>>>>> how
>>>>> a two dimensional negotiation requirement (negotiating max values for
>>>>> framerate and image size--which, in turn, also has two-dimensional
>>>>> properties) leads to better interop than a one dimensional negotiation
>>>>> (pixels per second processing requirement).
>>>> Stephan,
>>>> 
>>>> I do not see this (or the requirement from the use-cases document)
>>>> first
>>>> and foremost a decoder complexity negotiation; it is a negotiation of
>>>> how much data it is useful to send, given the recipient's intended use
>>>> of that data.
>>> Then such a negotiation should be executed in addition.  Decoder cycle
>>> requirement do matter in practical implementations.
>> Feel free to propose language that captures this requirement. As noted,
>> my I-D fragment doesn't.
>> 
>> 
> 
> 
> _______________________________________________
> payload mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/payload