[rtcweb] VP8 payload, decoder processing capabilities (was Re: Resolution negotiation - a contribution)

[Stephan Wenger <[e-mail]>]

Hi all,

For context: Harald and myself have been at odds for a while now about the
lack of support for a code point in the VP8 payload that can be used to
negotiate a maximum decoder/bitstream complexity.  Specifically, Harald
(and other VP8 payload folks) suggested that generic mechanisms, such as
the a=framerate attribute of RFC4566 in conjunction with the picture size
aspect of the imageattr of RFC 6236 can be used, at least in the rtcweb
context.  However, as far as I understood our argument, these two
mechanisms in combination are not meant as a limit for decoder complexity
(in terms of samples/sec processing rate), but rather as an indication,
from receiver to sender, of an upper bound of what is "useful to send".
See the email below.  To me, it's quite obvious that an indication of
"useful to send" includes "my decoder can handle this"; however, it could
be more restrictive in that factors other than decoder horsepower could
also be at play, such as screen size, user interface settings, and so on.

I believe that the combination of what can be signaled using the above
mechanisms should be sufficient for rtcweb.  However, I also believe that
it is insufficient for general purpose use, mostly because it requires the
support of RFC 6236, which is not exactly a widely deployed technology.
Further, the a=framerate attribute is not a particularly useful attribute
these days anymore, because variable frame rates, at least for software
encoding/decoding, are the norm.

In previous posts on the payload list (in response to the VP8 payload
WGLC), I have commented on the practical shortcomings of the (lack of)
complexity negotiation, and suggested that this needs to be fixed.

Two options:

1) codify Harald's mechanism (based on a=framerate and imageattr in the
VP8 payload draft, at MUST strength.  "In a declarative context, a
prospective media sender supporting this (VP8 payload) specification MUST
support RFC 4566 a=framerate and RFC6236 imageattr, and MUST include code
points according to both mechanisms to identify the properties of the
media stream.  In an offer-answer context, both offerer and answerer
receiver supporting this VP8 payload specification MUST support
a=framertate and imageattr, and MUST include both in their respective
offer/answer messages, so to identify an operation point that will not
overload the media decoder's capabilities.

The issue with this approach, IMO, is that we are dealing here with three
individual code points (framerate, horizontal and vertical picture size),
where a single code point ought to be sufficient for determining whether a
décor is capable of decoding a stream, at least from a complexity
viewpoint).

2) include, in the V8 payload, a negotiable SDP code point indicating the
complexity of a stream, in units of samples per second processing
requirements or a derivative thereof (such as: levels as used in the MPEG
world).  For example, the VP8 payload could include a single, optional,
negotiable parameter "SamplePerSecond".  If SamplePerSecond were absent in
the SDP, a value of xxxxx must be inferred.  (a sensible value for xxxxx
could be, for example 9216000, which is the number of samples per second
for VGA resolution at 30 Hz).  If SamplePerSecond is present in a
declarative context, it indicates the minimum processing requirements a
decoder must support in order to successfully decode the stream.  In a
symmetric offer-answer context, SamplePerSecond can be used to "dial down"
the complexity of the stream to a value that both encoder and decoder can
support.

My preference is obviously the second proposal, but I'm willing to help
fleshing out either or both of them, just not today :-)

Regards,
Stephan
 


On 4.13.2012 00:45 , "Harald Alvestrand" <[e-mail]> wrote:

>On 04/12/2012 11:13 PM, Stephan Wenger wrote:
>>
>> On 4.12.2012 12:08 , "Harald Alvestrand"<[e-mail]>  wrote:
>>
>>> On 04/12/2012 08:19 PM, Stephan Wenger wrote:
>>>> Hi Harald,
>>>> Thanks for this strawman.  I believe it should work, but I fail to see
>>>> how
>>>> a two dimensional negotiation requirement (negotiating max values for
>>>> framerate and image size--which, in turn, also has two-dimensional
>>>> properties) leads to better interop than a one dimensional negotiation
>>>> (pixels per second processing requirement).
>>> Stephan,
>>>
>>> I do not see this (or the requirement from the use-cases document)
>>>first
>>> and foremost a decoder complexity negotiation; it is a negotiation of
>>> how much data it is useful to send, given the recipient's intended use
>>> of that data.
>> Then such a negotiation should be executed in addition.  Decoder cycle
>> requirement do matter in practical implementations.
>Feel free to propose language that captures this requirement. As noted,
>my I-D fragment doesn't.
>
>