Re: [payload] VP8 payload, decoder processing capabilities

["Sandy (Alexander) MacInnis" <[e-mail]>]

Thanks Olivier.

I was hoping to hear from someone at Google about this.

BTW, there might be a misunderstanding here. What I have in mind is that a sender would offer a stream indicating how many coefficient partitions that stream uses, along with the frame size and rate (or whatever alternative such as pixel rate or macroblock rate the group decides on). A receiver would be able to decide based on this information whether or not it can decode the stream. The receiver might need at least a certain number of coefficient partitions to be able to decode a stream with a certain frame size & rate. 

Note 1: This does not apply to all receivers, but it might apply to some. 

Note 2: If the stream has more coefficient partitions than the receiver needs, that's not a problem, but if it has too few, that could potentially be a problem. 

In this scenario it doesn't matter whether the receiver's capability are static or dynamic; all that matters is that the receiver knows what its capabilities are at the time it decides whether or not it can decode the stream. 

If the concern is that the receiver's capability might be reduced after it starts to decode the stream, that opens a bigger issue that I was not trying to address. However, I don't think that concern applies to the specific question of how many coefficient partitions the stream has. If the receiver either requires a certain number of coefficient partitions initially, or anticipates that due to dynamic behavior it might require them in the future, it can use that information to decide whether or not to accept the stream. On the other hand, if the receiver might in the future require a smaller frame size & rate stream after the stream has started, that's another matter altogether.

--Sandy (Alexander) MacInnis
Broadcom


-----Original Message-----
From: [e-mail] [mailto:[e-mail]] On Behalf Of Olivier Crête
Sent: Tuesday, April 24, 2012 2:41 PM
To: [e-mail]
Subject: Re: [payload] VP8 payload, decoder processing capabilities

On Tue, 2012-04-24 at 21:31 +0000, Sandy (Alexander) MacInnis wrote:
> I have another question about this.
> 
> Background:
> In the VP8 bitstream spec, and in the payload draft, the use of multiple coefficient partitions is optional and not required. 
> 
> A VP8 decoder might use multi-threaded processing, such that it can achieve significantly greater throughput by decoding multiple coefficient partitions in parallel. Such a decoder might rely on the presence of a certain number of coefficient partitions in order to achieve a certain level of throughput.
> 
> For example, some receiver might comprise a multi-threaded decoder that can decode 1920x1080p60 if the stream has at least four coefficient partitions, but if the stream has only one coefficient partition it might be limited to a significantly smaller product of frame size and frame rate.
> 
> Question: With the current draft, how does a multi-threaded decoder know in advance how many coefficient partitions the stream uses, so it can decide whether it can decode a stream of a given frame size and frame rate? Or does this aspect not matter?

I think the receiver should include all of that in its SDP, maybe
something like: SamplesPerCoefficientPartition=X
MaxParallelCoeffficientPartitions=Y

That said, I'm concerned that all of this is a really static view of
things, which is useful on a fixed device. But is completely pointless
when writing software that runs on a general purpose computer (such as a
desktop or a smartphone), in which case we want something more dynamic.
So maybe all of this discussion is mostly pointless and we should push
for a more dynamic solution (maybe some kind of RTCP feedback?)

-- 
Olivier Crête
[e-mail]