Re: [video-codec] Strategy for an RF video codec

[John Koleszar <[e-mail]>]

On Wed, Jan 16, 2013 at 9:50 AM, Timothy B. Terriberry <[e-mail]>wrote:

> 1) every separable part of the codec becomes a separate feature and
>> both sides negotiate the features they support. I'm imaging 100 or
>> more features many of which depend on others or you can't use them.
>> The minimal feature might just be uncompressed video. The odds of
>> managing for all parts of an advanced video codec to be RF are low
>> and this allows us to mitigate the risk by if some part if found to
>> not be RF, deployments can disable that feature and the features that
>> depend on them. This also makes for a much smother watt to
>> incrementally advance the codec over time. Doing this means we are
>> not doing a single codec but really a family of codecs .
>>
>
> So, two months ago I wasn't prepared to call this a bad idea. Having
> thought about it a lot more since then, I think I'm prepared to call it
> that now.
>
> 1. In reality you often can't "disable" a feature... you need to replace
> it with something. If you "disable" the entropy coder, you have to do
> something in its place. This complicates the design.  Now you have to
> design a bunch of fallbacks you wouldn't otherwise have needed.
>
> 2. It also complicates the IPR story. You need to prove the fallbacks are
> just as RF as the original features. Even if you just disable some function
> f, computing g(f(x)) might have been unpatented while computing g(x)
> directly is patented. Since mere combinations of features can be patented,
> more combinations means more risk.
>
> 3. I raised the testing issue elsewhere... and while it's easy to dismiss
> this and say we only test a few cases, this poses real security issues: how
> do you know disabling feature X, Y, and Z doesn't expose your
> implementation to a buffer overflow? I know you think codecs are so complex
> they can never be adequately tested anyways, but in my experience video
> moves enough data through the codec that you actually can test the vast
> majority of branches and combinations of things relatively easily with
> automated testing, and the pieces that remain are small enough that you can
> reason about them. The small number of security vulnerabilities found in
> the codecs I've worked on compared to the average in projects like, say,
> FFmpeg bears this out. But when you multiply that work by 2^100, then you
> have a problem.
>
> 4. The testing problem means this won't work interoperably. What will
> actually happen is that people will only test the streams that other
> parties send them, and then suddenly one day people will want to disable
> some flag that was in use before, and you'll find that a) the fallback is
> buggy, b) it simply isn't implemented, c) maybe it is implemented, but in
> unoptimized software instead of hardware, and now things are too slow to
> actually be useful, or d) any number of other problems. Even if we provide
> test vectors, and even if (through astounding diligence) every
> implementation actually tests against them, as you point out we can only
> cover a few cases, which likely won't be the ones we wind up actually
> interested in.
>
> 5. More likely, what will actually happen is that implementors will use
> the flags to simply avoid implementing whatever random features they deem
> unimportant for whatever reason. We already see this with things like H.264
> which have a set of well-defined profiles... and then a separate set of the
> things that actually work in real, deployed implementations. The sets are
> not at all the same, and this is what allows Google to post VP8 performance
> benchmarks that beat "the features of H.264 that actually have broad enough
> support that we can enable them in our YouTube encodes." I.e., there's a
> real, measurable cost here, not just a theoretical one.
>
> 6. People willing to act unscrupulously can easily intimidate others into
> disabling features even when they don't really need to. The reality is that
> most people, confronted with someone telling them that "feature X might be
> unsafe" will simply disable it instead of trying to find out if that's
> really true. For many the analysis itself would be more costly than any
> loss of compression (especially device manufacturers who don't even bear
> the network transmission costs). But those that do bear sufficient cost to
> justify a real analysis have to interoperate with those who don't. So you
> enter a race to the bottom.
>
> 7. By definition you don't know which features need flags and fallbacks.
> If you did, you would have just taken that feature out in the first place.
> So we're making decisions about which features justify all the costs of a
> flag (in design time, implementation time, analysis time, testing time,
> attack surface, increased risk of interoperability failure) without any
> knowledge of the benefits.
>
> So in sum, this strategy has significant up-front costs, and a low
> probability of actually solving any problems that arise.
>
> If you're willing to disable a bunch of features without regards to
> compression performance to get something that's RF, a much simpler strategy
> with a much higher probability of success is to just fall back to a much
> older codec. Like, say, Theora. Or if that's not safe enough for you, H.261.


I agree with Tim on all of the above.