Re: [video-codec] Lessons Learned from Audio Codec Group - Normative Text not Code

["Timothy B. Terriberry" <[e-mail]>]

Gregory Maxwell wrote:
>> I prefer a bit-exact decoder operation spec
>> over the performance based decoder spec we had in Opus.
>
> For video there is no question in my mind for that. Even ignoring the norms and the conformance testing simplifications, the signal characteristics ('low' bit depth and low sampling frequency) in video mostly preclude auto-regressive prediction as we used in Opus and benefit significantly from unity-gain prediction (which requires bit accuracy).

More to the point, we were using bit-exact decoder definitions in 
VP3/Theora back when MPEG 1, MPEG 2, and MPEG 4 part 2 were still using 
transforms specified only up to a set of tolerances with metrics 
designed to avoid drift (and macroblock refresh requirements in the 
encoder) [1]. I consider the latter approach to be moderately crazy, 
even without the rapid accumulation of error that would be produced in a 
single frame by intra prediction (which is why H.264 was forced to go 
the bit-exact route), but that is just my personal opinion.

> But if test implementations are separate from the draft then I don't really
> see that it merits any real discussion. People can create whatever they

Even if the code is non-normative, that doesn't mean we can't include it 
in the draft. Some in the IETF seemed to strongly support the idea of 
providing reference code in the RFC [2], even if there were objections 
to making it normative.

As for speed, this is a trade-off, like anything else. There are 
definite benefits to having a fast implementation, but this requires 
more work than it does for audio. For Opus, the entire codec can almost 
fit in L1 cache, and the reference includes no SIMD assembly. For video, 
cache considerations are extremely important (and complicate the code), 
and SIMD assembly is a basic necessity for adequate speed on modern 
general purpose CPUs. It's also time-consuming to develop, and has 
compiler/toolchain dependency issues we did not have for Opus. Time 
spent working on those things is time that isn't being spent improving 
basic coding performance, testing reliability, or a host of other 
things. Particular individuals may be much more effective at some of 
those things than others, and slow code may make the other things more 
difficult, but I don't believe that the best approach is to have no 
attempt at optimization, nor is it to invest all the effort needed to 
make a commercial-grade optimized implementation. Though we will do the 
latter, and release it as open-source (because that is what we do), I'm 
not sure that needs to be gating for standardization.

[1] We were, of course, not the first to use bit-exact integer 
transforms. That dates back to at _least_ W.K. Pratt, J. Kane, and H.C. 
Andrews, "Hadamard transform image coding," in Proc. IEEE 57(1):58--68, 
Jan 1969.

[2] 
https://www.ietf.org/ibin/c5i?mid=6&rid=49&gid=0&k1=933&k2=64944&tid=1349448928