Re: [video-codec] Lessons Learned from Audio Codec Group - Normative Text not Code

[Jean-Marc Valin <[e-mail]>]

On 12-10-05 12:23 AM, Stephan Wenger wrote:
> I also support normative text, covering bitstream syntax and decoder
> operation, especially since there is a volunteer for editorship (which can
> well be a full-time job).  I prefer a bit-exact decoder operation spec
> over the performance based decoder spec we had in Opus.

Well, considering that video predictors use a unity gain (i.e. they
don't just re-converge like audio predictors), then "bit-exact" and
"matches closely like Opus" are essentially equivalent. If you're not
bit exact for video, you're going to diverge badly after a while. So no
objection to stating "bit exact" explicitly.

> A) final spec text is the single normative reference, and defines
> bitstream syntax and bit-exact decoder operation.  Not the encoder, and no
> wiggle-room for performance tunes of the decoder.  (The latter is
> different wrt. Opus, where conformance is considered achieved when certain
> performance hurdles are passed.)

Generally agree, though just like Opus I'd advocate for leaving things
like packet loss concealment out of the normatively specified part.

> This is different from Opus in the sense that the Opus implementation of
> encoder and decoder try to be reasonably well optimized for real-world
> use, at least on PC architectures.  Video codec test models, at least the
> encoders, are typically several orders of magnitudes slower than required
> for real-time operation (on the machines available at the time the
> standard is being developed).  Quite often that translates into several
> order of magnitudes slower than possible on a state-of-the-art machine.

Any particular issue with having a reasonably fast reference
implementation. Seems like it's generally a good thing, no? If only to
make sure the codec isn't too complex.

> F) As the draft standard matures, quite often certain aspects of the draft
> standard documented in the text are not fully implemented in the software;
> certainly not the encoder but sometimes also not in the decoder.  For
> example, certain exotic coding tools may not make it into the encoder (and
> the decoder code exercising this syntax may be relatively untested).  Even
> without optional tools, it is unrealistic to assume that all allowed
> syntax combinations will be exercised even by a well-designed software
> package; and statistical analysis through artificial test vectors (like
> done in Opus) is, in video coding, mush less likely to exercise all
> options than it is in audio.

There may be exceptions (there were a handful for Opus), but I tend to
think that it's good to be able to test that tools are actually useful
before adding them to a standard. If anything, I suspect we might
sometimes be in a case where it's the text that haven't caught up with
the reference implementation, no?

	Jean-Marc

> Stephan
> 
> 
> 
> On 10.4.2012 20:23 , "Jean-Marc Valin" <[e-mail]> wrote:
> 
>> I think "code as spec" worked reasonably well for Opus. That being said,
>> considering my experience with the IETF (e.g. when it comes to review as
>> Cullen pointed out) and the fact that video codecs tend to be defined
>> with normative text (which Tim volunteers to write!), I fully support
>> "text is the spec" for this video effort. Of course, I also support
>> having a good reference implementation to go with the text.
>>
>> 	Jean-Marc
>>
>>
>> On 12-10-04 11:01 PM, Timothy B. Terriberry wrote:
>>> Bernard Aboba wrote:
>>>> So having both a spec and code is important -- but for heavens sake,
>>>> let's not conflate them!
>>>
>>> +1. There are a number of successful audio codec standards where "the
>>> code is the spec", but the video codec culture is very different. I
>>> think to be taken seriously, we will need a written spec, and that's the
>>> approach I personally prefer. I knew if I objected too strenuously back
>>> when the codec WG was getting underway that I would be volunteering to
>>> be the one writing that spec. I'm explicitly volunteering to do that
>>> work (with help, hopefully!) for this effort.
>>> _______________________________________________
>>> video-codec mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/video-codec
>>>
>> _______________________________________________
>> video-codec mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/video-codec
>>
> 
> 
> _______________________________________________
> video-codec mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/video-codec
>