Re: [video-codec] Charter issues from BoF

["Timothy B. Terriberry" <[e-mail]>]

Spencer Dawkins wrote:
> I'm not seeing a problem yet for this proposed working group, but I am
> remembering that "the Internet" covers a LOT of different points in the
> "speed/delay/reliability" space, from interconnected data centers to
> environments that are slightly better-connected than DTNRG networks :-)
> ... it would be great to be clear on what environments are being targeted.

This is good advice. My primary interest is public, consumer-level, 
best-effort networks (i.e., where people run browsers). I can try to add 
some language to the charter to that effect if no one objects to that 
being the focus.

> program. I'd love for a chartered videocodec working group to have a
> good plan for liaison relationships, and I say that without reference to
> any other IETF working group having, or not having, a good plan for
> liaison relationships :-)

I've been approached by a couple of people from the MPEG IVC effort who 
think that we can do better than liaison-style collaboration, but I'm 
not sure what the best approach is here, yet. I will try to encourage 
them to post their thoughts on the list.

> I'd suggest the proposed working group community discuss whether it's
> worth producing a video codec that is known to not be RF, early in the
> process.

There is some difficulty in deciding what "known to not be RF" means. 
Some people would take the existence of an IPR disclosure---any IPR 
disclosure---that does not make that IPR available under RF terms to be 
an indication that the work is not RF... and in the case that the 
disclosure is by the author of the technique in question, this is very 
compelling. Other people are willing to investigate the real risk 
associated with those disclosures, evaluate the effectiveness of 
design-arounds (which do not incur an obligation to update that 
disclosure), etc.

Experience with Opus suggests to me that this should be handled by WG 
consensus on a case-by-case basis, instead of with an iron-clad rule in 
the charter. This was discussed a fair bit at the bar BoF in Vancouver, 
and the "developed under the IPR rules of the IETF" language that's 
currently in the charter was strongly suggested to me as the best way to 
phrase this.

Speaking personally, I would not support publishing as an RFC any codec 
that I did not believe was RF.

> I'm not saying that I have an opinion about "one codec or many", I'm
> saying it would be helpful to understand whether one codec could meet
> the need for a codec that's "optimized for the Internet", before
> deciding whether to produce more than one codec.

I think it can. The biggest challenge comes in deciding whether or not 
to support coding tools that are not robust in the face of packet loss. 
This includes, for example, anything which makes decisions about what 
values to read from the bitstream based on the decoded, reconstructed 
pixels. In the presence of loss, making one of those decisions 
incorrectly would corrupt the rest of the frame.

To illustrate this, one of the items mentioned in the VP9 presentation 
included scoring candidate motion vectors by matching the 
(already-decoded) edge pixels from neighboring blocks to the reference 
frame at the same offset. I believe this is okay in its current form, 
because the codebook used to decode the motion vector residual has no 
dependency on the value of the MV being used as a predictor. But if we 
wanted to use a scheme that _did_ depend on that value, we'd have to 
give up the ability to look at those pixels in neighboring blocks. I've 
had in mind for a while now a coding scheme which allows you to code a 
MV relative to two different predictors, but unlike the normal approach 
of "code which predictor is closer, and then code an offset relative to 
that predictor," doesn't introduce any redundancy in the coding. This 
depends on exactly what those two predictors are (consider the case 
where they're the same, for example), so it wouldn't be compatible with 
this VP9 technique for deriving the list of predictors.

But even in this case, we could pick which of the two approaches works 
better on average, or even allow both techniques to be used, and switch 
between them. Video frames have enough bits to make this kind of 
switching at the frame or even smaller levels quite practical, and you 
could pick up quite a lot of alternate algorithms like this before it 
made sense to give up all the things that _are_ common in the two use 
cases and have two completely separate codecs.

> If I'm understanding what other people are saying, I'm not hearing "need
> old technology where patents have expired" as a strategy for video
> codecs, but It might be helpful for the proposed working group community
> to be clear on how much new technology people are planning to define,
> early on.

I think we should be doing both things. If you look carefully at some of 
the drafts that have already been submitted, you'll see some of the 
references are _very_ old. There's a lot of research out there that's 
been around for a long time, but never made it into a widely deployed 
codec. Sometimes that's for very good reasons. Sometimes it's because 
design decisions which were important at the time have become less so as 
Moore's law has changed the best operating point on the chip area/clock 
speed/power usage frontier. Sometimes it's because the technique 
introduced practical problems which are solvable, but no one ever 
attempted to solve them.

We (Mozilla) are also exploring ways we can actually publish solid 
information about why we think the the codec will be royalty-free (once 
it is finished). As you may know, it is normally not in our interest to 
do so (it is basically mapping out your defense, or at least part of it, 
to any potential litigants), but it may wind up being worth the trade-off.

> I hope this is helpful. And again, I think the important thing is that
> the proposed working group community take Cullen's suggestions
> seriously, more than what the community decides about them.

I think this (and Cullen's feedback generally) has been very helpful. I 
hope to submit an updated version of the charter incorporating a bunch 
of the feedback received thus far before next week.