Re: [codec] draft-ietf-codec-oggopus: attached pictures

[Ron <[e-mail]>]

On Sun, Aug 31, 2014 at 11:29:14PM -0700, Mark Harris wrote:
> Timothy B. Terriberry wrote:
> > There are two possibilities:
> >
> > 1) A proposal for adding binary metadata breaks already-encoded files, in
> > which case I don't think we should do it, considering people have been
> > producing those files for over two years now under the expectation that they
> > would continue to work, or
> >
> > 2) A proposal for adding binary metadata does not break already-encoded
> > files, in which case we have exactly the future extension mechanism you want
> > (emphasis on "future").
> 
> 
> None of my proposals should break already-encoded files, so that
> should not be an issue.  I think that the only potential issue is that
> insufficient guidance in the Ogg Opus draft may lead to newly encoded
> files being handled poorly by older tools that are not aware of later
> extensions.  It would be best for this document to provide additional
> guidance even if details such as anything specific to attached
> pictures is specified later in another document.

As a general rule I think we should be wary of carving speculative
things in stone here.  If there is something you can see in the
existing document that might unnecessarily make something impossible
later, then we should look at that - but I don't think we should be
staking claims on things we might never need or use.  That just
increases the risk of making some fatal mistake we'll regret later.


> For proposal (1) (tags named with an initial "@" are binary rather
> than UTF-8): even if no tags are specified in the document, the
> document should state that values of tags whose name begins with "@"
> need not conform to UTF-8, so that binary tags can be defined later
> without having conforming tools reject them as invalid or treating
> such tags poorly.  For the record, Firefox and Chrome do not seem to
> have any issue playing files with such tags.  Only tools that try to
> validate the UTF-8 or that assume that all tags are UTF-8 text should
> be impacted, and if they do things like display or allow editing of
> arbitrary tags they may want to provide a different way of doing that
> for binary tags.  Tools are likely to already provide different
> display or editing mechanisms for certain tags such as
> METADATA_BLOCK_PICTURE, but currently the only way they can do that is
> to check for specific tag names.  A naming convention allows tools to
> provide a somewhat more appropriate display and editing capability
> even for unknown future tags if they so choose.

I think we're 20 years too late to be reserving a single character
in a tag name with the intent of changing the rules for its content.
And perversely, the more 'improbable' the character you pick for
this seems, the more probable it becomes that someone else also
picked it for precisely that characteristic too.

If this does turn out to be the optimal solution, then it needs to
be done with a much longer prefix, that has a much better chance of
actually being unique.  In which case, we don't need to reserve it
now, since if it won't still be unique by the time someone has more
formally proposed a complete solution to this, then it was also a
bad choice and something different should be picked.

But given the other problems with this, which Tim already pointed out
some of, I suspect one of the alternative solutions is most likely to
trump this one once all things are considered.  In which case, we'd
have reserved something uselessly, which might come back to bite us
later with unintended consequences.

Either way, I don't think we need to rule this option out now, but
we also don't need to preemptively reserve things that its final
form may or may not use.


> For proposal (2) (new OpusTags section for binary blocks, separate
> from comments): even if the format of pictures and any other binary
> blocks are defined later in another document, this document should
> provide guidance for tools that wish to modify tags in an existing
> file or provide padding for other tools to do so without re-writing
> the whole file.  Currently no such guidance is provided, which could
> lead to interoperability failure between some tools which write files
> and other tools that modify existing files, if they have different
> assumptions about the data in the OpusTags packet following the
> comment table.  For example without any guidance different tools
> modifying tags in the file might assume that additional data in the
> packet should be retained, or that it may be overwritten with the
> unused portion left as-is, or may be overwritten with the unused
> portion zero'd.  This makes it difficult to ensure that any extension
> to this packet will be handled reasonably by any conforming tool.

I think this has much the same problem as above.  It we try to
preemptively reserve or define something now, that might actually
backfire and *prevent* us from implementing what could otherwise
turn out to be the optimal solution to this later.

It's too late to make or change any assumptions that people are
already making about how to handle these sections based on the
guidance for vorbis comments, and if you're worried about the
overhead of base64 encoding an image, then the idea of having
"padding" the size of the entire image should be horrifying.

Much like tags in TIFF or other similar formats, if you can't
recognise them, you can't possibly safely modify or copy them.
Your only choice is to not touch that space at all, or discard
anything you don't understand how to handle.

There are far too many perfectly reasonable ways to handle
modifying tags, and it's a fair bet that most of them are
already implemented by some software or another.  Fortunately
we don't really have to care about that, since none of them
effect the ability to decode the audio stream.  If someone
chooses to edit them, with a particular tool of their choice,
then they'll get what they chose to have, which hopefully they
chose because it's pretty close to what they wanted.

I can't see how we could possibly tell existing implementations
that people have been using for years "no no, you're doing it
wrong" with a straight face.  If we can't think of a way to
retrofit this that has a reasonable prospect of compatibility
with existing tools, without mandating new things in this spec,
then I think we'd have rule this option out.

Which is not to say out that we necessarily couldn't find a way
to make this work.  Just that if it requires us to define things
in *this* spec now, else it wouldn't work - then it probably
isn't going to be very workable anyway.  Not compared to the
other alternatives.


> For proposal (3) (use a different multiplexed stream): it might be
> nice if the document had at least a SHOULD-level suggestion that
> playback of an Ogg Opus stream not be impacted by the presence of
> unrecognized multiplexed streams.

But to end on a happy note, this one however, we already have :)

 'When Opus is concurrently multiplexed with other streams in an
  Ogg container, one SHOULD use one of the "audio/ogg", "video/ogg",
  or "application/ogg" mime-types, as defined in [RFC5334].'

Which gives us the best of both worlds.  People can create an
"Ogg Opus file", which is guaranteed to contain only sequential
streams in a form that even the most trivial decoder can handle,
or they can create an 'x/ogg' file (or stream), which requires a
more capable decoder, and for which RFC 5334 already says:

 "Furthermore, it is RECOMMENDED that implementations that identify
  a logical bitstream that they cannot decode SHOULD ignore it, while
  continuing to decode the ones they can.  Such precaution ensures
  backward and forward compatibility with existing and future data."


Which is at least partly why I think this is the early favourite
horse in this race.  The foundations for it to work were already
laid long ago.  We might be able to think of something clever
that makes one of the other option be a better choice, but if we
can't, this one is the safety net that means you can take the
Ogg Opus stream mapping already defined in this draft, and combine
it with our yet to be defined holo-pony mapping once that's defined
and lots of existing tools will only need to add support for the
latter to make the combination of them work, and will correctly
play the Opus stream until they do.

Any other option needs to at least get to that baseline of painless
to really be thought of as a serious alternative contender.

  Ron