Re: [Cellar] CodecIDs for non-standard and unregistered data formats

[Matt Gruenke <[e-mail]>]

Let's take this example of a camera having both a visible light sensor and
a depth sensor, as well as a stereo microphone.  For various reasons, its
depth sensor captures frames at different times and a different rate than
its color image sensor.  I'd like to record a video file containing an
audio track, a normal video track, and a depth track.  I'd like the audio
and video portions to be playable in normal video player apps.  When using
enhanced apps, the depth data may also be utilized.

Today, most players I've tried will behave as I've described, if I simply
write the depth data using an unspecified TrackType and having a custom
CodecID.  However, not all players handle it gracefully (or at all), and
there's no guarantee that the others will continue to work.

Having a dedicated TrackType (or range, thereof) would solve one potential
problem.  Perhaps the Ancillary TrackType already meets this need.

Using nonstandard CodecIDs is another concern.  I thought about schemes for
doing this, but none seemed as sensible as trying to build upon what IANA
already has.  The private & vendor-specific trees provide a mechanism by
which there's *at least* some record of a given format's origin, which is
clearly better than it being completely ad hoc.  The only downside I can
see is that two apps might start using the same media type, but with
different mappings to Matroska's syntactic structures (i.e. CodecPrivate
and CodecState).  Hopefully, any encoding common enough to have independent
implementations by multiple apps would've already been promoted to have an
official Matroska CodecID, whereupon such implementation details can be
specified.


> Although tempting I think it's better to avoid interleaving all kinds of
stuff with audio/video.

It's a valid and understandable position, but what attracted me to the
Matroska format was my understanding that it's an open and extensible
format.  If it's too resistant to innovation, then developers might seek
alternatives.  This is how file formats get superseded and fall into
disuse.  Therefore, I think it's important for Matroska to offer extension
mechanisms that strike the right balance between flexibility and
compatibility.


Matt


On Sun, Feb 26, 2017 at 11:29 AM, Steve Lhomme <[e-mail]> wrote:

> 2017-02-14 7:46 GMT+01:00 Matt Gruenke <[e-mail]>:
> > Related to my recent query about TrackTypes for non-standard data
> formats,
> > I'd like to discuss CodecIDs.  I had previously raised this issue, on the
> > matroska-devel list, and was advised to take it up, here.  In the
> interim,
> > my thinking on the matter has evolved.
> >
> > I wonder why not allow IANA Media Types (aka MIME types), as a sub-class
> of
> > CodecIDs?  Of course, the official Matroska CodecID would be preferred,
> for
> > all formats for which one has been designated.  However, this would
> enable
> > broader use of Matroska files, including for private,
> application-specific
> > data stream formats.
>
> The main goal of Matroska is multimedia interleaved/timestamped data.
> It's possible to have "complex" track and put in it whatever you want.
> If a timestamp is required. If it's not, using attachment is what you
> want and it already has a MIME type.
>
> > Perhaps a prefix like 'X_MEDIATYPE/' could simply be prepended to media
> type
> > names conforming with RFC 6838.  The value of TrackType would still
> > determine whether video, audio, or any other type-specific elements can
> be
> > used in such tracks.
>
> Although tempting I think it's better to avoid interleaving all kinds
> of stuff with audio/video. You could want to embed a "video/quicktime"
> but it would not be interleaved, or you'd need to parse/mux the data
> and then it's not a clean "video/quicktime" you get anymore.
>
> > Thank you for considering my suggestion.
> >
> >
> > Matt
> >
> >
> > _______________________________________________
> > Cellar mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/cellar
> >
>
>
>
> --
> Steve Lhomme
> Matroska association Chairman
>