Re: [Cellar] CodecIDs for non-standard and unregistered data formats

[Steve Lhomme <[e-mail]>]

2017-02-27 4:04 GMT+01:00 Matt Gruenke <[e-mail]>:
> Let's take this example of a camera having both a visible light sensor and a
> depth sensor, as well as a stereo microphone.  For various reasons, its
> depth sensor captures frames at different times and a different rate than
> its color image sensor.  I'd like to record a video file containing an audio
> track, a normal video track, and a depth track.  I'd like the audio and
> video portions to be playable in normal video player apps.  When using
> enhanced apps, the depth data may also be utilized.

It seems that the depth data cannot be used on their own and would
need to be coupled with the video frames. So that would probably go as
a BlockAddition data (or something similar as it's currently tied to
the Track codec). Even if they are repeated for each video frame which
the sensor doesn't change the value.

> Today, most players I've tried will behave as I've described, if I simply
> write the depth data using an unspecified TrackType and having a custom
> CodecID.  However, not all players handle it gracefully (or at all), and
> there's no guarantee that the others will continue to work.
>
> Having a dedicated TrackType (or range, thereof) would solve one potential
> problem.  Perhaps the Ancillary TrackType already meets this need.
>
> Using nonstandard CodecIDs is another concern.  I thought about schemes for
> doing this, but none seemed as sensible as trying to build upon what IANA
> already has.  The private & vendor-specific trees provide a mechanism by
> which there's at least some record of a given format's origin, which is
> clearly better than it being completely ad hoc.  The only downside I can see
> is that two apps might start using the same media type, but with different
> mappings to Matroska's syntactic structures (i.e. CodecPrivate and
> CodecState).  Hopefully, any encoding common enough to have independent
> implementations by multiple apps would've already been promoted to have an
> official Matroska CodecID, whereupon such implementation details can be
> specified.

It seems to me it should not be a different track, so I'll use this
hypothesis. We may need to add more elements to the Track to define
what each BlockAdditionID is, maybe using a MIME type or something
similar. Maybe a type like depth data or extra lossless complement
(for the only case I know of using BlockAddition: lossy part in the
regular Block and lossless complement in the BlockAddition).

>> Although tempting I think it's better to avoid interleaving all kinds of
>> stuff with audio/video.
>
> It's a valid and understandable position, but what attracted me to the
> Matroska format was my understanding that it's an open and extensible
> format.  If it's too resistant to innovation, then developers might seek
> alternatives.  This is how file formats get superseded and fall into disuse.
> Therefore, I think it's important for Matroska to offer extension mechanisms
> that strike the right balance between flexibility and compatibility.

You're right, it's extensible. But it would be wrong if people extend
it in a bogus way. For example non-interleaved data shouldn't go in
the interleaved parts of the format. There are already all kinds of
lesser known mechanism to extend Matroska in a clean way. We must
emphasis on that.

>
> Matt
>
>
> On Sun, Feb 26, 2017 at 11:29 AM, Steve Lhomme <[e-mail]> wrote:
>>
>> 2017-02-14 7:46 GMT+01:00 Matt Gruenke <[e-mail]>:
>> > Related to my recent query about TrackTypes for non-standard data
>> > formats,
>> > I'd like to discuss CodecIDs.  I had previously raised this issue, on
>> > the
>> > matroska-devel list, and was advised to take it up, here.  In the
>> > interim,
>> > my thinking on the matter has evolved.
>> >
>> > I wonder why not allow IANA Media Types (aka MIME types), as a sub-class
>> > of
>> > CodecIDs?  Of course, the official Matroska CodecID would be preferred,
>> > for
>> > all formats for which one has been designated.  However, this would
>> > enable
>> > broader use of Matroska files, including for private,
>> > application-specific
>> > data stream formats.
>>
>> The main goal of Matroska is multimedia interleaved/timestamped data.
>> It's possible to have "complex" track and put in it whatever you want.
>> If a timestamp is required. If it's not, using attachment is what you
>> want and it already has a MIME type.
>>
>> > Perhaps a prefix like 'X_MEDIATYPE/' could simply be prepended to media
>> > type
>> > names conforming with RFC 6838.  The value of TrackType would still
>> > determine whether video, audio, or any other type-specific elements can
>> > be
>> > used in such tracks.
>>
>> Although tempting I think it's better to avoid interleaving all kinds
>> of stuff with audio/video. You could want to embed a "video/quicktime"
>> but it would not be interleaved, or you'd need to parse/mux the data
>> and then it's not a clean "video/quicktime" you get anymore.
>>
>> > Thank you for considering my suggestion.
>> >
>> >
>> > Matt
>> >
>> >
>> > _______________________________________________
>> > Cellar mailing list
>> > [e-mail]
>> > https://www.ietf.org/mailman/listinfo/cellar
>> >
>>
>>
>>
>> --
>> Steve Lhomme
>> Matroska association Chairman
>
>



-- 
Steve Lhomme
Matroska association Chairman