[Cellar] Matroska Elements to support frame side data

[Dave Rice <[e-mail]>]

Hi all,

I’m looking for thoughts on pull request https://github.com/Matroska-Org/matroska-specification/pull/276 <https://github.com/Matroska-Org/matroska-specification/pull/276> for Matroska which addresses an issue described at https://github.com/Matroska-Org/matroska-specification/issues/270 <https://github.com/Matroska-Org/matroska-specification/issues/270>. This would add a new Master Element called “Metadata” within BlockGroup and the Metadata Element would contain a MetadataName which labels the content of either a MetadataString or MetadataBinary Element (this is a bit similar to the TagName/TagString/TagBinary set of elements). These elements would allow information to be associated directly with the contents of the Block.

I also considered Michael Niedermayer’s work in the nut specification at http://ffmpeg.org/~michael/nut.txt <http://ffmpeg.org/~michael/nut.txt> where a structure is defined to store side metadata per frame, see the section on "sm_data / side_data / meta_data”.

Here are a few use cases where I think this structure could be helpful:

=== timecode
We’ve made many starts and stops at defining how to store timecode in Matroska but haven’t found consensus or resolution on those attempts. With the Metadata Element I think a timecode value could be stored such as

Cluster/
	Timestamp=01:00:00.000
	BlockGroup/
		Block/
			...
		Metadata/
			MetadataName=“timecode"
			MetadataString="01:23:45;12”

The Metadata Element would here would be encoded as 0xD297D38874696D65636F6465D48B30313A32333A34353B3132 which adds 25 bytes per frame (about 2MB per hour for PAL).

Currently the decklink input device in libavdevice can pass along timecode as frame side metadata (see https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/0946c0ec177dc48ef0677f890aa42d95e667c417 <https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/0946c0ec177dc48ef0677f890aa42d95e667c417>) but Matroska offers no method to store such side data.

=== DPX -> MKV -> DPX lossless remuxing
When converting a DPX image sequence to FFV1/MKV the image data is losslessly stored but the DPX headers contain lots of information specific to DPX which is lost. The RawCooked project has an approach where that DPX header data is stored in Matroska attachments and then recalled and used when RawCooked is reverting from Matroska back to DPX (so the resulting DPX bitstream is identical as the input, similar to how flac can do this with wav->flac->wav. With this proposal the DPX header data could be stored within the Metadata Elements of the block group, such as:

Cluster/
	Timestamp=01:00:00.000
	BlockGroup/
		Block/
			...
		Metadata/
			MetadataName=“dpx_header"
			MetadataBinary=0x58504453…

Comments welcome.
Dave Rice