[Cellar] Ancillary data in Matroska

[Jerome Martinez <[e-mail]>]

We need additional codec to be supported in Matroska in order to be able 
to transwrap in a lossless manner content from older containers to 
Matroska, especially time codes (all the codecs defined in the patch can 
contain 1 or more time codes)
The global idea is to be able to transwrap e.g. MOV (from Apple), MXF 
(from SMPTE), GXF (from Grass Valley, also known as SMPTE ST 360 and 
SMPTE RDD 14, found in some broadcaster archives), LXF (from 
Leitch/Harris, found in some broadcaster archives) without losing 
metadata in some sidecar non visible streams.

After agreeing on the format, we will have to update tools e.g. FFmpeg 
in order to support such lossless transwrap.


I added an "Ancillary" section with several mappings in the attached 
patch, please comment.


Comments:

The frequent remark is "Why do you need so many time code formats, 
choose one and stop using the others": in the real world, people 
consider that one format is the best one, but not the same format as 
others, and they use only this time code for their work and don't plan 
to switch. You can have 10+ time codes in a file and each one is 
important for someone, as well as the sidecar information transported at 
the same time (never same between each format). There is no consensus 
about that for years, and I don't think we could achieve a consensus 
from the different actors (lot of them are not interested by our work at 
IETF) so the idea here is to be able to move from one container to 
another one without losing any metadata, absolutely not deciding about a 
reference time code (and other metadata) format, as Matroska does for 
e.g. video (Matroska does not say that VP9 should be used instead of 
AVC, it supports both).

"N_" prefix arbitrary used (2nd letter of "Ancillary" because first 
letter is already used)

N_QUICKTIME: is a pure copy of V_QUICKTIME and A_QUICKTIME already 
supported in Matroska, N_ would be used which would be used for e.g. 
tmcd codec (QuickTime time code). Exactly same principles as video and 
audio. Note that classic usage is to store only the first frame content 
and other content is computed from it, so a player would have to read 
the first frame even if there is a direct seek request to another place; 
this could be avoided with some "hack" in the track header, could be the 
next step after this one is accepted.

VBI and ANC: used to transport time codes (LTC, VITC, ATC...), Recording 
Information, bar and pan/scan data, captions (North American CEA-608, 
CEA-708, European and Australian WSS/Teletext, Japanese ARIB B37...), 
camera acquisition dynamic metadata, Audio Metadata, Film Transfer and 
Video Production Information (in theory, I never saw it)...
The "perfect" solution would be to decode VBI and ANC, and put each 
format in its own specification, but it would be an huge task and we 
still would have to store opaque content (content is present in the 
VBI/ANC but we ignore all from it right now), so it is better to start 
by the beginning and we consider all opaque without trying to decode the 
data, as do other containers.

N_STRIPPEDTIMECODE: the Cluster part will contain no data, but despite 
of that it must be a complete standalone track, with Track Id ans so on, 
as it is a standalone track in the source container.

N_SDTICP: nearly 1:1 copy of SMPTE ST 385 elements. In order to have all 
data and being able to have only 1 Matroska block for all STDI-CP 
elements, I kept also the 2 last bytes of the KLV "Key" and the KLV 
"Length" in addition of the KLV "Value". Note that something weird, this 
is a standalone track in source container but the difference with other 
tracks (including ANC or VBI) is that this stream has no track header so 
no track number, and Matroska specs makes TrackNumber mandatory, I 
wonder what should be used during transwrap (fake number?)

TODO: support of GXF non stripped time codes


Jérôme