Re: [Cellar] AV1 mapping Matroska

[Steve Lhomme <[e-mail]>]

2018-07-02 13:14 GMT+02:00 Jerome Martinez <[e-mail]>:
> On 02/07/2018 12:51, Steve Lhomme wrote:
>>
>> 2018-07-02 12:34 GMT+02:00 Michael Niedermayer <[e-mail]>:
>>>
>>> Hi
>>>
>>> On Mon, Jul 02, 2018 at 09:47:45AM +0200, Moritz Bunkus wrote:
>>>>
>>>> Hey,
>>>>
>>>>> Looking at the term "coded video sequence" (CVS) in other codecs (H.264
>>>>> and H.265) it seems it's a common term. And for those codec one Segment
>>>>> correspond to one CVS, with the parameters of that CVS stored in the
>>>>> CodecPrivate (SPS + PPS for H.264 for example). So we should probably
>>>>> go
>>>>> in that simple way.
>>>>
>>>> I have quite a lot of h.264 samples here, mainly M2TS from DVB, where
>>>> SPS/PPS change mid-stream, often multiple times. For such files the
>>>> first
>>>> occurrences of SPS/PPS make up the AvcC in CodecPrivate, but all key
>>>> frames
>>>> are still prefixed with the currently active SPS/PPS — because that's
>>>> the
>>>> only way to signal that stuff has actually changed.
>>>
>>> IMHO All SPS/PPS should be in the "global header" (CodecPrivate) which
>>> the
>>> global header applies to, not just the first.
>
>
> Not possible with e.g. real time streaming (you don't know in advance the
> next SPS/PPS)
>
>>> This way an application can know all resolutions, aspect ratios and so on
>>> from just the header without the need to scan through all random access
>>> points.
>>
>> My understanding is that it's how it's done in MP4:
>>
>> "Each AVC sample entry, which contains the AVC video stream decoder
>> specific information, includes a group of SPSs and PPSs. This group of
>> parameter sets functions much like a codebook. Each parameter set has
>> an identifier, and each slice references the parameter set it was
>> coded against using the parameter set's identifier."
>>
>> I have to check but I don't think Sequence Headers OBUs have
>> identifiers like that in AV1.
>
>
> Not in AV1 (I just updated my own code for AV1  1.0.0 spec, still not
> there).
> But it actually does not answer the issue because in the example there is
> usually still only the "set id" of 0 (the new SPS replaces the old one, not
> a new number).
>
> Actually IIRC in HEVC (same for AVC) there are explicitly 2 different codec
> identifiers:
> - "hvc1" is for *all* SPS/PPS in the global header (CodecPrivate), and there
> is no SPS/PPS in the stream itself
> - "hev1" is for *no* SPS/PPS in the global header (CodecPrivate), and key
> frames are prefixed with the currently active SPS/PPS
> (I personally don't see the need of 2 different identifiers, as it quick to
> see if SPS/PPS is in the file, but there is surely a reason)
>
> The issue is not only for AV1, but for all formats permitting to have the
> init in the stream itself (AVC, HEVC, AV1...).
>
> IMO we should not restrict to only one possibility in Matroska spec, as they
> are for different needs (non real time vs real time), whatever is the format
> (AVC, HEVC, AV1...).
>
>
>
>> By the way, this is one mapping for V_AV1. There could be other AV1
>> mapping in Matroska where there is no CodecPrivate and there could be
>> multiple coded video sequences in it. But it would require the use of
>> CodecState and CueCodecState which is currently not an option for
>> WebM. So for now I will concentrate on a single coded video sequence.
>> And if needed in the future we can do a different one.
>
>
> If WebM decides that real time AV1 is for them, they may decide to discard
> CodecPrivate usage and put sequence header in the "Block" (as they currently
> do with up to date AOM build)
> So restriction will not be there in practice.

Seeking in such a file would not be possible. I'm not sure it's
desirable even when doing adaptive streaming with short sequences. We
will need CodecState and CueCodecState for that. We could define such
a case (with a different codec ID) for Matroska and WebM could pick
that from there.

Also while working on the case where the Sequence Header OBU doesn't
change, I noticed an issue. The Sequence Header OBU can still change
the [operating_parameters_info] during the lifetime of the stream
(paragraph 7.5 pointed out by Tim). That means even applying the
stricter restrictions of the Codec Video Sequence in AV1 means we'd
still need CodecState and CueCodecState for seeking to work. This
change is only possible in a specific decoder model (Annex E) when
[decoder_model_info_present_flag] is 1. When it's 0 then the
[operating_parameters_info] doesn't apply. So we should probably
restrict the single Sequence Header OBU to streams where
[decoder_model_info_present_flag] is 0.

> Jérôme
>
>
> _______________________________________________
> Cellar mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/cellar



-- 
Steve Lhomme
Matroska association Chairman