[rtcweb] Transcoding Delay

[John Leslie <[e-mail]>]

Stephan Wenger <[e-mail]> wrote:
> 
> As for the delay introduced by the transcoding engine alone, let me note
> that the single or two frame delay commonly attributed to transcoding is
> realistic only for straightforward PPP type coding.

   I agree that seems optimistic...

> My understanding is that VP8, in its default encoder settings, uses
> much more complex GOP structures for error resilience and coding
> efficiency reasons.  H.264 constrained baseline certainly also allows
> that (it?s even more flexible in that regard). Complex GOP structures
> are anywhere between helpful to essential (depending on your viewpoint)
> for error resilience.

   I would hope that error resilience could become orthogonal to
transcoding -- though I agree it doesn't seem to be there yet.

> Complex GOP structures do not necessarily increase the latency in an
> overall system design in the absence of errors (they are not based on
> MPEG-2 B frame type of thinking), but when error correction becomes
> necessary, a transcoder would need to emulate receiver behavior, and
> that certainly would induce delay of several hundred milliseconds
> which would not be observable in a transcoder-free operation (under
> the assumption that the end system chooses to display slightly
> distorted video) but will be observable and unavoidable in the
> transcoder-based design.

   The mistake here is optimizing for resilience assuming the same
coding/decoding rules when, in fact, the endpoints don't share the
same rules. Yes, that very much gets in the way of minimal latency.

> I?m very familiar with H.264 baseline, and somewhat familiar with the VP8
> syntax.  Based on this knowledge, I doubt that one can transcode between
> the two formats in the compressed domain, i.e. without full reconstructed
> to the sample level.  I am absolutely convinced that compressed domain
> transcoding in the direction from H.264 to VP8 is impossible, due to the
> larger feature set of H.264.  ?Impossible? means here that I could create
> an H.264 compliant bitstream that cannot be transcoded into VP8 in the
> compressed domain, because there is no syntax equivalent for an H.264 tool
> in VP8.

   I agree. (And this is worth re-reading several times for anyone
participating in discussion of transcoding.)

> For example, H.264 constrained baseline supports more than three
> reference pictures, whereas VP8 uses no more than three (last frame,
> golden frame, and alternate reference frame).  It wouldn?t take me long to
> extend this list to several pages.  I?m also fairly certain that people
> more familiar with the VP8 syntax could similarly identify VP8 features
> that have no direct counterpart in H.264.  Insofar, I take any statement
> about ?availability? of compressed domain transcoding between the two
> coding schemes with a large grain of salt.  It may be possible (I don?t
> know) to transcode in the compressed domain between H.264/VP8 bitstreams
> when the respective input bitstream is specifically tailored for that
> purpose, and perhaps that is what people have in mind.

   It would also be possible to declare transcoding failure in cases like
these -- that's a tradeoff between precision and latency which should
be available at application layer.

   But, IMHO, it's probably better to disable error-resilience features
that poison transcoding when we know transcoding will be used.

   YMMV, of course...

--
John Leslie <[e-mail]>