Re: [rtcweb] RTP Usage: Is RTP Retransmission REQUIRED or RECOMMENDED

[Randell Jesup <[e-mail]>]

On 6/29/2012 3:30 AM, Stefan Holmer wrote:
> Doesn't this in the end boil down to what tools we have for baseline
> resilience? Retransmissions are fairly easy to implement and will in
> most cases give a much better experience than for example relying on
> periodic key frames or doing key frame requests. Also it isn't necessary
> for the decoder to support decoding streams with lost packets.

"Better experience" incorporates a bunch of assumptions.

Assuming we're talking video only:

If you want it to be seamless, you have to be running a jitter buffer 
depth of at least 1 RTT plus time-to-decide-packet-may-be-lost plus 
random-constant.  The second part of that can be tricky and 
network-state dependent.  On corporate networks, I'd see sudden 100ms 
changes in base delay over 1-3 packets; a short time-to-decide would 
flag each of those as missing.  This is ok (the re-xmit will get ignored 
as a dup), but adds packets at an apparent bad point for the network.

That said, small RTTs certainly happen within LANs and corporations, and 
between neighbors in a single provider often.

Note that often/normally with small RTTs you also get reasonably small 
jitter, and thus the adaptive jitter buffer might run very small 
numbers, so one might need to put a lower limit on the jitter buffer in 
these cases.

The alternatives:

1) request re-xmit and freeze video queue when the jitter buffer runs 
dry.  If the packet comes in ASAP, freeze <= RTT+constant (modulo 
application scheduling delay at the sender side).  Then you can repair 
the broken frame, decode it, and then decode any following frames. 
Typically after a freeze you drop a frame or more; you might end up 
showing some of them depending on decode speed and how long the freeze 
lasted.  If the re-xmit (or the request) is lost, then delays can 
stretch much longer, since you need to wait RTT+bigger constant before 
requesting again.

In some conference apps even if the mixer just forwards video packets, 
the RTT is receiver<->mixer, not receiver<->mixer<->sender (this 
requires the mixer to buffer packets for possible retransmission).

2) request IDR (or refresh of corrupted slice).  Downside here is that 
you need to wait for the next frame encode at the sender side (so 0-33ms 
normally, or perhaps 0-100ms), and then the frame typically takes many 
packets to send (which also may be lost, and take time to receive 
especially on low-BW links), and IDR initial quality may be noticeably 
lower.  The receiver may freeze the video until the IDR is received or 
keep decoding with errors.  If there's a significant chance of the IDR 
losing a packet, a freeze could be substantial time.

3) request repair.  Note that #2 and #3 may be the same at the receiver 
side, with the sender deciding the best available repair.  In this case, 
the sender would instead of an IDR use a some other set of (smaller) 
packets to create an error-free up-to-date state at the receiver, such 
as encoding using a long-term-reference-frame, or using some other 
known-error-free frame in the receiver.  Note that repair is basically 
just a normal p-frame, albeit with somewhat lower quality and/or 
somewhat higher bandwidth used.  A big plus is that the end state is 
close to the same as re-transmit, but you don't have to speed through 
decoding any skipped frames.  Repair also allows the receiver to use 
alternate recovery mechanisms than simply freezing, including simply 
continuing to decode p-frames ignoring the lost frame.  This induces 
artifacts, but keeps motion loss to a minimum, especially in longer-RTT 
links.

Note that #3 in particular should result in similar freeze length as #1, 
perhaps 1 frame longer, if it freezes.  If it plays with errors instead, 
the freeze is typically 1 frame (the lost one).

(Others?)

My normal preference is #3. But, as discussed above, in low-RTT networks 
it may be a good option and might give 0-frames of freeze.  But, you 
still have all the complexities about how to decide when and how to use 
re-xmit; I think the 1 frame freeze is a reasonable option and a 
reasonable fallback if the source doesn't re-transmit.

Also, if retransmit is REQUIRED, and the media is gatewayed to other 
sources (non-webrtc), and they don't support retransmit (likely), then 
the gateway would need to buffer and act as a retransmit agent 
(complicating the gateway).  If it's RECOMMENDED, this is not a big deal 
and the gateway can stay simpler.
-- 
Randell Jesup
[e-mail]