Re: [AVTCORE] RTP circuit breakers practical experience feedback

[Colin Perkins <[e-mail]>]

Simon, 

Many thanks for implementing this, and reporting back to the group.

On 7 Aug 2015, at 15:29, Simon Perreault <[e-mail]> wrote:
> All,
> 
> We implemented draft-ietf-avtcore-rtp-circuit-breakers-10 and put it in
> production here at Jive. It is exposed to all kinds of traffic:
> intra-DC, inter-DC, SIP peer to SIP peer, and wild public Internet
> traffic. I obviously can't discuss traffic volume. Here are some lessons
> learned...
> 
> - The "media timeout" circuit breaker is unusable in practice because at
> least one popular B2BUA implementation we routinely talk to (not going
> to name names) lies in its RTCP reports. In one particular instance, it
> sends RTCP with non-increasing maximum sequence numbers even though it
> is receiving the media we send it. We have found no way to work around
> this in general apart from fingerprinting the implementation of the
> remote endpoint (which we are *not* considering, for obvious reasons).
> 
> - The "RTCP timeout" circuit breaker is unusable in practice because it
> is common to stop receiving RTCP when an RTP relay we are sending to
> changes its destination from an RTCP-sending receiver to a
> non-RTCP-sending receiver. There is no indication at the signalling
> level that this is happening because what happens beyond the RTP relay
> is invisible to us. We have found no way to work around this in general.

These are clearly bugs in the devices you're communicating with. I agree that if such devices are common in some environments, then an RTP circuit breaker that checks connectivity using RTCP will be difficult to deploy in those environments. I'm not sure there's much we can do about this, other than report the bugs and hope the broken devices eventually get fixed. I will make sure the circuit breaker draft notes the issue. 

> - The "congestion" circuit breaker has never triggered. I don't really
> know how to interpret this: am I supposed to rejoice or is the circuit
> breaker simply useless in practice?

It perhaps addresses the concern that the circuit breaker is overly sensitive... 

(If you have data you're able to share privately, I'd be interested in learning more about the performance you're seeing, and how the circuit breaker is behaving in your deployment.)

> In addition, the text describing the algorithm leaves much to the
> programmer's interpretation. I've discussed this with the authors and I
> expect a revision with tighter text.

We're working on this.

> - We have not implemented the "media usability" circuit breaker as we
> were unable to find a good practical criteria for usability. I'd be
> surprised if anyone else did.

I think it depends very much on the application. If I were implementing an interactive video conferencing tool, I might treat a series of consecutive RTCP reports showing an RTT in excess of a couple of seconds as not usable media, for example. It's a deliberately vague fallback: if what you're receiving is clearly unusable for your application, consider stopping sending.

Cheers,
Colin






-- 
Colin Perkins
https://csperkins.org/