[rmcat] RMCAT raw minutes

["Eggert, Lars" <[e-mail]>]

Hi,

below are the raw notes from our minute taker (thanks, Michael!)

If you spoke on the mike, please make sure your points were sufficiently well captured and send corrections/additions by mail.

Lars



RTP Media Congestion Avoidance Techniques (RMCAT) at IETF 86, Orlando
Monday March 11 2013, 9:00


Presentation:
Lars Eggert (LE), Administrativa & WG Status

Harald Alvestrand (HA): You said "add a year to milestones". I would hope that we could get adoption done within the first year.
LE: Yes, this was meant for finishing the documents, not adopting them.


Presentation:
Randell Jesup (RJ), Congestion Control Requirements For RMCAT
draft-jesup-rmcat-reqs-01 (milestone cc-requirements) 

Michael Ramalho (MR): I would challenge us to come up with a metric of fairness in terms of self-fairness and types of traffic that they're competing against, and have it independent of whether you're in the core or access network.

HA: I think we know how to detect blatant unfairness. We should say "here's the tests. if something fails one of these tests, it's not fair". I'm happy with a definition of fairness that says "we can't tell whether you're fair or not, but we can tell whether you passed these tests or not"

RJ: Agreed. Still comes down to a subjective evaluation, to check if on the whole .

LE: This list could get very long. It would be nice if we could find some principles up front. e.g., "you shouldn't start a new flow persistently". Then define cases for checking against those. The people who have solution candidates must have thought about this, so these people should make proposals. If we don't solve it now in the meeting we should see if we can make some progress later in ths meeting.

Toerless Eckert (TE): Let's agree on a priority list. It seems to be clear that total blatant unfairness is on the top of the list. Second, how self-friendly or unfriendly is existing real-time traffic already, just as a comparison metric. I would love to see what we can achieve not only over an Internet path with other traffic, but also if there was no other traffic (self-fairness).

LE: It's not only us competing against TCP. rtcweb has a data channel.
Matt Mathis: TCP fairness is a very deep rathole, guaranteed not to converge. I would suggest making a list of fairness criteria, evaluate based on simple principles. Preventing starvation, for instance, would be very important, and that is something you could define rigorously.

LE: We should start a design team on this.

Faisan Hasan (FH): Shall we go down to "friendliness" instead of fairness?

Dan Weber (DW): MOS scores as in audio quality or something like that might be good enough.

RJ: Problematic to evaluate.

LE: It's a bit harder, if you produce poor audio quality that's only your own problem, but a bad congestion controller harms everyone.

Colin Perkins (CP): "Compete reasonably with" - TCP. But we could decide initially that that's not a goal and only compete fairly with ourselves.

RJ: When competing with TCP, at some point you have to transition to loss based mode. So therefore it 

CP: Decision, what do we want to give up?

RJ: In the short term, we definitely want to compete with TCP. Longer term, either give up or transition.

??: One has to judge the importance of fairness over other characteristics. Reaction time, throughput loss, might be more problematic. Don't want to lose throughput because of fairness constraints.

LE: At some point we have to make a decision. If you want to actively help work on this stay here, we will try to form a design team.


Presentation:
Varun Singh (VS), Evaluating Congestion Control for Interactive Real-time Media
draft-singh-rmcat-cc-eval-02 (milestone eval-criteria) 

RJ: Discard rate = maximum beyond which we shouldn't resume and … if you achieved a low average delay … my real question is: is this relative to shortest delay or is it an absolute value?

VS: Absolute value.

MR: Should have some user related metric. like …  find out where the tail is, wait a reasonable amount of time for a packet to come in. What's a reasonable amount of time?

VS: Playout delay when you receive packets.

MR: small jitter buffer will consider well it's audio or video. Small jitter buffer doesn't use a number.

Zaheduzzaman Sarker (ZS): get rid of delay metric. 300 ms …   There will be some queuing in the bottlenecks. You have to look into this delay figure and compare with RTT etc …   CDF of queuing delay: you get an idea of values … I think we shouldn't have delay.

VS: Discard rate to check if trade-off make sense or not.

ZS: But then it shouldn't be a fixed value.

DW: You can have buffering somewhere, e.g. on the video card, so how does this fit in?

VS: The way I use it is that you get a feedback value from the receiver to calculate. It's only for evaluation case, not for deploying it.

ZS: We need a minimum set of guidelines, just those you really care about for real-time interactive communication.

VS: So is this list sufficient?

ZS: We have to discuss. On the mailing list I had 3 more suggestions to this one.

MR: when you say video frame rate etc., are you creating something artificial or are you using real video sources?

VS: That's an open question. Do we use a packet generation or actual video streams?

MR: Suggest to simplify to something we can characterize.

DW: Video is highly inconsistent. Using any form of constant bitrate creates serious issues in terms of evaluation. I-frame can blast 100kbytes / second.

VS: Should we remove this and say each app starts with different values?

DW: Pick five, use the same streams for all tests. We need to compress it before we even test this. Then we can focus on network related constraints.

VS: But with congestion control you change the rates on the fly. It's not a fixed stream. Depends on the encoder.

RJ: We'll have to add cases to this as we go along. But must have some understanding of edge cases. Single set of parameters will not help us determine those. Another issue, I would suggest to keep the number of variables down to something reasonable. Suggest: encoder can use all bits it has available, starts with a given rate. When congestion control gives feedback, controller changes rate (and sticks with it), with some defined delay based on the frame rate. Trying to characterize all the encoder behaviors will make too many problems. I-Frames etc. should be avoided anyway, I would not try to model that. Simple packet generator would be better.

HA: Not try to use real video streams for the tests we do when we develop things. But we need test scenario that can show that cc. is not ruining the video experience.

Gorry Fairhurst (GF): Maximum end-to-end delay of 300ms stands out, what is it? Application expectation? Network requirement? Is RMCAT going to break the circuit beyond 300ms?

VS: No it's not a circuit breaker. The packet would not be fed into the decoder, would be discarded among arrival.

GF: RMCAT might be used for various things… let's be clear to separate audio / video / transport part of it.

ZS: We're talking about the wrong things here now. We should focus on test scenarios, not values. Evaluation draft should take this approach, to test if guidelines have been followed, but fixed parameters won't work.

MR: I agree, we have to take out some uncertainty. Packet generator that should follow some rule of the cc. you want to test. TCP tracks bits in flight which is a round trip measure. It worked long, so our algorithm is more likely if it has a measure of capacity, measure of bits in flight. We need to have a mechanism that's stable, behaves well… let's not worry about video MOS score yet. Not start with a real video source, but packet generator fit to test a cc. mechanism.

Mo Zanaty (MZ): we're looking at it from different perspectives. Application perspective vs. network perspective. Now we're mixing all those, not good. In the requirements and evaluation, let's have a section for application-level objective metrics. Need to start putting down, what are the real needs of video applications - start rate, loss tolerance, … then network-centric flow behavior.

RJ: Good to separate network parameters, video parameters, application parameters. What are the parameters of the networks that connect the nodes for evaluation? Critical for determining the results of the algorithm. e.g. low-delay network, WiFi network … not a single set of points, a multi-dimensional surface of results.

Magnus Westerlund (MW1): Important to evaluate that algorithms are stable. Minimum path delay is important. Video is dynamic, this is the challenge here. For evaluation, let's get as many bits across as possible is also a possible evaluation for these cases.

Slide Evaluation Scenarios 1/3
MR: is this delay the propagation delay or the queuing delay? Queuing delay is rather important (buffer bloat).

Slide Evaluation Scenarios 2/3
ZS: each user will have different capacity. Where is the bottleneck? Capacity changes.

MM: single queue assumption or using different classes?

VS: single

MM: it should be stated for cases where throughput maximizing and delay-based mechanisms competing are guaranteed to fail.

MR: We could design a topology such that we're only competing against ourselves and stay in a delay-based mode. But maybe I'm wrong, I thought this group was about competing against other traffic too.

LE: assumption is correct. I think we want to have a set of isolated scenarios but also cover running on the Internet with other traffic.

MM: Have to ask: when you have to compete, and when you have separate queues, and when mode switching works.

LE: 20 hands for adopting. 0 hands against. We can call that rough consensus.

Evaluation criteria draft: 10 hands for adopting. 4 or 5 hands against. So not clear consensus.



Presentation:
Michael Welzl (MW2), Coupled congestion control for RTP media
draft-welzl-rmcat-coupled-cc-00 (milestone group-cc) 

*** FROM ANNA

LE: Fairness is not priorities (slide 5)

MW2: Proportional share

MW: How do you handle mix of RTP, SCTP flows etc. Can you handle all these cases.

MW2: Current version can handle 5-tuples, beyond that need shared bottleneck detection, false negatives will not be a problem. Integrating rate-based and window-based is a challenge. Could have multiple FSEs.

MW: what we have in rtcweb is priorities 

LE: assume kernel impl will not change on short time-scale, application could do more quick adaptation.

DW: Would it make sense to broadcast state table on local LAN so everyone can see what goes on?

MW2: Interesting research topic, but raises too many issues for rmcat.

ZS: Do the flows using the FSE run the same cong control algorithm? Could cover more scenarios than rmcat

MW2: in my tests yes, but we could use different ones.

LE: Are more working on coupled cong control. No.



Presentation:
Colin Perkins (CP), On the Use of RTP Control Protocol (RTCP) Feedback for Unicast Multimedia Congestion Control
draft-perkins-rmcat-rtp-cc-feedback (related to milestone rtcp-requirements) 

MZ: I always assumed that we would do RSize extension.

CP: I was picking pessimistic assumption.

MZ: I think it's even doable in the per-packet case.

CP: Depends on the bandwidth. But there's maybe not even a point.

Hannes Tschofenig (HT): Sending feedback quickly has its origin in cc. for traffic that is very quickly adaptable. In this case, there are reasons why you can't change it so fast anyway. And you may not want to do it anyway because of the user experience.

CP: Agreed. But you do need enough feedback so you can tell when problems are happening, to look at change in queuing delay for example. Frame feedback seems like a reasonable compromise, with also sending feedback faster when problems are developing.

HT: Agreed. I think RTCP is in fact a suitable mechanism for cc. feedback.

MR: Draft is very well written, thank you. Per-frame or per-packet feedback is a sampling problem. Ideally you do every single packet because backchannel is unreliable. Feedback has to be often enough for the control system to work. The fact that video encoders are limited in how fast they can change the rate is somehow irrelevant for the question of convergence, stability etc. of the controller.

CP: Agreed, this is a balancing act. We need to figure out what is appropriate. If something roughly in the order of an RTT is enough I think we can do that with RTCP very well.

MR: My point is that it's a control theory problem wrt. what types of feedback we want in the system.

RJ: Agreed. But these are just worst case scenarios, best case scenarios are much better. You may need to change rates dramatically in certain situations, to recover appropriately and deal with bursty situations.

TE: Is there anything video specific here? Can we ignore that it's video? People talk about VBR audio codecs that can change arbitrarily. Is the feedback for a video codec not totally decorrelated from congestion feedback?

CP: With audio, you may quickly run into situations where you send as much RTCP feedback as data.

RJ: RTCP-fair feedback will work doesn't mean that an algorithm should use RTCP feedback. This feedback is important and needed for certain cases. Piggybacking may be a useful technique in conjunction with RTCP.

CP: Point is, RTCP is a valid player in the space, potentially the right sort of feedback.

MZ: Agree. Even if RTCP feedback could do it, from an efficiency point of view the packet rate often matters more than the total bandwidth consumption, especially in wireless networks. Especially piggybacking could help.

CP: Not sending any RTCP packets that you wouldn't send anyway.

MR: Feedback could be very low in terms of bandwidth, but it needs to come often enough, but perhaps not more often than necessary. I see it as a sampling problem.

Kevin Gross (KG): How often do you sample, how often do you get that data? You can cause control system instability by having too much latency in feedback.

CP: People developing CC. algorithms must answer that for their algorithm. I'm trying to show that for reasonable amounts of feedback, RTCP can work. Don't discard it because of the assumption it's too slow.

ZS: Do we know what the minimum desired amount of feedback is? It depends on how we design our controller, that is not answered anywhere yet.

MR: TCP does it via clocking. You have to take the RTT into consideration. I want to make that point really clear, in order to get the control loop right, you have to think about that.

LE: What is the intended goal of the draft?

CP: Create awareness, raise discussion. I consider this as success. Not proposing to progress it any further.

LE: I could see the material in an appendix of a mechanism, to explain design choices that were made.



Presentation:
Rong Pan (RP), NADA: A Unified Congestion Control Scheme for Real-Time Media
draft-zhu-rmcat-nada 

ZS: PCN based signaling is something new that you want to design, it's not there, right?

LE: PCN standards are out.

ZS: Yes, but not used yet. We're going with things for the current Internet.

LE: We do a mechanism for the current Internet, but good if it works better in the future Internet.

ZS: I have similar results, and shown in the CC. workshop. Second thing: you have a rate-shaping buffer at the sender. Does it relate to video frames? It's per packet, right?

RP: Video frames are there, buffer basically smooths into the network.

ZS: Packet shaper on the packet level that delays packets may affect the subjective quality, e.g. when doing it with I-frame. Importance of packet is missing.

RJ: Queue is a fair number of packets beyond a certain window in your simulation. Once you're overloaded, do you end up with delay even when competing with itself?

RP: Depends on where your settle, on how heavily congested things are.

RJ: Loss rates on one slide were in the 3-5% range. That's a pretty high loss rate for video.

RP: If that's the way the network gives the feedback then that's how it is. But algorithm could be tuned to be more sensitive to delay.

RJ: Latency is a critical parameter. Throughput includes any recovery or FEC or whatever is needed to make for a high loss rate. Must be factored in. FEC has delay issues.

KR: Have you looked at Codel?

RP: We should evaluate that.

KR: Codel works without ECN or PCN.

RP: But it does drop packets.

KR: But the way it drops packets based on delay measurements.

Jim Gettys (JG): Codel is happy to do ECN.

RP: Have to look at it. If it's ECN that's fine. Algorithm can be adopted here. As a result you'll have less losses.

KR: How does it compete with TCP?

RP: Good question. Here we look at self-fairness. If TCP is the concern, we have a similar design where, we can respond with sqrt(p) when competing with loss based.

HA: Next time you update the draft, please include a reference for PCN.

TE: Agree with ZS, bringing in more codec knowledge might be used to optimize the behavior.

ZS: Encoder would have to know that this packet has been delayed or lost.

TE: Showed that it is fair … test should take into account whether ECN has some benefit over delay variation, especially when there is other traffic. Do we have clear evidence that we could ask network operators to enable it? That's still the open question now.

MZ: In Nada, losses are translated into delays, in Dflow, delays are translated into losses, in Google yet different… what is the right way to mix the signals?

LE: Evaluation criteria are more about objective quality measures of the result. Up to the mechanism to decide what's best.

MZ: There should be objective criteria on how to mix the signals - e.g. this combination of delay and loss is better than that combination of delay and loss.

LE: agreed.

RJ: If you take in more loss you effectively reduce your bit rate because you have to add FEC. Subjective quality is more important in the end.

ZS: Different algorithms handle delay and loss rates in different ways. Let's define how many % loss rate are okay. Google's mechanism says 10% is okay.

LE: It's not so easy. In a shared bottleneck you're not going to be the one who sees all the losses.