Re: RTP over QUIC experiments

Vidhi Goel <vidhi_goel@apple.com> Mon, 15 November 2021 23:41 UTC

Return-Path: <vidhi_goel@apple.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 6ED893A0C75; Mon, 15 Nov 2021 15:41:14 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.8
X-Spam-Level:
X-Spam-Status: No, score=-2.8 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.701, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=apple.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Obmve0xMNSor; Mon, 15 Nov 2021 15:41:10 -0800 (PST)
Received: from rn-mailsvcp-ppex-lapp15.apple.com (rn-mailsvcp-ppex-lapp15.rno.apple.com [17.179.253.34]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E6B1B3A0AEB; Mon, 15 Nov 2021 15:41:09 -0800 (PST)
Received: from pps.filterd (rn-mailsvcp-ppex-lapp15.rno.apple.com [127.0.0.1]) by rn-mailsvcp-ppex-lapp15.rno.apple.com (8.16.1.2/8.16.1.2) with SMTP id 1AFNbR8f016063; Mon, 15 Nov 2021 15:41:04 -0800
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=from : message-id : content-type : mime-version : subject : date : in-reply-to : cc : to : references; s=20180706; bh=nY9yaa1aLDusZPXrPT/NjwdZz7yxBBexrXBIB/C7zl0=; b=jj5hokY+DHEwu8o2kZ8mxzrFtXikPKWwvOD80Be9mAENIv/6t9B5rRXd0YaLT9qfj8RU 3ITdKinzIs+0BrRHFofgMUrn/ws5Aq3yfnfBR3/6kSO451CrapSpUSxcw30r/3nAg370 2Kuhj9RLtvEisAiLGhfMr/mevNKQAueZTtzUX/3tDNGosc4RdcumTeXgA+f1/b/wdYBO yuQdv3r7jUoEJmZdvn1iTX2TkMCUP9H8qUF+t9mbrDptcD+rYpxg+ZVO8Um5lrG0xnCq ap4NmPNMIOor6vLangqfhKubvlN1KmqgHTcSB7227Y87OBMnRWuHsIpr3k1cmUfAnuug dg==
Received: from rn-mailsvcp-mta-lapp02.rno.apple.com (rn-mailsvcp-mta-lapp02.rno.apple.com [10.225.203.150]) by rn-mailsvcp-ppex-lapp15.rno.apple.com with ESMTP id 3cabtfty59-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Mon, 15 Nov 2021 15:41:04 -0800
Received: from rn-mailsvcp-mmp-lapp01.rno.apple.com (rn-mailsvcp-mmp-lapp01.rno.apple.com [17.179.253.14]) by rn-mailsvcp-mta-lapp02.rno.apple.com (Oracle Communications Messaging Server 8.1.0.12.20210903 64bit (built Sep 3 2021)) with ESMTPS id <0R2N00AO80GF3M40@rn-mailsvcp-mta-lapp02.rno.apple.com>; Mon, 15 Nov 2021 15:41:03 -0800 (PST)
Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp01.rno.apple.com by rn-mailsvcp-mmp-lapp01.rno.apple.com (Oracle Communications Messaging Server 8.1.0.12.20210903 64bit (built Sep 3 2021)) id <0R2N007000F7SX00@rn-mailsvcp-mmp-lapp01.rno.apple.com>; Mon, 15 Nov 2021 15:41:03 -0800 (PST)
X-Va-A:
X-Va-T-CD: eb3895f047c5eb2445f3e46e3b2a76ee
X-Va-E-CD: b8411a1bf168b464d3aa17ac85edded5
X-Va-R-CD: 8d5afaa6f26b7358d7d6159f3f28c0b3
X-Va-CD: 0
X-Va-ID: a3ae96e8-09ea-478b-b1a8-9e3894a99d96
X-V-A:
X-V-T-CD: eb3895f047c5eb2445f3e46e3b2a76ee
X-V-E-CD: b8411a1bf168b464d3aa17ac85edded5
X-V-R-CD: 8d5afaa6f26b7358d7d6159f3f28c0b3
X-V-CD: 0
X-V-ID: 73f8ac31-de91-4da8-9a34-1647cf74deb6
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.790 definitions=2021-11-15_14:2021-11-15, 2021-11-15 signatures=0
Received: from smtpclient.apple (unknown [17.234.110.229]) by rn-mailsvcp-mmp-lapp01.rno.apple.com (Oracle Communications Messaging Server 8.1.0.12.20210903 64bit (built Sep 3 2021)) with ESMTPSA id <0R2N0014H0GED500@rn-mailsvcp-mmp-lapp01.rno.apple.com>; Mon, 15 Nov 2021 15:41:03 -0800 (PST)
From: Vidhi Goel <vidhi_goel@apple.com>
Message-id: <8CB16CED-5C42-4320-9CEE-B2FE8A78899D@apple.com>
Content-type: multipart/signed; boundary="Apple-Mail=_C5052DF3-7079-464C-BCA0-6F03E6F53036"; protocol="application/pkcs7-signature"; micalg="sha-256"
MIME-version: 1.0 (Mac OS X Mail 14.0 \(3654.100.0.2.11\))
Subject: Re: RTP over QUIC experiments
Date: Mon, 15 Nov 2021 15:40:52 -0800
In-reply-to: <2c965999-21f8-9a62-e008-47094a47430f@in.tum.de>
Cc: Vidhi Goel <vidhi_goel=40apple.com@dmarc.ietf.org>, Ingemar Johansson S <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>, "mathis.engelbart@gmail.com" <mathis.engelbart@gmail.com>, Ingemar Johansson S <ingemar.s.johansson@ericsson.com>, IETF QUIC WG <quic@ietf.org>, "avt@ietf.org" <avt@ietf.org>
To: Joerg Ott <ott@in.tum.de>
References: <AM8PR07MB8137AD430709AC54E6EA361CC2959@AM8PR07MB8137.eurprd07.prod.outlook.com> <236C1AB5-D0F5-4453-A4F3-DEAB5D7113CB@apple.com> <2c965999-21f8-9a62-e008-47094a47430f@in.tum.de>
X-Mailer: Apple Mail (2.3654.100.0.2.11)
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425, 18.0.790 definitions=2021-11-15_14:2021-11-15, 2021-11-15 signatures=0
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/vndswwEpSLd2PTWG9PFt3tMrt60>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 15 Nov 2021 23:41:15 -0000

>>> + Page 18 : Inferring the receive timestamp. What is suspect is that you will essentially halve the estimated queue delay (I assume here that the reverse path is uncongested). One alternative could be to compute
>>> receive-ts = send-ts + latest_rtt + min_rtt
>>> where min_rtt is the min RTT over a given time interval
>> You are right that halving latest_rtt / 2 may not be accurate if the reverse path is not congested, but there is no guarantee for that. So, a more accurate way to compute receive-ts is to use One way timestamps which can be added to QUIC as new Frames.
> 
> As long as the metric is taken for what it actually is, a rough
> approximation, we can probably work with a number of ways to measure.
> More to experiment.

I forgot to mention earlier, but the equation for recieve-ts should be (assuming queuing is only in one direction)

receive-ts = send-ts + min_rtt/2 + (latest_rtt - min_rtt)

Right?

Thanks,
Vidhi

> On Nov 15, 2021, at 12:34 PM, Joerg Ott <ott@in.tum.de> wrote:
> 
> Hi,
> 
>> What I am trying to understand is, what is the motivation behind running real time congestion control like SCReAM over QUIC congestion control? The results (as Ingemar also mentioned) are not encouraging.
> 
> we should clarify that this wasn't a design choice but rather part of
> systematically looking at the four combinations you get and then see
> what happens to each one of them (we didn't expect this one to fare
> particularly well but we were initially a bit surprised how poorly it
> performed).
> 
>> If we want to use a single QUIC connection for media (audio/video using RTP) and other reliable streams, then would it be better to not use QUIC CC for media streams and only use it for reliable streams? Obviously this will violate the current spec which applies congestion control on connection level. But maybe this use case can be specialized.
> 
> This would indeed be one option in the (more desirable?) design space:
> the question is if you should allow libraries out there without
> congestion control just because something claims it's real-time media
> and does its own.
> 
> Somebody may have mentioned the circuit breaker last week in some
> context (too many slots, sorry).
> 
> Indeed, one idea could be having a QUIC "enforce" a limit that it
> considers acceptable for the current connection and provide the
> RTP CC with the necessary parameters to come to a meaningful rate
> itself; as long as the offered load from the RTP CC fits the
> enveloped computed by QUIC CC, the DATAGRAMs could just flow;
> above that rate, queuing or dropping (or local ECN-style signals)
> could follow.
> 
> It remains to be seen if shared congestion control between real-time
> flows, other datagrams, and regular QUIC streams can be done in a
> sensible manner, with acceptable complexity and little brittleness.
> And if there is a strong use case for such.  This was quite a bit
> discussed in the MOQ side meeting.
> 
>>> + Split of network congestion control and media rate control : QUIC already today has the congestion control on the connection level, it is then up to the individual streams to deliver media, subject to the individual stream priorities. SCReAM is quite similar in that respect, one difference is perhaps the implementation of the media rate control. I think that with QUIC one should do a full split and do the network congestion control on the QUIC connection level. The congestion control would then be some low latency version, perhaps BBRv2? or something similar, I am not sure that the network congestion control in SCReAM is the idea choice here as it is quite a lot tailored for RTP media. 
>> The impact of cascading two congestion controllers (with different input and output parameters) has not been studied extensively yet. And is a real time CC like SCReAM by itself not enough to control the congestion in the network? In other words, does it need another congestion controller to make sure that the real time data doesn’t cause more congestion in the network?
> 
> Right.  The main question is: should a QUIC connection trust the
> arbitrary datagram source or should it check.  Given that all sources
> (datagrams and otherwise) would often be part of the same application,
> there is probably not much point in trying to cheat on itself.  Maybe
> some sanity checks would make sense, paired with mild adjustments of
> the share given to the reliable streams as the codec source traffic
> won't be able to follow exactly a given target data rate.
> 
>>> My SCReAM experience is that one need to leak some of the congestion signals from the connection level congestion control up to the stream rate control, to make the whole thing responsive enough. In the SCReAM code one can see that the exotic variable queueDelayTrend as well as ECN marks and loss events are used for this purpose. I believe that something like that is needed for an RTP (or whatever low latency) media over QUIC. I believe that it is necessary to leak congestion information from the connection level up to the stream level, especially to be able to exploit L4S fully, even though it is a bit of a protocol layer violation.
>> We absolutely should allow sharing of network events, like RTT, ECN, packet loss from QUIC to RTP. Not sure how it is protocol layer violation.
> 
> Yes.
> 
>>> + Page 18 : Inferring the receive timestamp. What is suspect is that you will essentially halve the estimated queue delay (I assume here that the reverse path is uncongested). One alternative could be to compute
>>> receive-ts = send-ts + latest_rtt + min_rtt
>>> where min_rtt is the min RTT over a given time interval
>> You are right that halving latest_rtt / 2 may not be accurate if the reverse path is not congested, but there is no guarantee for that. So, a more accurate way to compute receive-ts is to use One way timestamps which can be added to QUIC as new Frames.
> 
> As long as the metric is taken for what it actually is, a rough
> approximation, we can probably work with a number of ways to measure.
> More to experiment.
> 
> Best,
> Jörg
> 
>>> On Nov 12, 2021, at 8:28 AM, Ingemar Johansson S <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org <mailto:ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>> wrote:
>>> 
>>> Hi Jörg, Mathis + others
>>> It was nice to learn about your activity to try and use SCReAM as example algorithm to integrate with QUIC. Pages 14-25 in
>>> https://datatracker.ietf.org/meeting/112/materials/slides-112-avtcore-ietf-112-avtcore-03 <https://datatracker.ietf.org/meeting/112/materials/slides-112-avtcore-ietf-112-avtcore-03>
>>> Did you use the new gsteamer plugin fromhttps://github.com/EricssonResearch/scream/tree/master/gstscream <https://github.com/EricssonResearch/scream/tree/master/gstscream>  ?
>>> Observations/Comments:
>>> + SCReAM + Reno : Strange that the throughput dropped like that but perhaps an unlucky outcome of two cascaded congestion controls.
>>> + Split of network congestion control and media rate control : QUIC already today has the congestion control on the connection level, it is then up to the individual streams to deliver media, subject to the individual stream priorities. SCReAM is quite similar in that respect, one difference is perhaps the implementation of the media rate control.
>>> I think that with QUIC one should do a full split and do the network congestion control on the QUIC connection level. The congestion control would then be some low latency version, perhaps BBRv2? or something similar, I am not sure that the network congestion control in SCReAM is the idea choice here as it is quite a lot tailored for RTP media.
>>> The media rate control is done on the stream level and is then subject to stream priority. This should give a more clean split of functionality.
>>> My SCReAM experience is that one need to leak some of the congestion signals from the connection level congestion control up to the stream rate control, to make the whole thing responsive enough. In the SCReAM code one can see that the exotic variable queueDelayTrend as well as ECN marks and loss events are used for this purpose. I believe that something like that is needed for an RTP (or whatever low latency) media over QUIC. I believe that it is necessary to leak congestion information from the connection level up to the stream level, especially to be able to exploit L4S fully, even though it is a bit of a protocol layer violation.
>>> + Stream prioritization : … is a problematic area, especially if one stream is low latency video and another stream is a large chunk of data for e.g. a large web page. With a simple round robin scheduler, the stream with the large chunk of data will easily win because it is quite likely to always have data to transmit. So some WRR is needed. I have even had problems with the algorithm in SCReAM that prioritizes between two cameras/video coders, this because the two cameras see different views and thus provide differing information content/compression need.
>>> + Page 18 : Inferring the receive timestamp. What is suspect is that you will essentially halve the estimated queue delay (I assume here that the reverse path is uncongested). One alternative could be to compute
>>> receive-ts = send-ts + latest_rtt + min_rtt
>>> where min_rtt is the min RTT over a given time interval
>>> Regards
>>> /Ingemar
>