Re: [tsvwg] QUIC with L4S

Vidhi Goel <vidhi_goel@apple.com> Tue, 05 July 2022 23:24 UTC

Return-Path: <vidhi_goel@apple.com>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0B0EEC13CF7E for <tsvwg@ietfa.amsl.com>; Tue, 5 Jul 2022 16:24:44 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.851
X-Spam-Level:
X-Spam-Status: No, score=-2.851 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.745, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=apple.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hKWErlretj35 for <tsvwg@ietfa.amsl.com>; Tue, 5 Jul 2022 16:24:41 -0700 (PDT)
Received: from ma1-aaemail-dr-lapp01.apple.com (ma1-aaemail-dr-lapp01.apple.com [17.171.2.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B6382C157B55 for <tsvwg@ietf.org>; Tue, 5 Jul 2022 16:24:41 -0700 (PDT)
Received: from pps.filterd (ma1-aaemail-dr-lapp01.apple.com [127.0.0.1]) by ma1-aaemail-dr-lapp01.apple.com (8.16.0.42/8.16.0.42) with SMTP id 265NJfDI053570; Tue, 5 Jul 2022 16:24:37 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=apple.com; h=from : message-id : content-type : mime-version : subject : date : in-reply-to : cc : to : references; s=20180706; bh=Al0JenyO3cVkaxaM3uBs0JsiY/0bAR9+lQbtv8bnuXs=; b=alLOX68GDrrxg6IhQXlaosAUD0I0R9IKsi975KoACfK8OPXctZatVHFnnjyuPYd6V/y9 rnmB5NisTe0Je/rMiDwhRW+1e/eXP6LNNu2tYdtSCWkmkBF9rM3u/FLPegsoUe8phXK8 k2G2KM6lOpBHZj5/c1+tNF/aUEX4mIOy5uvpLAAvOyOyBG5ZUFOin0ja1WvvKDGs93Dr /DSS7JxCrfiePAfxVd6NcirhUESo4im3MqxSVFQSjC165VexFipW3m//o0zLmJ7031q9 s3LoRE81aJmtM+rss1hi5nTjsqLtSRuvpvNhRNLBB9Vj3eIifNbqutUiZqa9Sd7cQhvA ww==
Received: from rn-mailsvcp-mta-lapp03.rno.apple.com (rn-mailsvcp-mta-lapp03.rno.apple.com [10.225.203.151]) by ma1-aaemail-dr-lapp01.apple.com with ESMTP id 3h4ucb2s7k-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Tue, 05 Jul 2022 16:24:37 -0700
Received: from rn-mailsvcp-mmp-lapp04.rno.apple.com (rn-mailsvcp-mmp-lapp04.rno.apple.com [17.179.253.17]) by rn-mailsvcp-mta-lapp03.rno.apple.com (Oracle Communications Messaging Server 8.1.0.18.20220407 64bit (built Apr 7 2022)) with ESMTPS id <0REK005PCMD0Q5B0@rn-mailsvcp-mta-lapp03.rno.apple.com>; Tue, 05 Jul 2022 16:24:36 -0700 (PDT)
Received: from process_milters-daemon.rn-mailsvcp-mmp-lapp04.rno.apple.com by rn-mailsvcp-mmp-lapp04.rno.apple.com (Oracle Communications Messaging Server 8.1.0.18.20220407 64bit (built Apr 7 2022)) id <0REK00200MAE4700@rn-mailsvcp-mmp-lapp04.rno.apple.com>; Tue, 05 Jul 2022 16:24:36 -0700 (PDT)
X-Va-A:
X-Va-T-CD: 058bbac8ca772bcfc9e38720b87faa94
X-Va-E-CD: 0ff45e6f85150148ecdebf385bc1bb20
X-Va-R-CD: 718bdfe25fd35d942b441ddf275b2c45
X-Va-CD: 0
X-Va-ID: 9a16c834-df2a-4642-9168-93f891345432
X-V-A:
X-V-T-CD: 058bbac8ca772bcfc9e38720b87faa94
X-V-E-CD: 0ff45e6f85150148ecdebf385bc1bb20
X-V-R-CD: 718bdfe25fd35d942b441ddf275b2c45
X-V-CD: 0
X-V-ID: 98a1bb18-f72a-4dfd-bde5-ba99cba61c3f
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517, 18.0.883 definitions=2022-07-05_19:2022-06-28, 2022-07-05 signatures=0
Received: from smtpclient.apple (vimac.scv.apple.com [17.192.154.53]) by rn-mailsvcp-mmp-lapp04.rno.apple.com (Oracle Communications Messaging Server 8.1.0.18.20220407 64bit (built Apr 7 2022)) with ESMTPSA id <0REK00CI4MCZ9600@rn-mailsvcp-mmp-lapp04.rno.apple.com>; Tue, 05 Jul 2022 16:24:36 -0700 (PDT)
From: Vidhi Goel <vidhi_goel@apple.com>
Message-id: <CAB9FB5F-A00D-4924-9DF4-008380B1D4C9@apple.com>
Content-type: multipart/alternative; boundary="Apple-Mail=_C25BA1F6-6A94-4086-97C8-CAFCB6BC33FC"
MIME-version: 1.0 (Mac OS X Mail 16.0 \(3720.101.4.1.51\))
Date: Tue, 05 Jul 2022 16:24:25 -0700
In-reply-to: <77332295-c7b7-21aa-7661-af5770b4c249@huitema.net>
Cc: Ingemar Johansson S <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>, "tsvwg@ietf.org" <tsvwg@ietf.org>
To: Christian Huitema <huitema@huitema.net>
References: <AM8PR07MB8137710DD707BF2DB4FDEA9DC2B89@AM8PR07MB8137.eurprd07.prod.outlook.com> <AM8PR07MB81378A432301907ABFDFC2B8C2B89@AM8PR07MB8137.eurprd07.prod.outlook.com> <77332295-c7b7-21aa-7661-af5770b4c249@huitema.net>
X-Mailer: Apple Mail (2.3720.101.4.1.51)
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517, 18.0.883 definitions=2022-07-05_19:2022-06-28, 2022-07-05 signatures=0
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/RMKhB-dKuyMqLTYPY_67mYpsDJA>
Subject: Re: [tsvwg] QUIC with L4S
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Jul 2022 23:24:44 -0000

Thank you Christian for resuming the work on scalable CC. I have been using picoQUIC as a receiver for my testing and once you merge it to main, I would use it as a sender too.

> Is there a new Prague draft? I could only find an obsolete draft from 2019. Maybe I did not look in the right places.

This is the latest - https://datatracker.ietf.org/doc/html/draft-briscoe-iccrg-prague-congestion-control-00

All your points are valid and I have noted them.

> I also had to try a couple of different settings for the threshold value used in the simulation. Too low, and the throughput drops. Too high, and the amount of losses increases too much. In the tests, the router has a queue size of 1 BDP, and setting the threshold to BDP/4 appears to work well.

Which AQM are you using on your router? For me setting fixed threshold = 2ms when testing over Ethernet and threshold = 4ms when testing over Wi-Fi worked well.


Vidhi

> On Jul 5, 2022, at 3:39 PM, Christian Huitema <huitema@huitema.net> wrote:
> 
> Ingemar,
> 
> The implementation of Prague in Picoquic that you mentioned was really old. I am updating it before checking it in the main branch, and I wrote a little blog about it: https://huitema.wordpress.com/2022/07/05/a-low-latency-internet-with-l4s/
> 
> Is there a new Prague draft? I could only find an obsolete draft from 2019. Maybe I did not look in the right places.
> 
> I found a few issue when doing the implementation:
> 
>  * Alpha smoothing does not handle well the sudden onset of congestion. My implementation overrides the smoothing if there are sudden onset of marks, setting "alpha" to "frac" immediately, instead of using 1/16 smoothing. Not doing that improves performance, but also causes a sharp increase in the number of losses.
> 
> * Hysteresis. This is an issue with many  congestion control algorithms. When congestion is noticed, it was caused by the CWIND set 2 RTT before the notification -- the packets sent in the last RTT are still in flight. This is mitigated in Reno by not acting on losses of packets sent before entering recovery. In Prague, ECN marks triggered by these packets do increase the ECN counters, which means alpha increases and we get an extra round of CWIND reduction 1 RTT after the first one.
> 
> * Restart after idle. In my tests, I simulate a suggestion of "GET 1MB", each one starting after the previous is service. That means the sender is idle for 1RTT. On restart, the sender starts with a large CWIND, sends a lot of packets, which causes a bump in latency and a large number of ECN marks. Maybe that's linked to my implementation of pacing using a leaky bucket, and the leaky bucket is too large. I will try fix that, but I suspect the draft should say something about resuming from idle.
> 
> We may also need to do some work on "slow start". My implementation exist slow start on first ECN-CE mark, but at that point there is a full flight of packets in transit, which is going to cause either packet losses or spikes in latency.
> 
> I also had to try a couple of different settings for the threshold value used in the simulation. Too low, and the throughput drops. Too high, and the amount of losses increases too much. In the tests, the router has a queue size of 1 BDP, and setting the threshold to BDP/4 appears to work well.
> 
> I am going to test more scenarios, such as starting with a high bandwidth and simulating a rapid decrease, or vice versa. Also, testing what happens with higher bandwidth and RTT. And looking at ways to plug the Prague concepts in BBR, possibly as a way to replace the "scan for min RTT" phases of BBR. In any case, the initial results seem encouraging.
> 
> -- Christian Huitema
> 
> On 6/28/2022 1:02 AM, Ingemar Johansson S wrote:
>> And to clarify, as indicated by the subject line. QUIC with L4S support
>> 
>> /Ingemar
>> 
>> From: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
>> Sent: Tuesday, 28 June 2022 09:47
>> To: tsvwg@ietf.org
>> Cc: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
>> Subject: QUIC with L4S
>> 
>> Hi
>> 
>> A fairly short question.
>> What QUIC implementations, open source and well as propietary exist today.
>> I know of the following
>> + Apples implementation, presented by Vihdi Goel
>> + Pico-quic with Prague maintained/presented by Bob, Koen and others ( https://github.com/qdeconinck/picoquic/tree/quic-prague  )
>> 
>> Are there any other examples available ?
>> 
>> Regards
>> /Ingemar
>> ==================================
>> Ingemar Johansson  M.Sc.
>> Master Researcher
>> 
>> Ericsson AB
>> GFTL ER Networks RNS Protocol & E2E Perf
>> Labratoriegränd 11
>> 977 53, Luleå, Sweden
>> Phone +46-1071 43042
>> SMS/MMS +46-73 078 3289
>> ingemar.s.johansson@ericsson.com <mailto:ingemar.s.johansson@ericsson.com><mailto:ingemar.s.johansson@ericsson.com>
>> www.ericsson.com <http://www.ericsson.com/><http://www.ericsson.com/>
>> 
>>   Your're always standing on the shoulders
>>     of something that you've experienced
>>                        May El-Toukhy
>> ===================================