Re: [tsvwg] QUIC with L4S

Christian Huitema <huitema@huitema.net> Tue, 05 July 2022 22:40 UTC

Return-Path: <huitema@huitema.net>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 717B8C14F743 for <tsvwg@ietfa.amsl.com>; Tue, 5 Jul 2022 15:40:37 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.782
X-Spam-Level:
X-Spam-Status: No, score=-3.782 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, NICE_REPLY_A=-1.876, RCVD_IN_DNSWL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id SpVLAReWn6hO for <tsvwg@ietfa.amsl.com>; Tue, 5 Jul 2022 15:40:32 -0700 (PDT)
Received: from mx36-out10.antispamcloud.com (mx36-out10.antispamcloud.com [209.126.121.30]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4F440C13CF7E for <tsvwg@ietf.org>; Tue, 5 Jul 2022 15:39:16 -0700 (PDT)
Received: from xse368.mail2web.com ([66.113.197.114] helo=xse.mail2web.com) by mx258.antispamcloud.com with esmtp (Exim 4.92) (envelope-from <huitema@huitema.net>) id 1o8rCJ-0003eY-Sf for tsvwg@ietf.org; Wed, 06 Jul 2022 00:39:15 +0200
Received: from xsmtp22.mail2web.com (unknown [10.100.68.61]) by xse.mail2web.com (Postfix) with ESMTPS id 4LcyJs0lVkz9vS for <tsvwg@ietf.org>; Tue, 5 Jul 2022 15:39:09 -0700 (PDT)
Received: from [10.5.2.49] (helo=xmail11.myhosting.com) by xsmtp22.mail2web.com with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.92) (envelope-from <huitema@huitema.net>) id 1o8rCG-0007BK-VD for tsvwg@ietf.org; Tue, 05 Jul 2022 15:39:08 -0700
Received: (qmail 1893 invoked from network); 5 Jul 2022 22:39:08 -0000
Received: from unknown (HELO [192.168.1.104]) (Authenticated-user:_huitema@huitema.net@[172.58.46.172]) (envelope-sender <huitema@huitema.net>) by xmail11.myhosting.com (qmail-ldap-1.03) with ESMTPA for <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>; 5 Jul 2022 22:39:08 -0000
Message-ID: <77332295-c7b7-21aa-7661-af5770b4c249@huitema.net>
Date: Tue, 05 Jul 2022 15:39:08 -0700
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0
Content-Language: en-US
To: Ingemar Johansson S <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>, "tsvwg@ietf.org" <tsvwg@ietf.org>
References: <AM8PR07MB8137710DD707BF2DB4FDEA9DC2B89@AM8PR07MB8137.eurprd07.prod.outlook.com> <AM8PR07MB81378A432301907ABFDFC2B8C2B89@AM8PR07MB8137.eurprd07.prod.outlook.com>
From: Christian Huitema <huitema@huitema.net>
In-Reply-To: <AM8PR07MB81378A432301907ABFDFC2B8C2B89@AM8PR07MB8137.eurprd07.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"; format="flowed"
Content-Transfer-Encoding: 8bit
X-Originating-IP: 66.113.197.114
X-Spampanel-Domain: xsmtpout.mail2web.com
X-Spampanel-Username: 66.113.197.0/24
Authentication-Results: antispamcloud.com; auth=pass smtp.auth=66.113.197.0/24@xsmtpout.mail2web.com
X-Spampanel-Outgoing-Class: unsure
X-Spampanel-Outgoing-Evidence: Combined (0.15)
X-Recommended-Action: accept
X-Filter-ID: Pt3MvcO5N4iKaDQ5O6lkdGlMVN6RH8bjRMzItlySaT9WLQux0N3HQm8ltz8rnu+BPUtbdvnXkggZ 3YnVId/Y5jcf0yeVQAvfjHznO7+bT5wK33BxCZ4Q1O8tWwOlQNPYfYzfQXcfqmra3dmoHS4ygiUd LwfG6zr0RNjvUCTnGYhWuRWrkPihq53YqAd1ENNqBHtNXu1E6L4+KyOXc4QYanQOD0r6/AaHZiEt dTMtMlia0Lmg/jgHfCNZd+W+PXf62CLEEqYPGKN9+TfCRpjw8iue9TLOhN8AYRsvkjfngQDCCqjo gBy6Z7Z1H0a6XOEbzDkBvlIN1pUDU5DU5DggD98cjIN3reG9z0FKKQ5m2Qpw7sOVVcM1Xk+Tdz6g /UMvfWqyN3veeFIMJz/vumcqAwMU9kjfE7EFo+kP5riIEUmxU01QhuxnshSbl6nxbLZ35/xY0uvo WBEOfzq3RG28wI7w4vcwqZanLHsZM8r4s5ZjlHoGly8aneNxj+pRyx6DAzHPcWsnfqGSaNoXhWPo OpFVgpT1b21uZVckGp0ccOZtuBWXiK6eoWgQZnNLL6SbpUc7peFeo3eDQNYbhOKhzzgqmaDn5SlD Y9mmtv6e91aWBLor1oCWetcUjeG94V2XTy8q18r5F7mwejpNkMpMQ89aQIYBqczAPuapU9BOtVQR gX9p3fVDcQNjp/0rmm98DRojSVizNl0ce/s7u0P9b7Oijoc3SCZfWp1RjkjWCw/vIUzTXkDAiiJi mGhLUFuS2lhaIetXfCg1JdAVrOwKfO3duG+GtWTZWmQCpO3J0mlUoFIvD3sIcP1fhJPM6B/8eYqd nlH0oXEAllp/aagGWCZxm4fR8yOHA06pZKvCwY2J+dym1L8cD17Js0v4cp1MFwdo/a56xB6VIQoh vHKo1jcKVNeVJ9BXyu9+ceCqThTYg2px1fSoqxQCCHnLMo/m9VKh99btUAanjnMCAH2co+fBoeG+ Hs0afhsY/5zhNYWRVYKU9W9tbmVXJBqdHHDmZEKhyNAv1N35kYWaEdgLurFV5oTvAcwA4rM3FkfW 8/1kE/e7sUnsVpINvARNxpFO
X-Report-Abuse-To: spam@quarantine11.antispamcloud.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/wHNoZNFotn5wiOiLjqXwH_q0xtg>
Subject: Re: [tsvwg] QUIC with L4S
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Jul 2022 22:40:37 -0000

Ingemar,

The implementation of Prague in Picoquic that you mentioned was really 
old. I am updating it before checking it in the main branch, and I wrote 
a little blog about it: 
https://huitema.wordpress.com/2022/07/05/a-low-latency-internet-with-l4s/

Is there a new Prague draft? I could only find an obsolete draft from 
2019. Maybe I did not look in the right places.

I found a few issue when doing the implementation:

  * Alpha smoothing does not handle well the sudden onset of congestion. 
My implementation overrides the smoothing if there are sudden onset of 
marks, setting "alpha" to "frac" immediately, instead of using 1/16 
smoothing. Not doing that improves performance, but also causes a sharp 
increase in the number of losses.

* Hysteresis. This is an issue with many  congestion control algorithms. 
When congestion is noticed, it was caused by the CWIND set 2 RTT before 
the notification -- the packets sent in the last RTT are still in 
flight. This is mitigated in Reno by not acting on losses of packets 
sent before entering recovery. In Prague, ECN marks triggered by these 
packets do increase the ECN counters, which means alpha increases and we 
get an extra round of CWIND reduction 1 RTT after the first one.

* Restart after idle. In my tests, I simulate a suggestion of "GET 1MB", 
each one starting after the previous is service. That means the sender 
is idle for 1RTT. On restart, the sender starts with a large CWIND, 
sends a lot of packets, which causes a bump in latency and a large 
number of ECN marks. Maybe that's linked to my implementation of pacing 
using a leaky bucket, and the leaky bucket is too large. I will try fix 
that, but I suspect the draft should say something about resuming from idle.

We may also need to do some work on "slow start". My implementation 
exist slow start on first ECN-CE mark, but at that point there is a full 
flight of packets in transit, which is going to cause either packet 
losses or spikes in latency.

I also had to try a couple of different settings for the threshold value 
used in the simulation. Too low, and the throughput drops. Too high, and 
the amount of losses increases too much. In the tests, the router has a 
queue size of 1 BDP, and setting the threshold to BDP/4 appears to work 
well.

I am going to test more scenarios, such as starting with a high 
bandwidth and simulating a rapid decrease, or vice versa. Also, testing 
what happens with higher bandwidth and RTT. And looking at ways to plug 
the Prague concepts in BBR, possibly as a way to replace the "scan for 
min RTT" phases of BBR. In any case, the initial results seem encouraging.

-- Christian Huitema

On 6/28/2022 1:02 AM, Ingemar Johansson S wrote:
> And to clarify, as indicated by the subject line. QUIC with L4S support
>
> /Ingemar
>
> From: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
> Sent: Tuesday, 28 June 2022 09:47
> To: tsvwg@ietf.org
> Cc: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
> Subject: QUIC with L4S
>
> Hi
>
> A fairly short question.
> What QUIC implementations, open source and well as propietary exist today.
> I know of the following
> + Apples implementation, presented by Vihdi Goel
> + Pico-quic with Prague maintained/presented by Bob, Koen and others ( https://github.com/qdeconinck/picoquic/tree/quic-prague  )
>
> Are there any other examples available ?
>
> Regards
> /Ingemar
> ==================================
> Ingemar Johansson  M.Sc.
> Master Researcher
>
> Ericsson AB
> GFTL ER Networks RNS Protocol & E2E Perf
> Labratoriegränd 11
> 977 53, Luleå, Sweden
> Phone +46-1071 43042
> SMS/MMS +46-73 078 3289
> ingemar.s.johansson@ericsson.com<mailto:ingemar.s.johansson@ericsson.com>
> www.ericsson.com<http://www.ericsson.com/>
>
>    Your're always standing on the shoulders
>      of something that you've experienced
>                         May El-Toukhy
> ===================================
>