Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S

Sebastian Moeller <moeller0@gmx.de> Tue, 10 March 2020 15:29 UTC

Return-Path: <moeller0@gmx.de>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B646C3A1540; Tue, 10 Mar 2020 08:29:43 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.648
X-Spam-Level:
X-Spam-Status: No, score=-1.648 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=gmx.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 79M8TAndNOX7; Tue, 10 Mar 2020 08:29:41 -0700 (PDT)
Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 47C043A153F; Tue, 10 Mar 2020 08:29:40 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1583854167; bh=4EcII2Az6/qX3CvOPwaU1Imdu/jRZ3ClEs7rTarIsdU=; h=X-UI-Sender-Class:Subject:From:In-Reply-To:Date:Cc:References:To; b=Cf5ris3esf28adli/AiHNgFeE8BdJlB/7FVUbFvBc8XuG8LuMS0DXqRVt7w0v1XUs nAG6YS10PcMZ2WkGATjPiSBlVZJdu/thiudyiOuRnjFlJy8F7k8D4UNbAM1STdbVLQ CKOe8pcNVZJs8fYyqPkeiV586e4FT1n31+3yVwP8=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from [10.11.12.22] ([134.76.241.253]) by mail.gmx.com (mrgmx105 [212.227.17.168]) with ESMTPSA (Nemesis) id 1N4zAy-1jJI5T1uOy-010wZ2; Tue, 10 Mar 2020 16:29:27 +0100
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
From: Sebastian Moeller <moeller0@gmx.de>
In-Reply-To: =?utf-8?q?=3CHE1PR07MB44255CED94938F9C38515FD6C2FF0=40HE1PR07MB?= =?utf-8?q?4425=2Eeurprd07=2Eprod=2Eoutlook=2Ecom=3E?=
Date: Tue, 10 Mar 2020 16:29:24 +0100
Cc: Ingemar Johansson S <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>, "tsvwg@ietf.org" <tsvwg@ietf.org>, "iccrg@irtf.org" <iccrg@irtf.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <AC10E219-46C2-4345-B98F-71689F788B13@gmx.de>
References: =?utf-8?q?=3CHE1PR07MB44251B019947CDB6602B30B2C2FF0=40HE1PR07MB4?= =?utf-8?q?425=2Eeurprd07=2Eprod=2Eoutlook=2Ecom=3E?= =?utf-8?q?=3CA2300F8D-5F87-461E-AD94-8D7B22A6CDF3=40gmx=2Ede=3E_=3CHE1PR07M?= =?utf-8?q?B4425B105AFF56D1566164900C2FF0=40HE1PR07MB4425=2Eeurprd07=2Eprod?= =?utf-8?q?=2Eoutlook=2Ecom=3E?= <1C969A05-A4B7-43E9-B694-3195A2FC086A@gmx.de> =?utf-8?q?=3CHE1PR07MB44255CED94938F9C38515FD6C2FF0=40HE1PR07MB4425=2Eeurpr?= =?utf-8?q?d07=2Eprod=2Eoutlook=2Ecom=3E?=
To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
X-Mailer: Apple Mail (2.3445.104.11)
X-Provags-ID: V03:K1:jZJCuV+DzScowlqpIM1kQ5zFX2Ko4cbXXgCzKBnHz1+JPF7lBvJ ol8Ye/zA2cnYfKQ8SiMnA9p3kDq8qJQ30Giqq8jamfx6PbWs25CjCNnQ9I5gvHVSpG8VRYD xvfU8lkz0X4hcp1tZfMhK8VKRUj8LsbfXlkcLfXXCbl1mKY5ix4SqCg6xHg6PW/Ph05HT2v 45Dd3xMgkbKnhIQmirorg==
X-UI-Out-Filterresults: notjunk:1;V03:K0:jhivl3GKyGE=:aCErdHVOtfaLpYMJSVUnOR KdBvvhfiYjkIQjWUpL3CsDGok219hM5fbqZprDJQfHrXEFeoDLP9kWxbo6IZviVd9OCpMGfRc oi3TWFr9nXjk0RH4S5KoMkEKt+I9q6V0onkP/RHfY8rGT/1+pRRv9CG/w2eQQUw7UVvtZ6z+Q w1+nOTtrWxlSYDcpGcry0Uc4L6TI8mQ65WB4PL3wGD2OMtdNCxvuEsGEh5dV1qDDZ9yKqZzIC 3wxpWgeRC0mS+70tH2xqx7XbXPAUfTNpZpzOfxa5C3vceFWeWyfTsrEq6K71GnEMRg/9RwTmC 2H1xcrhepsoqaWFhW/sU7EstPR6SChlKS+SsCM1WPqbq4zfhymYvmBZWKVC/9qdIQC4szVAQH ZYGWzIGbt3HOHl7hPhn6d68MRRed4zxLdiGx6cd4Dpxp9o5vnA9cqFZYRHxmsM6rK2bdjHWoB 6grmu/u2VxAWw41enMRRuBMyiiBWxgFqxsF8ZlsmtiY2YsL+DTqPFTpEumTFPjxPUUt0bmbA0 r2SzcGafZGua1i6Rm2wrlz2kcyfw+2+cqAqgEWBe1605oF3qmPl2YCJhWA/42TInOZFphoVhy 3clLbphy0b6nxXTqynqw43Y0kfDLhNYQE6V6FAMyY9aZBXyBOExqyRYpPJjSiccSNOdGFFbJz e2prZk/WNfX9b8tBJXxeBvTLiRMWqAPTdaiITz9hucMGWogspf4HeGO27/bdetLldnkSBZ3Mh 5MzUKhSeM15m/8HIYXobTX/U/9ltmxjwmUaQg54QASz5mXRhxtue25cIQGEIQ6WevepE9a7Fl 4+y/IRNbzzMPSz1bAI2DmdjLyfn1sdX4DuYkmWx5Pwwfoku9pntXq7SxWpONDU6ltm+zT7rMS IJje7s0KuNMb/3NUEOMtI35zxLo83OX0GXPJzX+IU/AEMiQ7nyg6AeqtuG8uO8QYlZLTEdeVa Tb5KEBGbBaiPBX2raU/p2uFKmMc26yaYzC0UPSl724vx5o0bwsuNEPF0d+dhvoC287h9xlR3t gWx4Ub7r7wRjBQkHbtUE66RUWHZBr8+lWL7NYQK+CKmSkS2aE9iZz+Keskyuz8EpKtmcTe9v7 XyAvRFW0714CfviAMEqYgjVxS1ex2iix96wd+kRdZKCDaeMpQ6ulYaczZh/vLIynYhmChsb6D xYbHFwehGQdtdV84htUPkPrOL+aD2iayxqKZios4JfjC1Tr/R0uKj7fJ+lEyJVs1PO/sG3toE KOHsaDrhbcsLUoQsYA7vX6ZvgzFHq5zPEkGkDjIgpr5RHWmeKJts5h83rWm/ADgen2om8/vHr 4pYqnu1S28OiP8VjxIJv5xP2VZ+weKChaapn3U9fCgnTUw3Dh2J5mRHye1pS943y7KqENoNL
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/laBRptkAsxhcJWdNT5yHLALH3cw>
Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 10 Mar 2020 15:29:44 -0000

Hi Ingemar,



> On Mar 10, 2020, at 13:37, Ingemar Johansson S <ingemar.s.johansson@ericsson.com> wrote:
> 
> Hi
> 
> The SCReAM code is freely available on https://github.com/EricssonResearch/scream for anybody interested to run their own experiment with whatever AQM/ECN configuration. 

	[SM] Well, if you are not interested in figuring out what is happening with with rfc3168 ECN ans SCReAM that is your prerogative... 


> Please note that SCReAM is configured in an L4S mode when the network AQM does L4S marking (mimicking ECT(1)). For CoDel-ECN however, SCReAM runs in normal ECN mode with a beta of 0.8 (=20% reduction on CWND for each congestion event)

	[SM] Okay, that is a far cry from Reno's 50% though, have you tested different reduction percentages or how did you arrive at 20%?
But is it really 0.8? ScreamTx.h has the following:

// CWND scale factor upon ECN-CE event
static const float kEcnCeBeta = 0.9f;

I am probably missing something somewhere...

> 
> I tried also with other different ramp markers (1ms/10ms), (2ms/10ms),(5ms/15ms).

	[SM] That means for L4S, or do you mean you tried different values for Codel's target ans ce_threshold parameters?


> There are slight variations  in throughput and latency but not dramatic.

	[SM] Well, these are also pretty slight variations in numerical values, at least if compared to codel's default interval of 100ms...

> And truth to be told, the ECN behavior is better tuned in the code than the L4S behavior. 

	[SM] For a defintion of better that ignores that you seem to be much happier with the results L4S gives you.

> There is room for improvement as regards to the L4S behavior (for instance faster ramp-up) and it may well be the case that SCReAM is completely scrapped in favor of new designs. 
> But the bottomline, the L4S thresholds and L4S code is not carefully picked to show a good performance.

	[SM] So, you tried different L4S parameters (and found the to have similar performance) but for rfc3168 ECN and Codel you only tried the default parameters?

Best Regards
	Sebastian

> 
> /Ingemar
> 
>> -----Original Message-----
>> From: Sebastian Moeller <moeller0@gmx.de>
>> Sent: den 10 mars 2020 11:28
>> To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
>> Cc: Ingemar Johansson S
>> <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>; tsvwg@ietf.org;
>> iccrg@irtf.org
>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>> 
>> Hi Ingemar,
>> 
>> 
>>> On Mar 10, 2020, at 11:07, Ingemar Johansson S
>> <ingemar.s.johansson@ericsson.com> wrote:
>>> 
>>> Hi
>>> 
>>> For the future studies we will only focus on L4S as the scope is to study the
>> performance gain that L4S give for instance for AR/VR, gaming and remote
>> control applications.
>> 
>> 	[SM] How are you going to "study the performance gain that L4S
>> give[s]" if you do not compare it with the best of class alternatives? I am truly
>> puzzled.
>> 
>>> Flow aware AQMs with RTT estimates as metadata in the packets is outside
>> the scope as it would require packet inspection, which is not feasible if queues
>> build up on the RLC layer in the 3GPP stack.
>> 
>> 	[SM] Fair enough. What is this comparison intended to show us then?
>> 
>> As far as I can see you paired an application designed for 1/p-type congestion
>> feed-back with an 1/sqrt(p)-type AQM that was also set for sub-optimal RTT and
>> latency target for the test path. And lo and behold, the application does
>> "better*" for the 1/p-type AQM (with lower latency target; I assume that  L4S
>> ramp-marker (Th_low=2ms, Th_high=10ms) was carefully selected to match
>> what SCReAM expects). IMHO that simply demonstrates, that in communication
>> it pays if sender and receiver of a symbol (CE here) assign the same meaning to
>> it.
>> 
>> But that can not be it, sohat am I missing here?
>> 
>> Best Regards
>> 	Sebastian
>> 
>> 
>> *) Assuming one buys into your definition of better, in which (instantaneous)
>> queueing delay is valued over video quality. From a network operators
>> perspective that seems a valid position
>> 
>> 
>> 
>> 
>>> 
>>> /Ingemar
>>> 
>>>> -----Original Message-----
>>>> From: Sebastian Moeller <moeller0@gmx.de>
>>>> Sent: den 10 mars 2020 10:45
>>>> To: Ingemar Johansson S
>>>> <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>
>>>> Cc: tsvwg@ietf.org; Ingemar Johansson S
>>>> <ingemar.s.johansson@ericsson.com>; iccrg@irtf.org
>>>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>>>> 
>>>> Hi Ingemar,
>>>> 
>>>> thanks for posting this interesting piece of data!
>>>> 
>>>>> On Mar 10, 2020, at 09:02, Ingemar Johansson S
>>>> <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org> wrote:
>>>>> 
>>>>> Hi
>>>>> 
>>>>> I recently updated the readme on the SCReAM github with a comparison
>>>>> with
>>>> SCReAM in three different settings
>>>>> 	• No ECN
>>>>> 	• CoDel ECN
>>>>> 	• L4S
>>>>> https://protect2.fireeye.com/v1/url?k=63019d27-3f884737-6301ddbc-0cc
>>>>> 47
>>>>> ad93e2a-489fa99c3277fb8a&q=1&e=5aab95a7-4aab-4a64-99a5-
>>>> 5b55606e303b&u=
>>>>> https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream%23ecn-
>> explicit-
>>>> co
>>>>> ngestion-notification
>>>>> 
>>>>> Even though it is more than a magnitude difference in queue delay
>>>>> between CoDel-ECN and L4S,
>>>> 
>>>> 
>>>> 	[SM] So, in this simulations of a 20ms path, SCReAM over L4S gives
>>>> ~10 times less queueing delay, but also only ~2 less bandwidth
>>>> compared to SCReAM over codel. You describe this as "L4S reduces the
>>>> delay considerably more" and "L4S gives a somewhat lower media rate".
>>>> I wonder how many end-users would tradeoff these 25ms in queueing
>>>> delay against the decrease in video quality from halving the bitrate?
>>>> Could you repeat the Codel test with interval set to 20 and target to
>>>> 1ms, please?
>>>> 
>>>> If that improves things considerably it would argue for embedding the
>>>> current best RTT estimate into SCReAM packets, so an AQM could tailor
>>>> its signaling better to individual flow properties (and yes, that
>>>> will require a flow-aware AQM).
>>>> 
>>>> 
>>>> 
>>>>> it is fair to say that these simple simulations should of course be
>>>>> seen as just a
>>>> snapshot.
>>>> 
>>>> 	[SM] Fair enough.
>>>> 
>>>>> We hope to present some more simulations with 5G access, and not
>>>>> just
>>>> simple bottlenecks with one flow, after the summer.
>>>> 
>>>> 	[Looking] forward to that.
>>>> 
>>>>> 
>>>>> Meanwhile, the SCReAM code on github is freely available for anyone
>>>>> who
>>>> wish to make more experiments.
>>>>> 
>>>>> /Ingemar
>>>>> ================================
>>>>> Ingemar Johansson  M.Sc.
>>>>> Master Researcher
>>>>> 
>>>>> Ericsson Research
>>>>> RESEARCHER
>>>>> GFTL ER NAP NCM Netw Proto & E2E Perf Labratoriegränd 11
>>>>> 971 28, Luleå, Sweden
>>>>> Phone +46-1071 43042
>>>>> SMS/MMS +46-73 078 3289
>>>>> ingemar.s.johansson@ericsson.com
>>>>> www.ericsson.com
>>>>> 
>>>>> Reality, is the only thing… That’s real!
>>>>>     James Halliday, Ready Player One
>>>>> =================================
>