Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S

Sebastian Moeller <moeller0@gmx.de> Thu, 12 March 2020 11:52 UTC

Return-Path: <moeller0@gmx.de>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A3AD83A0E1A; Thu, 12 Mar 2020 04:52:08 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.647
X-Spam-Level:
X-Spam-Status: No, score=-1.647 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=gmx.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S3BRA8XO2Gxz; Thu, 12 Mar 2020 04:52:06 -0700 (PDT)
Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D07233A0E2D; Thu, 12 Mar 2020 04:52:03 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1584013918; bh=8ETsVEzueaWIPe/ZAQ1QiuDYC1wEOc9ZJQtRIdW6ayU=; h=X-UI-Sender-Class:From:Subject:Date:In-Reply-To:Cc:To:References; b=hNloxoAc9vlOoaYXY76NOtruhfY6+v7cME9xWPj3izMg0mRvQx5iWxjTex3j/o9dm uOCKsWfsozriNOoI0hvP3PKW8houckJAkawUCIdNS2BN7irDRzKbo01z+tTWJY8GYD gXydYK4y8kTEKYyIMMf3g3XlqfxEpI4gt8bAYEyw=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from [10.11.12.22] ([134.76.241.253]) by mail.gmx.com (mrgmx104 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MD9XF-1j3no710PR-009CEQ; Thu, 12 Mar 2020 12:51:58 +0100
From: Sebastian Moeller <moeller0@gmx.de>
Message-Id: <8EBC3700-B93B-4077-A052-43CBF1A8B3E5@gmx.de>
Content-Type: multipart/alternative; boundary="Apple-Mail=_9B622E0D-C710-458C-98B9-99892DD37E2C"
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.11\))
Date: Thu, 12 Mar 2020 12:51:55 +0100
In-Reply-To: =?utf-8?q?=3CHE1PR07MB4425AA2A7976C1CCF594D3B2C2FF0=40HE1PR07MB?= =?utf-8?q?4425=2Eeurprd07=2Eprod=2Eoutlook=2Ecom=3E?=
Cc: Ingemar Johansson S <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>, "tsvwg@ietf.org" <tsvwg@ietf.org>, "iccrg@irtf.org" <iccrg@irtf.org>
To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
References: =?utf-8?q?=3CHE1PR07MB44251B019947CDB6602B30B2C2FF0=40HE1PR07MB4?= =?utf-8?q?425=2Eeurprd07=2Eprod=2Eoutlook=2Ecom=3E?= =?utf-8?q?=3CA2300F8D-5F87-461E-AD94-8D7B22A6CDF3=40gmx=2Ede=3E_=3CHE1PR07M?= =?utf-8?q?B4425B105AFF56D1566164900C2FF0=40HE1PR07MB4425=2Eeurprd07=2Eprod?= =?utf-8?q?=2Eoutlook=2Ecom=3E?= <1C969A05-A4B7-43E9-B694-3195A2FC086A@gmx.de> =?utf-8?q?=3CHE1PR07MB44255CED94938F9C38515FD6C2FF0=40HE1PR07MB4425=2Eeurpr?= =?utf-8?q?d07=2Eprod=2Eoutlook=2Ecom=3E?= =?utf-8?q?=3CAC10E219-46C2-4345-B98F-71689F788B13=40gmx=2Ede=3E_=3CHE1PR07M?= =?utf-8?q?B4425AA2A7976C1CCF594D3B2C2FF0=40HE1PR07MB4425=2Eeurprd07=2Eprod?= =?utf-8?q?=2Eoutlook=2Ecom=3E?=
X-Mailer: Apple Mail (2.3445.104.11)
X-Provags-ID: V03:K1:ZYDBUOejuIdx6eZ3hGDpQ9ECOW9rTYvF2x5LvClUvOt1kQLSDTp RpAozMyJHpxEkx/6V3iVkITpbOavGf6hjDRRUbwHvDFQHhcVc4Q3O+73UFAewMHvtIYxEeV EslD+BRWoaL5ILzqdomTWhM8mU6BL5loeBFGFjXCfifhS5eagIEMU2Tz4DTwDj+qplhb7eM t+aOph2dkLNKhJaBDMPBw==
X-UI-Out-Filterresults: notjunk:1;V03:K0:12FzWSbXShI=:e+G51O2cW5BcZrKVtgggBr ROYCT0N5PNwESOD+3sWqVd6Szbk6289DjBVP6hQm6/QLVR19izt4UcSHG4dlUlh3B9CWIdgMD jdyjqfz7U6JpKgeNOsf8VoqS1RUcrz6yqrYfj+r+Rm2DtlHhvK+KpE4YkoRYyJZ4dFe5/WSKp zNjZWLvjiQKoPCjLVP9aoCNvJMqlkE5I2MAx35MGeij0cgXeiErfoQMd06muyw530oIWf2YB/ igraknAqPugr+cp1+8oEneuEQOzuxfMZclKy4ZWvUdXOAXkZfff604Xf8Ov85sClccKO6YEU5 XJrWUi38PUNcIJyN1bGgHUMy+HmD5TDVjDOZF53a5nFn4/8SavyjU/3zPhV+dYcL8+E/NcNAl kL1sQqdqzLJ4mFDQ91Er/QCx6kGAPox2mZwLK3dsM1udY6WN8HqZUxXiBAJPiP4fSmaHjddHa PhC7KGayvTMZIUPVkTZnYmmQDFubwkXjqyOHYuLJ65DTH0ZwL2gam9hlWMzSjM3lhEJbcC9WC rPJtYNhKo7Z5Os/VFqnMw62Ga3M5kZQnr9BUVNbVJbKmP9OlJtgTUgJKSHsz6Fed03fVLGShS ZKNHrbhY2tWWwWNBU4M4yQ9ZA/U3MveMrrMWHcIp4xfCbz9Uu2tpA3tthLPPuOFBHk58ka+si c9ijNldsdn7gqhjWW45LeejbiUFt86Y+CjZcQpwrE/592Dratd8iWothDFCPogtT8UcwGltvQ 5R9geE29TVJVCb2KOvlDBXHRvDFV+MOqtm9LxjjGefFqBeNMCADuroxVQ27wPw6VDZzRLHNjQ LOMSsvJ2jaS5NE+/LbTdNnnJJMBcIcFVN32dxZGlwNxPqKiyqF3bDwxDSsJChXRNzrGxBuMdY TTw5f4czt1sVFz4pVoOn3y5mkOGYW70t6t34wR+2BkHz8QOJhRd9QYbiPq/0JGEk3EPVtBBmP ggC8nEIMnsT3OLX1JyThCJgnGqtBRRcYQLMbZZmnV85ApFhZ+YBap7pJpfksY13jaBejOR3yb QUeL0minhetzFlOd1ysTmgGFh8SR9vlphrgVRA+efsRfB0W8u/N2nrvFS3Yd+C55H3ysjae40 kSQszv/6HeKlj+JNHTLCH8qw6B5SebNxrqwx7FVOt++kFF8md4amhtKuglvNuyT84rZy4ruS9 uf8MQI2Ss0V6NQYYEdFboIDGMMf2M25G9WrZ5JosGWi2uOebmnFtPoMKp8uaVbqE6oZ5DNlE9 n4AAbo/oitkhswAIX5lg4nvCfb8dWNc4t+0rOzjuM6eGF59iEsjlgGMJegNR/EBYZz2JYyd5A z2eNfrZzJIxLxXFsLGsCSp9afpnhdw==
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/ecyDHfoi_PPkKdH7lCeLAJc99dY>
Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Mar 2020 11:52:09 -0000

Hi Ingemar,

Thanks for following up with interesting data.


> On Mar 10, 2020, at 22:04, Ingemar Johansson S <ingemar.s.johansson@ericsson.com <mailto:ingemar.s.johansson@ericsson.com>> wrote:
> 
> Hi
> Attached is a simulation of the same simple bottleneck case with target=1ms and interval=20ms. 
> Two ECN beta values : 0.8 and 0.6. A larger backoff (0.6) does actually not improve things as the CWND reduction leads to that packets from the video coder are queued up in the RTP queue. 
> As you can see there is still a way to go to reach the L4S delays. 

	[SM] I see. But I also note that the way you describe scream's operation as being a bad match for rfc3168's one mark per RTT operations mode, you really want to track the instantaneous congestion state, so more instantaneous signaling is a better match for your application. 

> 
> Perhaps the SCReAM's response to CoDel - ECN marks

	[SM] Question, in your test-bed/simulations, is there a linux host with a codel qdisc, or is this using seperate simulation code like ns-3?


> can be optimized further, don't know, but I already spent a considerable time to try and get where the code is now, and I spent a lot more time on this than I spent on the response to the L4S signal. 

	[SM] Yes, as far as I can see scream really wants high resolution (low-magnitude) congestion information and hence seems like a perfect match for the higher signaling rates of 1/p-type congestion signaling, like L4S or (dare I say it SCE).


> My impression is that it is the fractional congestion signal with L4S that makes it easier to get a good algorithm behavior (and I definitely believe that there is room for improvement)

	[SM] I would naively guess that is is both the higher signaling rate and the magnitude that helps, your ECN experiments with 90, 80, and 60% backoff indicate that it does not seem to be the magnitude alone.

> 
> And as said earlier, the SCReAM code is freely available to experiment with and occasionally it is quite fun (https://www.youtube.com/watch?v=eU1crtEvMv4 <https://www.youtube.com/watch?v=eU1crtEvMv4> ). 

	[SM] So to run tests like yours, what else is needed in addition to the contents of the scream repository (and matlab/octave to generate the plots)?

Best Regards
	Sebastian


> 
> 
> /Ingemar
> 
>> -----Original Message-----
>> From: Sebastian Moeller <moeller0@gmx.de <mailto:moeller0@gmx.de>>
>> Sent: den 10 mars 2020 16:29
>> To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
>> Cc: Ingemar Johansson S
>> <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>rg>; tsvwg@ietf.org;
>> iccrg@irtf.org
>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>> 
>> Hi Ingemar,
>> 
>> 
>> 
>>> On Mar 10, 2020, at 13:37, Ingemar Johansson S
>> <ingemar.s.johansson@ericsson.com> wrote:
>>> 
>>> Hi
>>> 
>>> The SCReAM code is freely available on
>> https://protect2.fireeye.com/v1/url?k=3fa4aff0-6370a99d-3fa4ef6b-
>> 867011091b6c-c4d35656f5c9b268&q=1&e=16e2b451-36c7-4814-af9d-
>> 4acb21ac792b&u=https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream
>> for anybody interested to run their own experiment with whatever AQM/ECN
>> configuration.
>> 
>> 	[SM] Well, if you are not interested in figuring out what is happening
>> with with rfc3168 ECN ans SCReAM that is your prerogative...
>> 
>> 
>>> Please note that SCReAM is configured in an L4S mode when the network
>>> AQM does L4S marking (mimicking ECT(1)). For CoDel-ECN however, SCReAM
>>> runs in normal ECN mode with a beta of 0.8 (=20% reduction on CWND for
>>> each congestion event)
>> 
>> 	[SM] Okay, that is a far cry from Reno's 50% though, have you tested
>> different reduction percentages or how did you arrive at 20%?
>> But is it really 0.8? ScreamTx.h has the following:
>> 
>> // CWND scale factor upon ECN-CE event
>> static const float kEcnCeBeta = 0.9f;
>> 
>> I am probably missing something somewhere...
>> 
>>> 
>>> I tried also with other different ramp markers (1ms/10ms),
>> (2ms/10ms),(5ms/15ms).
>> 
>> 	[SM] That means for L4S, or do you mean you tried different values for
>> Codel's target ans ce_threshold parameters?
>> 
>> 
>>> There are slight variations  in throughput and latency but not dramatic.
>> 
>> 	[SM] Well, these are also pretty slight variations in numerical values, at
>> least if compared to codel's default interval of 100ms...
>> 
>>> And truth to be told, the ECN behavior is better tuned in the code than the L4S
>> behavior.
>> 
>> 	[SM] For a defintion of better that ignores that you seem to be much
>> happier with the results L4S gives you.
>> 
>>> There is room for improvement as regards to the L4S behavior (for instance
>> faster ramp-up) and it may well be the case that SCReAM is completely scrapped
>> in favor of new designs.
>>> But the bottomline, the L4S thresholds and L4S code is not carefully picked to
>> show a good performance.
>> 
>> 	[SM] So, you tried different L4S parameters (and found the to have
>> similar performance) but for rfc3168 ECN and Codel you only tried the default
>> parameters?
>> 
>> Best Regards
>> 	Sebastian
>> 
>>> 
>>> /Ingemar
>>> 
>>>> -----Original Message-----
>>>> From: Sebastian Moeller <moeller0@gmx.de>
>>>> Sent: den 10 mars 2020 11:28
>>>> To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
>>>> Cc: Ingemar Johansson S
>>>> <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>rg>; tsvwg@ietf.org;
>>>> iccrg@irtf.org
>>>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>>>> 
>>>> Hi Ingemar,
>>>> 
>>>> 
>>>>> On Mar 10, 2020, at 11:07, Ingemar Johansson S
>>>> <ingemar.s.johansson@ericsson.com> wrote:
>>>>> 
>>>>> Hi
>>>>> 
>>>>> For the future studies we will only focus on L4S as the scope is to
>>>>> study the
>>>> performance gain that L4S give for instance for AR/VR, gaming and
>>>> remote control applications.
>>>> 
>>>> 	[SM] How are you going to "study the performance gain that L4S
>>>> give[s]" if you do not compare it with the best of class
>>>> alternatives? I am truly puzzled.
>>>> 
>>>>> Flow aware AQMs with RTT estimates as metadata in the packets is
>>>>> outside
>>>> the scope as it would require packet inspection, which is not
>>>> feasible if queues build up on the RLC layer in the 3GPP stack.
>>>> 
>>>> 	[SM] Fair enough. What is this comparison intended to show us then?
>>>> 
>>>> As far as I can see you paired an application designed for 1/p-type
>>>> congestion feed-back with an 1/sqrt(p)-type AQM that was also set for
>>>> sub-optimal RTT and latency target for the test path. And lo and
>>>> behold, the application does "better*" for the 1/p-type AQM (with
>>>> lower latency target; I assume that  L4S ramp-marker (Th_low=2ms,
>>>> Th_high=10ms) was carefully selected to match what SCReAM expects).
>>>> IMHO that simply demonstrates, that in communication it pays if
>>>> sender and receiver of a symbol (CE here) assign the same meaning to it.
>>>> 
>>>> But that can not be it, sohat am I missing here?
>>>> 
>>>> Best Regards
>>>> 	Sebastian
>>>> 
>>>> 
>>>> *) Assuming one buys into your definition of better, in which
>>>> (instantaneous) queueing delay is valued over video quality. From a
>>>> network operators perspective that seems a valid position
>>>> 
>>>> 
>>>> 
>>>> 
>>>>> 
>>>>> /Ingemar
>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Sebastian Moeller <moeller0@gmx.de>
>>>>>> Sent: den 10 mars 2020 10:45
>>>>>> To: Ingemar Johansson S
>>>>>> <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>
>>>>>> Cc: tsvwg@ietf.org; Ingemar Johansson S
>>>>>> <ingemar.s.johansson@ericsson.com>om>; iccrg@irtf.org
>>>>>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>>>>>> 
>>>>>> Hi Ingemar,
>>>>>> 
>>>>>> thanks for posting this interesting piece of data!
>>>>>> 
>>>>>>> On Mar 10, 2020, at 09:02, Ingemar Johansson S
>>>>>> <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org> wrote:
>>>>>>> 
>>>>>>> Hi
>>>>>>> 
>>>>>>> I recently updated the readme on the SCReAM github with a
>>>>>>> comparison with
>>>>>> SCReAM in three different settings
>>>>>>> 	• No ECN
>>>>>>> 	• CoDel ECN
>>>>>>> 	• L4S
>>>>>>> https://protect2.fireeye.com/v1/url?k=63019d27-3f884737-6301ddbc-0
>>>>>>> cc
>>>>>>> 47
>>>>>>> ad93e2a-489fa99c3277fb8a&q=1&e=5aab95a7-4aab-4a64-99a5-
>>>>>> 5b55606e303b&u=
>>>>>>> https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream%23ecn-
>>>> explicit-
>>>>>> co
>>>>>>> ngestion-notification
>>>>>>> 
>>>>>>> Even though it is more than a magnitude difference in queue delay
>>>>>>> between CoDel-ECN and L4S,
>>>>>> 
>>>>>> 
>>>>>> 	[SM] So, in this simulations of a 20ms path, SCReAM over L4S gives
>>>>>> ~10 times less queueing delay, but also only ~2 less bandwidth
>>>>>> compared to SCReAM over codel. You describe this as "L4S reduces
>>>>>> the delay considerably more" and "L4S gives a somewhat lower media
>> rate".
>>>>>> I wonder how many end-users would tradeoff these 25ms in queueing
>>>>>> delay against the decrease in video quality from halving the bitrate?
>>>>>> Could you repeat the Codel test with interval set to 20 and target
>>>>>> to 1ms, please?
>>>>>> 
>>>>>> If that improves things considerably it would argue for embedding
>>>>>> the current best RTT estimate into SCReAM packets, so an AQM could
>>>>>> tailor its signaling better to individual flow properties (and yes,
>>>>>> that will require a flow-aware AQM).
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> it is fair to say that these simple simulations should of course
>>>>>>> be seen as just a
>>>>>> snapshot.
>>>>>> 
>>>>>> 	[SM] Fair enough.
>>>>>> 
>>>>>>> We hope to present some more simulations with 5G access, and not
>>>>>>> just
>>>>>> simple bottlenecks with one flow, after the summer.
>>>>>> 
>>>>>> 	[Looking] forward to that.
>>>>>> 
>>>>>>> 
>>>>>>> Meanwhile, the SCReAM code on github is freely available for
>>>>>>> anyone who
>>>>>> wish to make more experiments.
>>>>>>> 
>>>>>>> /Ingemar
>>>>>>> ================================
>>>>>>> Ingemar Johansson  M.Sc.
>>>>>>> Master Researcher
>>>>>>> 
>>>>>>> Ericsson Research
>>>>>>> RESEARCHER
>>>>>>> GFTL ER NAP NCM Netw Proto & E2E Perf Labratoriegränd 11
>>>>>>> 971 28, Luleå, Sweden
>>>>>>> Phone +46-1071 43042
>>>>>>> SMS/MMS +46-73 078 3289
>>>>>>> ingemar.s.johansson@ericsson.com
>>>>>>> www.ericsson.com
>>>>>>> 
>>>>>>> Reality, is the only thing… That’s real!
>>>>>>>    James Halliday, Ready Player One
>>>>>>> =================================
>>> 
> 
>