Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S

[Sebastian Moeller <[e-mail]>]

Hi Ingemar,



> On Mar 10, 2020, at 13:37, Ingemar Johansson S <[e-mail]> wrote:
> 
> Hi
> 
> The SCReAM code is freely available on https://github.com/EricssonResearch/scream for anybody interested to run their own experiment with whatever AQM/ECN configuration. 

	[SM] Well, if you are not interested in figuring out what is happening with with rfc3168 ECN ans SCReAM that is your prerogative... 


> Please note that SCReAM is configured in an L4S mode when the network AQM does L4S marking (mimicking ECT(1)). For CoDel-ECN however, SCReAM runs in normal ECN mode with a beta of 0.8 (=20% reduction on CWND for each congestion event)

	[SM] Okay, that is a far cry from Reno's 50% though, have you tested different reduction percentages or how did you arrive at 20%?
But is it really 0.8? ScreamTx.h has the following:

// CWND scale factor upon ECN-CE event
static const float kEcnCeBeta = 0.9f;

I am probably missing something somewhere...

> 
> I tried also with other different ramp markers (1ms/10ms), (2ms/10ms),(5ms/15ms).

	[SM] That means for L4S, or do you mean you tried different values for Codel's target ans ce_threshold parameters?


> There are slight variations  in throughput and latency but not dramatic.

	[SM] Well, these are also pretty slight variations in numerical values, at least if compared to codel's default interval of 100ms...

> And truth to be told, the ECN behavior is better tuned in the code than the L4S behavior. 

	[SM] For a defintion of better that ignores that you seem to be much happier with the results L4S gives you.

> There is room for improvement as regards to the L4S behavior (for instance faster ramp-up) and it may well be the case that SCReAM is completely scrapped in favor of new designs. 
> But the bottomline, the L4S thresholds and L4S code is not carefully picked to show a good performance.

	[SM] So, you tried different L4S parameters (and found the to have similar performance) but for rfc3168 ECN and Codel you only tried the default parameters?

Best Regards
	Sebastian

> 
> /Ingemar
> 
>> -----Original Message-----
>> From: Sebastian Moeller <[e-mail]>
>> Sent: den 10 mars 2020 11:28
>> To: Ingemar Johansson S <[e-mail]>
>> Cc: Ingemar Johansson S
>> <ingemar.s.johansson=[e-mail]>; [e-mail];
>> [e-mail]
>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>> 
>> Hi Ingemar,
>> 
>> 
>>> On Mar 10, 2020, at 11:07, Ingemar Johansson S
>> <[e-mail]> wrote:
>>> 
>>> Hi
>>> 
>>> For the future studies we will only focus on L4S as the scope is to study the
>> performance gain that L4S give for instance for AR/VR, gaming and remote
>> control applications.
>> 
>> 	[SM] How are you going to "study the performance gain that L4S
>> give[s]" if you do not compare it with the best of class alternatives? I am truly
>> puzzled.
>> 
>>> Flow aware AQMs with RTT estimates as metadata in the packets is outside
>> the scope as it would require packet inspection, which is not feasible if queues
>> build up on the RLC layer in the 3GPP stack.
>> 
>> 	[SM] Fair enough. What is this comparison intended to show us then?
>> 
>> As far as I can see you paired an application designed for 1/p-type congestion
>> feed-back with an 1/sqrt(p)-type AQM that was also set for sub-optimal RTT and
>> latency target for the test path. And lo and behold, the application does
>> "better*" for the 1/p-type AQM (with lower latency target; I assume that  L4S
>> ramp-marker (Th_low=2ms, Th_high=10ms) was carefully selected to match
>> what SCReAM expects). IMHO that simply demonstrates, that in communication
>> it pays if sender and receiver of a symbol (CE here) assign the same meaning to
>> it.
>> 
>> But that can not be it, sohat am I missing here?
>> 
>> Best Regards
>> 	Sebastian
>> 
>> 
>> *) Assuming one buys into your definition of better, in which (instantaneous)
>> queueing delay is valued over video quality. From a network operators
>> perspective that seems a valid position
>> 
>> 
>> 
>> 
>>> 
>>> /Ingemar
>>> 
>>>> -----Original Message-----
>>>> From: Sebastian Moeller <[e-mail]>
>>>> Sent: den 10 mars 2020 10:45
>>>> To: Ingemar Johansson S
>>>> <ingemar.s.johansson=[e-mail]>
>>>> Cc: [e-mail]; Ingemar Johansson S
>>>> <[e-mail]>; [e-mail]
>>>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>>>> 
>>>> Hi Ingemar,
>>>> 
>>>> thanks for posting this interesting piece of data!
>>>> 
>>>>> On Mar 10, 2020, at 09:02, Ingemar Johansson S
>>>> <ingemar.s.johansson=[e-mail]> wrote:
>>>>> 
>>>>> Hi
>>>>> 
>>>>> I recently updated the readme on the SCReAM github with a comparison
>>>>> with
>>>> SCReAM in three different settings
>>>>> 	• No ECN
>>>>> 	• CoDel ECN
>>>>> 	• L4S
>>>>> https://protect2.fireeye.com/v1/url?k=63019d27-3f884737-6301ddbc-0cc
>>>>> 47
>>>>> ad93e2a-489fa99c3277fb8a&q=1&e=5aab95a7-4aab-4a64-99a5-
>>>> 5b55606e303b&u=
>>>>> https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream%23ecn-
>> explicit-
>>>> co
>>>>> ngestion-notification
>>>>> 
>>>>> Even though it is more than a magnitude difference in queue delay
>>>>> between CoDel-ECN and L4S,
>>>> 
>>>> 
>>>> 	[SM] So, in this simulations of a 20ms path, SCReAM over L4S gives
>>>> ~10 times less queueing delay, but also only ~2 less bandwidth
>>>> compared to SCReAM over codel. You describe this as "L4S reduces the
>>>> delay considerably more" and "L4S gives a somewhat lower media rate".
>>>> I wonder how many end-users would tradeoff these 25ms in queueing
>>>> delay against the decrease in video quality from halving the bitrate?
>>>> Could you repeat the Codel test with interval set to 20 and target to
>>>> 1ms, please?
>>>> 
>>>> If that improves things considerably it would argue for embedding the
>>>> current best RTT estimate into SCReAM packets, so an AQM could tailor
>>>> its signaling better to individual flow properties (and yes, that
>>>> will require a flow-aware AQM).
>>>> 
>>>> 
>>>> 
>>>>> it is fair to say that these simple simulations should of course be
>>>>> seen as just a
>>>> snapshot.
>>>> 
>>>> 	[SM] Fair enough.
>>>> 
>>>>> We hope to present some more simulations with 5G access, and not
>>>>> just
>>>> simple bottlenecks with one flow, after the summer.
>>>> 
>>>> 	[Looking] forward to that.
>>>> 
>>>>> 
>>>>> Meanwhile, the SCReAM code on github is freely available for anyone
>>>>> who
>>>> wish to make more experiments.
>>>>> 
>>>>> /Ingemar
>>>>> ================================
>>>>> Ingemar Johansson  M.Sc.
>>>>> Master Researcher
>>>>> 
>>>>> Ericsson Research
>>>>> RESEARCHER
>>>>> GFTL ER NAP NCM Netw Proto & E2E Perf Labratoriegränd 11
>>>>> 971 28, Luleå, Sweden
>>>>> Phone [phone]
>>>>> SMS/MMS [phone]
>>>>> [e-mail]
>>>>> www.ericsson.com
>>>>> 
>>>>> Reality, is the only thing… That’s real!
>>>>>     James Halliday, Ready Player One
>>>>> =================================
>