Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S

[Sebastian Moeller <[e-mail]>]

Hi Ingemar,

Thanks for following up with interesting data.


> On Mar 10, 2020, at 22:04, Ingemar Johansson S <[e-mail] <mailto:[e-mail]>> wrote:
> 
> Hi
> Attached is a simulation of the same simple bottleneck case with target=1ms and interval=20ms. 
> Two ECN beta values : 0.8 and 0.6. A larger backoff (0.6) does actually not improve things as the CWND reduction leads to that packets from the video coder are queued up in the RTP queue. 
> As you can see there is still a way to go to reach the L4S delays. 

	[SM] I see. But I also note that the way you describe scream's operation as being a bad match for rfc3168's one mark per RTT operations mode, you really want to track the instantaneous congestion state, so more instantaneous signaling is a better match for your application. 

> 
> Perhaps the SCReAM's response to CoDel - ECN marks

	[SM] Question, in your test-bed/simulations, is there a linux host with a codel qdisc, or is this using seperate simulation code like ns-3?


> can be optimized further, don't know, but I already spent a considerable time to try and get where the code is now, and I spent a lot more time on this than I spent on the response to the L4S signal. 

	[SM] Yes, as far as I can see scream really wants high resolution (low-magnitude) congestion information and hence seems like a perfect match for the higher signaling rates of 1/p-type congestion signaling, like L4S or (dare I say it SCE).


> My impression is that it is the fractional congestion signal with L4S that makes it easier to get a good algorithm behavior (and I definitely believe that there is room for improvement)

	[SM] I would naively guess that is is both the higher signaling rate and the magnitude that helps, your ECN experiments with 90, 80, and 60% backoff indicate that it does not seem to be the magnitude alone.

> 
> And as said earlier, the SCReAM code is freely available to experiment with and occasionally it is quite fun (https://www.youtube.com/watch?v=eU1crtEvMv4 <https://www.youtube.com/watch?v=eU1crtEvMv4> ). 

	[SM] So to run tests like yours, what else is needed in addition to the contents of the scream repository (and matlab/octave to generate the plots)?

Best Regards
	Sebastian


> 
> 
> /Ingemar
> 
>> -----Original Message-----
>> From: Sebastian Moeller <[e-mail] <mailto:[e-mail]>>
>> Sent: den 10 mars 2020 16:29
>> To: Ingemar Johansson S <[e-mail]>
>> Cc: Ingemar Johansson S
>> <ingemar.s.johansson=[e-mail]>; [e-mail];
>> [e-mail]
>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>> 
>> Hi Ingemar,
>> 
>> 
>> 
>>> On Mar 10, 2020, at 13:37, Ingemar Johansson S
>> <[e-mail]> wrote:
>>> 
>>> Hi
>>> 
>>> The SCReAM code is freely available on
>> https://protect2.fireeye.com/v1/url?k=3fa4aff0-6370a99d-3fa4ef6b-
>> 867011091b6c-c4d35656f5c9b268&q=1&e=16e2b451-36c7-4814-af9d-
>> 4acb21ac792b&u=https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream
>> for anybody interested to run their own experiment with whatever AQM/ECN
>> configuration.
>> 
>> 	[SM] Well, if you are not interested in figuring out what is happening
>> with with rfc3168 ECN ans SCReAM that is your prerogative...
>> 
>> 
>>> Please note that SCReAM is configured in an L4S mode when the network
>>> AQM does L4S marking (mimicking ECT(1)). For CoDel-ECN however, SCReAM
>>> runs in normal ECN mode with a beta of 0.8 (=20% reduction on CWND for
>>> each congestion event)
>> 
>> 	[SM] Okay, that is a far cry from Reno's 50% though, have you tested
>> different reduction percentages or how did you arrive at 20%?
>> But is it really 0.8? ScreamTx.h has the following:
>> 
>> // CWND scale factor upon ECN-CE event
>> static const float kEcnCeBeta = 0.9f;
>> 
>> I am probably missing something somewhere...
>> 
>>> 
>>> I tried also with other different ramp markers (1ms/10ms),
>> (2ms/10ms),(5ms/15ms).
>> 
>> 	[SM] That means for L4S, or do you mean you tried different values for
>> Codel's target ans ce_threshold parameters?
>> 
>> 
>>> There are slight variations  in throughput and latency but not dramatic.
>> 
>> 	[SM] Well, these are also pretty slight variations in numerical values, at
>> least if compared to codel's default interval of 100ms...
>> 
>>> And truth to be told, the ECN behavior is better tuned in the code than the L4S
>> behavior.
>> 
>> 	[SM] For a defintion of better that ignores that you seem to be much
>> happier with the results L4S gives you.
>> 
>>> There is room for improvement as regards to the L4S behavior (for instance
>> faster ramp-up) and it may well be the case that SCReAM is completely scrapped
>> in favor of new designs.
>>> But the bottomline, the L4S thresholds and L4S code is not carefully picked to
>> show a good performance.
>> 
>> 	[SM] So, you tried different L4S parameters (and found the to have
>> similar performance) but for rfc3168 ECN and Codel you only tried the default
>> parameters?
>> 
>> Best Regards
>> 	Sebastian
>> 
>>> 
>>> /Ingemar
>>> 
>>>> -----Original Message-----
>>>> From: Sebastian Moeller <[e-mail]>
>>>> Sent: den 10 mars 2020 11:28
>>>> To: Ingemar Johansson S <[e-mail]>
>>>> Cc: Ingemar Johansson S
>>>> <ingemar.s.johansson=[e-mail]>; [e-mail];
>>>> [e-mail]
>>>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>>>> 
>>>> Hi Ingemar,
>>>> 
>>>> 
>>>>> On Mar 10, 2020, at 11:07, Ingemar Johansson S
>>>> <[e-mail]> wrote:
>>>>> 
>>>>> Hi
>>>>> 
>>>>> For the future studies we will only focus on L4S as the scope is to
>>>>> study the
>>>> performance gain that L4S give for instance for AR/VR, gaming and
>>>> remote control applications.
>>>> 
>>>> 	[SM] How are you going to "study the performance gain that L4S
>>>> give[s]" if you do not compare it with the best of class
>>>> alternatives? I am truly puzzled.
>>>> 
>>>>> Flow aware AQMs with RTT estimates as metadata in the packets is
>>>>> outside
>>>> the scope as it would require packet inspection, which is not
>>>> feasible if queues build up on the RLC layer in the 3GPP stack.
>>>> 
>>>> 	[SM] Fair enough. What is this comparison intended to show us then?
>>>> 
>>>> As far as I can see you paired an application designed for 1/p-type
>>>> congestion feed-back with an 1/sqrt(p)-type AQM that was also set for
>>>> sub-optimal RTT and latency target for the test path. And lo and
>>>> behold, the application does "better*" for the 1/p-type AQM (with
>>>> lower latency target; I assume that  L4S ramp-marker (Th_low=2ms,
>>>> Th_high=10ms) was carefully selected to match what SCReAM expects).
>>>> IMHO that simply demonstrates, that in communication it pays if
>>>> sender and receiver of a symbol (CE here) assign the same meaning to it.
>>>> 
>>>> But that can not be it, sohat am I missing here?
>>>> 
>>>> Best Regards
>>>> 	Sebastian
>>>> 
>>>> 
>>>> *) Assuming one buys into your definition of better, in which
>>>> (instantaneous) queueing delay is valued over video quality. From a
>>>> network operators perspective that seems a valid position
>>>> 
>>>> 
>>>> 
>>>> 
>>>>> 
>>>>> /Ingemar
>>>>> 
>>>>>> -----Original Message-----
>>>>>> From: Sebastian Moeller <[e-mail]>
>>>>>> Sent: den 10 mars 2020 10:45
>>>>>> To: Ingemar Johansson S
>>>>>> <ingemar.s.johansson=[e-mail]>
>>>>>> Cc: [e-mail]; Ingemar Johansson S
>>>>>> <[e-mail]>; [e-mail]
>>>>>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>>>>>> 
>>>>>> Hi Ingemar,
>>>>>> 
>>>>>> thanks for posting this interesting piece of data!
>>>>>> 
>>>>>>> On Mar 10, 2020, at 09:02, Ingemar Johansson S
>>>>>> <ingemar.s.johansson=[e-mail]> wrote:
>>>>>>> 
>>>>>>> Hi
>>>>>>> 
>>>>>>> I recently updated the readme on the SCReAM github with a
>>>>>>> comparison with
>>>>>> SCReAM in three different settings
>>>>>>> 	• No ECN
>>>>>>> 	• CoDel ECN
>>>>>>> 	• L4S
>>>>>>> https://protect2.fireeye.com/v1/url?k=63019d27-3f884737-6301ddbc-0
>>>>>>> cc
>>>>>>> 47
>>>>>>> ad93e2a-489fa99c3277fb8a&q=1&e=5aab95a7-4aab-4a64-99a5-
>>>>>> 5b55606e303b&u=
>>>>>>> https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream%23ecn-
>>>> explicit-
>>>>>> co
>>>>>>> ngestion-notification
>>>>>>> 
>>>>>>> Even though it is more than a magnitude difference in queue delay
>>>>>>> between CoDel-ECN and L4S,
>>>>>> 
>>>>>> 
>>>>>> 	[SM] So, in this simulations of a 20ms path, SCReAM over L4S gives
>>>>>> ~10 times less queueing delay, but also only ~2 less bandwidth
>>>>>> compared to SCReAM over codel. You describe this as "L4S reduces
>>>>>> the delay considerably more" and "L4S gives a somewhat lower media
>> rate".
>>>>>> I wonder how many end-users would tradeoff these 25ms in queueing
>>>>>> delay against the decrease in video quality from halving the bitrate?
>>>>>> Could you repeat the Codel test with interval set to 20 and target
>>>>>> to 1ms, please?
>>>>>> 
>>>>>> If that improves things considerably it would argue for embedding
>>>>>> the current best RTT estimate into SCReAM packets, so an AQM could
>>>>>> tailor its signaling better to individual flow properties (and yes,
>>>>>> that will require a flow-aware AQM).
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> it is fair to say that these simple simulations should of course
>>>>>>> be seen as just a
>>>>>> snapshot.
>>>>>> 
>>>>>> 	[SM] Fair enough.
>>>>>> 
>>>>>>> We hope to present some more simulations with 5G access, and not
>>>>>>> just
>>>>>> simple bottlenecks with one flow, after the summer.
>>>>>> 
>>>>>> 	[Looking] forward to that.
>>>>>> 
>>>>>>> 
>>>>>>> Meanwhile, the SCReAM code on github is freely available for
>>>>>>> anyone who
>>>>>> wish to make more experiments.
>>>>>>> 
>>>>>>> /Ingemar
>>>>>>> ================================
>>>>>>> Ingemar Johansson  M.Sc.
>>>>>>> Master Researcher
>>>>>>> 
>>>>>>> Ericsson Research
>>>>>>> RESEARCHER
>>>>>>> GFTL ER NAP NCM Netw Proto & E2E Perf Labratoriegränd 11
>>>>>>> 971 28, Luleå, Sweden
>>>>>>> Phone [phone]
>>>>>>> SMS/MMS [phone]
>>>>>>> [e-mail]
>>>>>>> www.ericsson.com
>>>>>>> 
>>>>>>> Reality, is the only thing… That’s real!
>>>>>>>    James Halliday, Ready Player One
>>>>>>> =================================
>>> 
> 
>