Re: [iccrg] [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S

[Sebastian Moeller <[e-mail]>]

Hi Ingemar,

> On Mar 17, 2020, at 14:44, Ingemar Johansson S <[e-mail]> wrote:
> 
> Hi
> 
> It is true that the "CoDel" implementation in the test code is a botched version. The CoDel algorithm implemented in our system simulator that was used to produce these examples is however _the_ CoDel

	[SM] Great! See, I was missing something ;)

Best Regards
	Sebastian


> 
> /Ingemar
> 
>> -----Original Message-----
>> From: Sebastian Moeller <[e-mail]>
>> Sent: den 17 mars 2020 14:16
>> To: Bob Briscoe <[e-mail]>
>> Cc: Ingemar Johansson S
>> <ingemar.s.johansson=[e-mail]>; Ingemar Johansson S
>> <[e-mail]>; [e-mail]; [e-mail]
>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>> 
>> Hi Bob,
>> 
>> 
>>> On Mar 17, 2020, at 02:32, Bob Briscoe <[e-mail]> wrote:
>>> 
>>> Sebastian,
>>> 
>>> I suspect it is the 20ms interval that is the main cause the extra queuing delay
>> with CoDel.
>> 
>> 	[SM] That and the fact that rfc3168 asserts ECE until it receives the CWR
>> from the sender and that on Ingemar's test path takes 20ms, so the temporal
>> fifelity of rfc3168 ECN is simply too low for SCReAM's feed-back fidelity
>> requirements.
>> 
>> 
>>> 
>>> The 20:1 ratio between interval and target was really designed around loss.
>> 
>> 	[SM] I do not think that this actually makes a big difference for rfc3168
>> ECN, CE will be asserted just as a drop would be for NON-ECN packets, and it
>> will take a while for a drop being registered by the receiver and send back to the
>> sender as well. Note how the 5% rule has no bearing on that.
>> 
>>> The need to filter out brief excursions in the queue is only necessary when
>> you're using impairments (losses) as signals.
>> 
>> 	[SM] I disagree, (limited) burst tolerance is not only helpful if loss is
>> involved.
>> 
>> 
>>> If you're using explicit signals, you can signal unfiltered queue variations,
>> because there is no harm having higher ECN marking levels.
>> 
>> 	[SM] Sure, but I strongly believe the bigger issue here is that L4S CE
>> feedback simply has higher temporal fidelity not being limited by the 1RTT ECE-
>>> CWR dance.
>> 
>> 
>>> For that matter, you might as well use Eric Dumazet's simple patch to CoDel
>> that adds immediate ECN marking at a shallow threshold, bypassing all the
>> CoDel machinery, which is really tailored for loss.
>> 
>> 	[SM] Note how I asked Ingemar about whether he tested codel's
>> ce_threshold... But I am still wondering whether Ingemar's test where done with
>> SCReAM's own limited Codel implementation or with a full fledged Linux AQM
>> node in the path (looking at the SCReAM repository makes me believe its own
>> Codel being very vrudimentary and not implementing ce_threshold and as it
>> appears not even the square law drop interval calculation, @Ingemar, I am sure
>> I am reading something wrong here, so please help me see what I am missing*).
>> 
>> 
>> *) I am 99.999% sure I am missing something ;)
>> 
>>> 
>>> Of course, unless you have FQ (which you can't at the RLC layer)
>> 
>> 	[SM] Then use feedback the queueing information from the RLC layer
>> back up to where at least IP packets are handled/seen, akin to say the wifi
>> airtime fairness approach?
>> 
>>> , you would have to work out some other way to ensure coexistence with
>> existing congestion controls. That is left as an exercise for the reader - I
>> wouldn't want to be seen to be pushing any particular solution. But hopefully,
>> through scenarios like this, you might start to understand the set of
>> requirements that led to one solution rather than another.
>> 
>> 	[SM] Well, I believe that neither 1/p type signaling, nor higer temporal
>> feed-back fidelity are really under discussion, we all want those, the question is
>> more what is the best and safest/backward compatiblest way of using those, but
>> I digress (this is not the thread for that discussion ;) )
>> 
>> Best Regards
>> 	Sebastian
>> 
>> 
>>> 
>>> 
>>> Bob
>>> 
>>> On 10/03/2020 21:04, Ingemar Johansson S wrote:
>>>> Hi
>>>> Attached is a simulation of the same simple bottleneck case with target=1ms
>> and interval=20ms.
>>>> Two ECN beta values : 0.8 and 0.6. A larger backoff (0.6) does actually not
>> improve things as the CWND reduction leads to that packets from the video
>> coder are queued up in the RTP queue.
>>>> As you can see there is still a way to go to reach the L4S delays.
>>>> 
>>>> Perhaps the SCReAM's response to CoDel - ECN marks can be optimized
>> further, don't know, but I already spent a considerable time to try and get where
>> the code is now, and I spent a lot more time on this than I spent on the response
>> to the L4S signal.
>>>> My impression is that it is the fractional congestion signal with L4S
>>>> that makes it easier to get a good algorithm behavior (and I
>>>> definitely believe that there is room for improvement)
>>>> 
>>>> And as said earlier, the SCReAM code is freely available to experiment with
>> and occasionally it is quite fun
>> (https://www.youtube.com/watch?v=eU1crtEvMv4 ).
>>>> 
>>>> 
>>>> /Ingemar
>>>> 
>>>>> -----Original Message-----
>>>>> From: Sebastian Moeller <[e-mail]>
>>>>> Sent: den 10 mars 2020 16:29
>>>>> To: Ingemar Johansson S <[e-mail]>
>>>>> Cc: Ingemar Johansson S
>>>>> <ingemar.s.johansson=[e-mail]>; [e-mail];
>>>>> [e-mail]
>>>>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>>>>> 
>>>>> Hi Ingemar,
>>>>> 
>>>>> 
>>>>> 
>>>>>> On Mar 10, 2020, at 13:37, Ingemar Johansson S
>>>>> <[e-mail]> wrote:
>>>>>> Hi
>>>>>> 
>>>>>> The SCReAM code is freely available on
>>>>> https://protect2.fireeye.com/v1/url?k=3fa4aff0-6370a99d-3fa4ef6b-
>>>>> 867011091b6c-c4d35656f5c9b268&q=1&e=16e2b451-36c7-4814-af9d-
>>>>> 
>> 4acb21ac792b&u=https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream
>>>>> for anybody interested to run their own experiment with whatever
>>>>> AQM/ECN configuration.
>>>>> 
>>>>> 	[SM] Well, if you are not interested in figuring out what is
>>>>> happening with with rfc3168 ECN ans SCReAM that is your prerogative...
>>>>> 
>>>>> 
>>>>>> Please note that SCReAM is configured in an L4S mode when the
>>>>>> network AQM does L4S marking (mimicking ECT(1)). For CoDel-ECN
>>>>>> however, SCReAM runs in normal ECN mode with a beta of 0.8 (=20%
>>>>>> reduction on CWND for each congestion event)
>>>>> 	[SM] Okay, that is a far cry from Reno's 50% though, have you
>>>>> tested different reduction percentages or how did you arrive at 20%?
>>>>> But is it really 0.8? ScreamTx.h has the following:
>>>>> 
>>>>> // CWND scale factor upon ECN-CE event static const float kEcnCeBeta
>>>>> = 0.9f;
>>>>> 
>>>>> I am probably missing something somewhere...
>>>>> 
>>>>>> I tried also with other different ramp markers (1ms/10ms),
>>>>> (2ms/10ms),(5ms/15ms).
>>>>> 
>>>>> 	[SM] That means for L4S, or do you mean you tried different values
>>>>> for Codel's target ans ce_threshold parameters?
>>>>> 
>>>>> 
>>>>>> There are slight variations  in throughput and latency but not dramatic.
>>>>> 	[SM] Well, these are also pretty slight variations in numerical
>>>>> values, at least if compared to codel's default interval of 100ms...
>>>>> 
>>>>>> And truth to be told, the ECN behavior is better tuned in the code
>>>>>> than the L4S
>>>>> behavior.
>>>>> 
>>>>> 	[SM] For a defintion of better that ignores that you seem to be
>>>>> much happier with the results L4S gives you.
>>>>> 
>>>>>> There is room for improvement as regards to the L4S behavior (for
>>>>>> instance
>>>>> faster ramp-up) and it may well be the case that SCReAM is
>>>>> completely scrapped in favor of new designs.
>>>>>> But the bottomline, the L4S thresholds and L4S code is not
>>>>>> carefully picked to
>>>>> show a good performance.
>>>>> 
>>>>> 	[SM] So, you tried different L4S parameters (and found the to have
>>>>> similar performance) but for rfc3168 ECN and Codel you only tried
>>>>> the default parameters?
>>>>> 
>>>>> Best Regards
>>>>> 	Sebastian
>>>>> 
>>>>>> /Ingemar
>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: Sebastian Moeller <[e-mail]>
>>>>>>> Sent: den 10 mars 2020 11:28
>>>>>>> To: Ingemar Johansson S <[e-mail]>
>>>>>>> Cc: Ingemar Johansson S
>>>>>>> <ingemar.s.johansson=[e-mail]>;
>>>>>>> [e-mail]; [e-mail]
>>>>>>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>>>>>>> 
>>>>>>> Hi Ingemar,
>>>>>>> 
>>>>>>> 
>>>>>>>> On Mar 10, 2020, at 11:07, Ingemar Johansson S
>>>>>>> <[e-mail]> wrote:
>>>>>>>> Hi
>>>>>>>> 
>>>>>>>> For the future studies we will only focus on L4S as the scope is
>>>>>>>> to study the
>>>>>>> performance gain that L4S give for instance for AR/VR, gaming and
>>>>>>> remote control applications.
>>>>>>> 
>>>>>>> 	[SM] How are you going to "study the performance gain that L4S
>>>>>>> give[s]" if you do not compare it with the best of class
>>>>>>> alternatives? I am truly puzzled.
>>>>>>> 
>>>>>>>> Flow aware AQMs with RTT estimates as metadata in the packets is
>>>>>>>> outside
>>>>>>> the scope as it would require packet inspection, which is not
>>>>>>> feasible if queues build up on the RLC layer in the 3GPP stack.
>>>>>>> 
>>>>>>> 	[SM] Fair enough. What is this comparison intended to show us then?
>>>>>>> 
>>>>>>> As far as I can see you paired an application designed for
>>>>>>> 1/p-type congestion feed-back with an 1/sqrt(p)-type AQM that was
>>>>>>> also set for sub-optimal RTT and latency target for the test path.
>>>>>>> And lo and behold, the application does "better*" for the 1/p-type
>>>>>>> AQM (with lower latency target; I assume that  L4S ramp-marker
>>>>>>> (Th_low=2ms,
>>>>>>> Th_high=10ms) was carefully selected to match what SCReAM expects).
>>>>>>> IMHO that simply demonstrates, that in communication it pays if
>>>>>>> sender and receiver of a symbol (CE here) assign the same meaning to it.
>>>>>>> 
>>>>>>> But that can not be it, sohat am I missing here?
>>>>>>> 
>>>>>>> Best Regards
>>>>>>> 	Sebastian
>>>>>>> 
>>>>>>> 
>>>>>>> *) Assuming one buys into your definition of better, in which
>>>>>>> (instantaneous) queueing delay is valued over video quality. From
>>>>>>> a network operators perspective that seems a valid position
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> /Ingemar
>>>>>>>> 
>>>>>>>>> -----Original Message-----
>>>>>>>>> From: Sebastian Moeller <[e-mail]>
>>>>>>>>> Sent: den 10 mars 2020 10:45
>>>>>>>>> To: Ingemar Johansson S
>>>>>>>>> <ingemar.s.johansson=[e-mail]>
>>>>>>>>> Cc: [e-mail]; Ingemar Johansson S
>>>>>>>>> <[e-mail]>; [e-mail]
>>>>>>>>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>>>>>>>>> 
>>>>>>>>> Hi Ingemar,
>>>>>>>>> 
>>>>>>>>> thanks for posting this interesting piece of data!
>>>>>>>>> 
>>>>>>>>>> On Mar 10, 2020, at 09:02, Ingemar Johansson S
>>>>>>>>> <ingemar.s.johansson=[e-mail]> wrote:
>>>>>>>>>> Hi
>>>>>>>>>> 
>>>>>>>>>> I recently updated the readme on the SCReAM github with a
>>>>>>>>>> comparison with
>>>>>>>>> SCReAM in three different settings
>>>>>>>>>> 	• No ECN
>>>>>>>>>> 	• CoDel ECN
>>>>>>>>>> 	• L4S
>>>>>>>>>> https://protect2.fireeye.com/v1/url?k=63019d27-3f884737-6301ddb
>>>>>>>>>> c-0
>>>>>>>>>> cc
>>>>>>>>>> 47
>>>>>>>>>> ad93e2a-489fa99c3277fb8a&q=1&e=5aab95a7-4aab-4a64-99a5-
>>>>>>>>> 5b55606e303b&u=
>>>>>>>>>> 
>> https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream%23ecn-
>>>>>>> explicit-
>>>>>>>>> co
>>>>>>>>>> ngestion-notification
>>>>>>>>>> 
>>>>>>>>>> Even though it is more than a magnitude difference in queue
>>>>>>>>>> delay between CoDel-ECN and L4S,
>>>>>>>>> 
>>>>>>>>> 	[SM] So, in this simulations of a 20ms path, SCReAM over L4S
>>>>>>>>> gives
>>>>>>>>> ~10 times less queueing delay, but also only ~2 less bandwidth
>>>>>>>>> compared to SCReAM over codel. You describe this as "L4S reduces
>>>>>>>>> the delay considerably more" and "L4S gives a somewhat lower
>>>>>>>>> media
>>>>> rate".
>>>>>>>>> I wonder how many end-users would tradeoff these 25ms in
>>>>>>>>> queueing delay against the decrease in video quality from halving the
>> bitrate?
>>>>>>>>> Could you repeat the Codel test with interval set to 20 and
>>>>>>>>> target to 1ms, please?
>>>>>>>>> 
>>>>>>>>> If that improves things considerably it would argue for
>>>>>>>>> embedding the current best RTT estimate into SCReAM packets, so
>>>>>>>>> an AQM could tailor its signaling better to individual flow
>>>>>>>>> properties (and yes, that will require a flow-aware AQM).
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> it is fair to say that these simple simulations should of
>>>>>>>>>> course be seen as just a
>>>>>>>>> snapshot.
>>>>>>>>> 
>>>>>>>>> 	[SM] Fair enough.
>>>>>>>>> 
>>>>>>>>>> We hope to present some more simulations with 5G access, and
>>>>>>>>>> not just
>>>>>>>>> simple bottlenecks with one flow, after the summer.
>>>>>>>>> 
>>>>>>>>> 	[Looking] forward to that.
>>>>>>>>> 
>>>>>>>>>> Meanwhile, the SCReAM code on github is freely available for
>>>>>>>>>> anyone who
>>>>>>>>> wish to make more experiments.
>>>>>>>>>> /Ingemar
>>>>>>>>>> ================================ Ingemar Johansson  M.Sc.
>>>>>>>>>> Master Researcher
>>>>>>>>>> 
>>>>>>>>>> Ericsson Research
>>>>>>>>>> RESEARCHER
>>>>>>>>>> GFTL ER NAP NCM Netw Proto & E2E Perf Labratoriegränd 11
>>>>>>>>>> 971 28, Luleå, Sweden
>>>>>>>>>> Phone [phone]
>>>>>>>>>> SMS/MMS [phone]
>>>>>>>>>> [e-mail] www.ericsson.com
>>>>>>>>>> 
>>>>>>>>>> Reality, is the only thing… That’s real!
>>>>>>>>>>    James Halliday, Ready Player One
>>>>>>>>>> =================================
>>> 
>>> --
>>> 
>> ________________________________________________________________
>>> Bob Briscoe                               http://bobbriscoe.net/
>