Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S

[Sebastian Moeller <[e-mail]>]

Hi Bob,


> On Mar 17, 2020, at 02:32, Bob Briscoe <[e-mail]> wrote:
> 
> Sebastian,
> 
> I suspect it is the 20ms interval that is the main cause the extra queuing delay with CoDel.

	[SM] That and the fact that rfc3168 asserts ECE until it receives the CWR from the sender and that on Ingemar's test path takes 20ms, so the temporal fifelity of rfc3168 ECN is simply too low for SCReAM's feed-back fidelity requirements.


> 
> The 20:1 ratio between interval and target was really designed around loss.

	[SM] I do not think that this actually makes a big difference for rfc3168 ECN, CE will be asserted just as a drop would be for NON-ECN packets, and it will take a while for a drop being registered by the receiver and send back to the sender as well. Note how the 5% rule has no bearing on that.

> The need to filter out brief excursions in the queue is only necessary when you're using impairments (losses) as signals.

	[SM] I disagree, (limited) burst tolerance is not only helpful if loss is involved.


> If you're using explicit signals, you can signal unfiltered queue variations, because there is no harm having higher ECN marking levels.

	[SM] Sure, but I strongly believe the bigger issue here is that L4S CE feedback simply has higher temporal fidelity not being limited by the 1RTT ECE->CWR dance.


> For that matter, you might as well use Eric Dumazet's simple patch to CoDel that adds immediate ECN marking at a shallow threshold, bypassing all the CoDel machinery, which is really tailored for loss.

	[SM] Note how I asked Ingemar about whether he tested codel's ce_threshold... But I am still wondering whether Ingemar's test where done with SCReAM's own limited Codel implementation or with a full fledged Linux AQM node in the path (looking at the SCReAM repository makes me believe its own Codel being very vrudimentary and not implementing ce_threshold and as it appears not even the square law drop interval calculation, @Ingemar, I am sure I am reading something wrong here, so please help me see what I am missing*).


*) I am 99.999% sure I am missing something ;)

> 
> Of course, unless you have FQ (which you can't at the RLC layer)

	[SM] Then use feedback the queueing information from the RLC layer back up to where at least IP packets are handled/seen, akin to say the wifi airtime fairness approach?

> , you would have to work out some other way to ensure coexistence with existing congestion controls. That is left as an exercise for the reader - I wouldn't want to be seen to be pushing any particular solution. But hopefully, through scenarios like this, you might start to understand the set of requirements that led to one solution rather than another.

	[SM] Well, I believe that neither 1/p type signaling, nor higer temporal feed-back fidelity are really under discussion, we all want those, the question is more what is the best and safest/backward compatiblest way of using those, but I digress (this is not the thread for that discussion ;) )

Best Regards
	Sebastian


> 
> 
> Bob
> 
> On 10/03/2020 21:04, Ingemar Johansson S wrote:
>> Hi
>> Attached is a simulation of the same simple bottleneck case with target=1ms and interval=20ms.
>> Two ECN beta values : 0.8 and 0.6. A larger backoff (0.6) does actually not improve things as the CWND reduction leads to that packets from the video coder are queued up in the RTP queue.
>> As you can see there is still a way to go to reach the L4S delays.
>> 
>> Perhaps the SCReAM's response to CoDel - ECN marks can be optimized further, don't know, but I already spent a considerable time to try and get where the code is now, and I spent a lot more time on this than I spent on the response to the L4S signal.
>> My impression is that it is the fractional congestion signal with L4S that makes it easier to get a good algorithm behavior (and I definitely believe that there is room for improvement)
>> 
>> And as said earlier, the SCReAM code is freely available to experiment with and occasionally it is quite fun (https://www.youtube.com/watch?v=eU1crtEvMv4 ).
>> 
>> 
>> /Ingemar
>> 
>>> -----Original Message-----
>>> From: Sebastian Moeller <[e-mail]>
>>> Sent: den 10 mars 2020 16:29
>>> To: Ingemar Johansson S <[e-mail]>
>>> Cc: Ingemar Johansson S
>>> <ingemar.s.johansson=[e-mail]>; [e-mail];
>>> [e-mail]
>>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>>> 
>>> Hi Ingemar,
>>> 
>>> 
>>> 
>>>> On Mar 10, 2020, at 13:37, Ingemar Johansson S
>>> <[e-mail]> wrote:
>>>> Hi
>>>> 
>>>> The SCReAM code is freely available on
>>> https://protect2.fireeye.com/v1/url?k=3fa4aff0-6370a99d-3fa4ef6b-
>>> 867011091b6c-c4d35656f5c9b268&q=1&e=16e2b451-36c7-4814-af9d-
>>> 4acb21ac792b&u=https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream
>>> for anybody interested to run their own experiment with whatever AQM/ECN
>>> configuration.
>>> 
>>> 	[SM] Well, if you are not interested in figuring out what is happening
>>> with with rfc3168 ECN ans SCReAM that is your prerogative...
>>> 
>>> 
>>>> Please note that SCReAM is configured in an L4S mode when the network
>>>> AQM does L4S marking (mimicking ECT(1)). For CoDel-ECN however, SCReAM
>>>> runs in normal ECN mode with a beta of 0.8 (=20% reduction on CWND for
>>>> each congestion event)
>>> 	[SM] Okay, that is a far cry from Reno's 50% though, have you tested
>>> different reduction percentages or how did you arrive at 20%?
>>> But is it really 0.8? ScreamTx.h has the following:
>>> 
>>> // CWND scale factor upon ECN-CE event
>>> static const float kEcnCeBeta = 0.9f;
>>> 
>>> I am probably missing something somewhere...
>>> 
>>>> I tried also with other different ramp markers (1ms/10ms),
>>> (2ms/10ms),(5ms/15ms).
>>> 
>>> 	[SM] That means for L4S, or do you mean you tried different values for
>>> Codel's target ans ce_threshold parameters?
>>> 
>>> 
>>>> There are slight variations  in throughput and latency but not dramatic.
>>> 	[SM] Well, these are also pretty slight variations in numerical values, at
>>> least if compared to codel's default interval of 100ms...
>>> 
>>>> And truth to be told, the ECN behavior is better tuned in the code than the L4S
>>> behavior.
>>> 
>>> 	[SM] For a defintion of better that ignores that you seem to be much
>>> happier with the results L4S gives you.
>>> 
>>>> There is room for improvement as regards to the L4S behavior (for instance
>>> faster ramp-up) and it may well be the case that SCReAM is completely scrapped
>>> in favor of new designs.
>>>> But the bottomline, the L4S thresholds and L4S code is not carefully picked to
>>> show a good performance.
>>> 
>>> 	[SM] So, you tried different L4S parameters (and found the to have
>>> similar performance) but for rfc3168 ECN and Codel you only tried the default
>>> parameters?
>>> 
>>> Best Regards
>>> 	Sebastian
>>> 
>>>> /Ingemar
>>>> 
>>>>> -----Original Message-----
>>>>> From: Sebastian Moeller <[e-mail]>
>>>>> Sent: den 10 mars 2020 11:28
>>>>> To: Ingemar Johansson S <[e-mail]>
>>>>> Cc: Ingemar Johansson S
>>>>> <ingemar.s.johansson=[e-mail]>; [e-mail];
>>>>> [e-mail]
>>>>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>>>>> 
>>>>> Hi Ingemar,
>>>>> 
>>>>> 
>>>>>> On Mar 10, 2020, at 11:07, Ingemar Johansson S
>>>>> <[e-mail]> wrote:
>>>>>> Hi
>>>>>> 
>>>>>> For the future studies we will only focus on L4S as the scope is to
>>>>>> study the
>>>>> performance gain that L4S give for instance for AR/VR, gaming and
>>>>> remote control applications.
>>>>> 
>>>>> 	[SM] How are you going to "study the performance gain that L4S
>>>>> give[s]" if you do not compare it with the best of class
>>>>> alternatives? I am truly puzzled.
>>>>> 
>>>>>> Flow aware AQMs with RTT estimates as metadata in the packets is
>>>>>> outside
>>>>> the scope as it would require packet inspection, which is not
>>>>> feasible if queues build up on the RLC layer in the 3GPP stack.
>>>>> 
>>>>> 	[SM] Fair enough. What is this comparison intended to show us then?
>>>>> 
>>>>> As far as I can see you paired an application designed for 1/p-type
>>>>> congestion feed-back with an 1/sqrt(p)-type AQM that was also set for
>>>>> sub-optimal RTT and latency target for the test path. And lo and
>>>>> behold, the application does "better*" for the 1/p-type AQM (with
>>>>> lower latency target; I assume that  L4S ramp-marker (Th_low=2ms,
>>>>> Th_high=10ms) was carefully selected to match what SCReAM expects).
>>>>> IMHO that simply demonstrates, that in communication it pays if
>>>>> sender and receiver of a symbol (CE here) assign the same meaning to it.
>>>>> 
>>>>> But that can not be it, sohat am I missing here?
>>>>> 
>>>>> Best Regards
>>>>> 	Sebastian
>>>>> 
>>>>> 
>>>>> *) Assuming one buys into your definition of better, in which
>>>>> (instantaneous) queueing delay is valued over video quality. From a
>>>>> network operators perspective that seems a valid position
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>>> /Ingemar
>>>>>> 
>>>>>>> -----Original Message-----
>>>>>>> From: Sebastian Moeller <[e-mail]>
>>>>>>> Sent: den 10 mars 2020 10:45
>>>>>>> To: Ingemar Johansson S
>>>>>>> <ingemar.s.johansson=[e-mail]>
>>>>>>> Cc: [e-mail]; Ingemar Johansson S
>>>>>>> <[e-mail]>; [e-mail]
>>>>>>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>>>>>>> 
>>>>>>> Hi Ingemar,
>>>>>>> 
>>>>>>> thanks for posting this interesting piece of data!
>>>>>>> 
>>>>>>>> On Mar 10, 2020, at 09:02, Ingemar Johansson S
>>>>>>> <ingemar.s.johansson=[e-mail]> wrote:
>>>>>>>> Hi
>>>>>>>> 
>>>>>>>> I recently updated the readme on the SCReAM github with a
>>>>>>>> comparison with
>>>>>>> SCReAM in three different settings
>>>>>>>> 	• No ECN
>>>>>>>> 	• CoDel ECN
>>>>>>>> 	• L4S
>>>>>>>> https://protect2.fireeye.com/v1/url?k=63019d27-3f884737-6301ddbc-0
>>>>>>>> cc
>>>>>>>> 47
>>>>>>>> ad93e2a-489fa99c3277fb8a&q=1&e=5aab95a7-4aab-4a64-99a5-
>>>>>>> 5b55606e303b&u=
>>>>>>>> https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream%23ecn-
>>>>> explicit-
>>>>>>> co
>>>>>>>> ngestion-notification
>>>>>>>> 
>>>>>>>> Even though it is more than a magnitude difference in queue delay
>>>>>>>> between CoDel-ECN and L4S,
>>>>>>> 
>>>>>>> 	[SM] So, in this simulations of a 20ms path, SCReAM over L4S gives
>>>>>>> ~10 times less queueing delay, but also only ~2 less bandwidth
>>>>>>> compared to SCReAM over codel. You describe this as "L4S reduces
>>>>>>> the delay considerably more" and "L4S gives a somewhat lower media
>>> rate".
>>>>>>> I wonder how many end-users would tradeoff these 25ms in queueing
>>>>>>> delay against the decrease in video quality from halving the bitrate?
>>>>>>> Could you repeat the Codel test with interval set to 20 and target
>>>>>>> to 1ms, please?
>>>>>>> 
>>>>>>> If that improves things considerably it would argue for embedding
>>>>>>> the current best RTT estimate into SCReAM packets, so an AQM could
>>>>>>> tailor its signaling better to individual flow properties (and yes,
>>>>>>> that will require a flow-aware AQM).
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> it is fair to say that these simple simulations should of course
>>>>>>>> be seen as just a
>>>>>>> snapshot.
>>>>>>> 
>>>>>>> 	[SM] Fair enough.
>>>>>>> 
>>>>>>>> We hope to present some more simulations with 5G access, and not
>>>>>>>> just
>>>>>>> simple bottlenecks with one flow, after the summer.
>>>>>>> 
>>>>>>> 	[Looking] forward to that.
>>>>>>> 
>>>>>>>> Meanwhile, the SCReAM code on github is freely available for
>>>>>>>> anyone who
>>>>>>> wish to make more experiments.
>>>>>>>> /Ingemar
>>>>>>>> ================================
>>>>>>>> Ingemar Johansson  M.Sc.
>>>>>>>> Master Researcher
>>>>>>>> 
>>>>>>>> Ericsson Research
>>>>>>>> RESEARCHER
>>>>>>>> GFTL ER NAP NCM Netw Proto & E2E Perf Labratoriegränd 11
>>>>>>>> 971 28, Luleå, Sweden
>>>>>>>> Phone [phone]
>>>>>>>> SMS/MMS [phone]
>>>>>>>> [e-mail]
>>>>>>>> www.ericsson.com
>>>>>>>> 
>>>>>>>> Reality, is the only thing… That’s real!
>>>>>>>>     James Halliday, Ready Player One
>>>>>>>> =================================
> 
> -- 
> ________________________________________________________________
> Bob Briscoe                               http://bobbriscoe.net/