Re: [iccrg] [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S

[Ingemar Johansson S <[e-mail]>]

Hi

It is true that the "CoDel" implementation in the test code is a botched version. The CoDel algorithm implemented in our system simulator that was used to produce these examples is however _the_ CoDel

/Ingemar

> -----Original Message-----
> From: Sebastian Moeller <[e-mail]>
> Sent: den 17 mars 2020 14:16
> To: Bob Briscoe <[e-mail]>
> Cc: Ingemar Johansson S
> <ingemar.s.johansson=[e-mail]>; Ingemar Johansson S
> <[e-mail]>; [e-mail]; [e-mail]
> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
> 
> Hi Bob,
> 
> 
> > On Mar 17, 2020, at 02:32, Bob Briscoe <[e-mail]> wrote:
> >
> > Sebastian,
> >
> > I suspect it is the 20ms interval that is the main cause the extra queuing delay
> with CoDel.
> 
> 	[SM] That and the fact that rfc3168 asserts ECE until it receives the CWR
> from the sender and that on Ingemar's test path takes 20ms, so the temporal
> fifelity of rfc3168 ECN is simply too low for SCReAM's feed-back fidelity
> requirements.
> 
> 
> >
> > The 20:1 ratio between interval and target was really designed around loss.
> 
> 	[SM] I do not think that this actually makes a big difference for rfc3168
> ECN, CE will be asserted just as a drop would be for NON-ECN packets, and it
> will take a while for a drop being registered by the receiver and send back to the
> sender as well. Note how the 5% rule has no bearing on that.
> 
> > The need to filter out brief excursions in the queue is only necessary when
> you're using impairments (losses) as signals.
> 
> 	[SM] I disagree, (limited) burst tolerance is not only helpful if loss is
> involved.
> 
> 
> > If you're using explicit signals, you can signal unfiltered queue variations,
> because there is no harm having higher ECN marking levels.
> 
> 	[SM] Sure, but I strongly believe the bigger issue here is that L4S CE
> feedback simply has higher temporal fidelity not being limited by the 1RTT ECE-
> >CWR dance.
> 
> 
> > For that matter, you might as well use Eric Dumazet's simple patch to CoDel
> that adds immediate ECN marking at a shallow threshold, bypassing all the
> CoDel machinery, which is really tailored for loss.
> 
> 	[SM] Note how I asked Ingemar about whether he tested codel's
> ce_threshold... But I am still wondering whether Ingemar's test where done with
> SCReAM's own limited Codel implementation or with a full fledged Linux AQM
> node in the path (looking at the SCReAM repository makes me believe its own
> Codel being very vrudimentary and not implementing ce_threshold and as it
> appears not even the square law drop interval calculation, @Ingemar, I am sure
> I am reading something wrong here, so please help me see what I am missing*).
> 
> 
> *) I am 99.999% sure I am missing something ;)
> 
> >
> > Of course, unless you have FQ (which you can't at the RLC layer)
> 
> 	[SM] Then use feedback the queueing information from the RLC layer
> back up to where at least IP packets are handled/seen, akin to say the wifi
> airtime fairness approach?
> 
> > , you would have to work out some other way to ensure coexistence with
> existing congestion controls. That is left as an exercise for the reader - I
> wouldn't want to be seen to be pushing any particular solution. But hopefully,
> through scenarios like this, you might start to understand the set of
> requirements that led to one solution rather than another.
> 
> 	[SM] Well, I believe that neither 1/p type signaling, nor higer temporal
> feed-back fidelity are really under discussion, we all want those, the question is
> more what is the best and safest/backward compatiblest way of using those, but
> I digress (this is not the thread for that discussion ;) )
> 
> Best Regards
> 	Sebastian
> 
> 
> >
> >
> > Bob
> >
> > On 10/03/2020 21:04, Ingemar Johansson S wrote:
> >> Hi
> >> Attached is a simulation of the same simple bottleneck case with target=1ms
> and interval=20ms.
> >> Two ECN beta values : 0.8 and 0.6. A larger backoff (0.6) does actually not
> improve things as the CWND reduction leads to that packets from the video
> coder are queued up in the RTP queue.
> >> As you can see there is still a way to go to reach the L4S delays.
> >>
> >> Perhaps the SCReAM's response to CoDel - ECN marks can be optimized
> further, don't know, but I already spent a considerable time to try and get where
> the code is now, and I spent a lot more time on this than I spent on the response
> to the L4S signal.
> >> My impression is that it is the fractional congestion signal with L4S
> >> that makes it easier to get a good algorithm behavior (and I
> >> definitely believe that there is room for improvement)
> >>
> >> And as said earlier, the SCReAM code is freely available to experiment with
> and occasionally it is quite fun
> (https://www.youtube.com/watch?v=eU1crtEvMv4 ).
> >>
> >>
> >> /Ingemar
> >>
> >>> -----Original Message-----
> >>> From: Sebastian Moeller <[e-mail]>
> >>> Sent: den 10 mars 2020 16:29
> >>> To: Ingemar Johansson S <[e-mail]>
> >>> Cc: Ingemar Johansson S
> >>> <ingemar.s.johansson=[e-mail]>; [e-mail];
> >>> [e-mail]
> >>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
> >>>
> >>> Hi Ingemar,
> >>>
> >>>
> >>>
> >>>> On Mar 10, 2020, at 13:37, Ingemar Johansson S
> >>> <[e-mail]> wrote:
> >>>> Hi
> >>>>
> >>>> The SCReAM code is freely available on
> >>> https://protect2.fireeye.com/v1/url?k=3fa4aff0-6370a99d-3fa4ef6b-
> >>> 867011091b6c-c4d35656f5c9b268&q=1&e=16e2b451-36c7-4814-af9d-
> >>>
> 4acb21ac792b&u=https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream
> >>> for anybody interested to run their own experiment with whatever
> >>> AQM/ECN configuration.
> >>>
> >>> 	[SM] Well, if you are not interested in figuring out what is
> >>> happening with with rfc3168 ECN ans SCReAM that is your prerogative...
> >>>
> >>>
> >>>> Please note that SCReAM is configured in an L4S mode when the
> >>>> network AQM does L4S marking (mimicking ECT(1)). For CoDel-ECN
> >>>> however, SCReAM runs in normal ECN mode with a beta of 0.8 (=20%
> >>>> reduction on CWND for each congestion event)
> >>> 	[SM] Okay, that is a far cry from Reno's 50% though, have you
> >>> tested different reduction percentages or how did you arrive at 20%?
> >>> But is it really 0.8? ScreamTx.h has the following:
> >>>
> >>> // CWND scale factor upon ECN-CE event static const float kEcnCeBeta
> >>> = 0.9f;
> >>>
> >>> I am probably missing something somewhere...
> >>>
> >>>> I tried also with other different ramp markers (1ms/10ms),
> >>> (2ms/10ms),(5ms/15ms).
> >>>
> >>> 	[SM] That means for L4S, or do you mean you tried different values
> >>> for Codel's target ans ce_threshold parameters?
> >>>
> >>>
> >>>> There are slight variations  in throughput and latency but not dramatic.
> >>> 	[SM] Well, these are also pretty slight variations in numerical
> >>> values, at least if compared to codel's default interval of 100ms...
> >>>
> >>>> And truth to be told, the ECN behavior is better tuned in the code
> >>>> than the L4S
> >>> behavior.
> >>>
> >>> 	[SM] For a defintion of better that ignores that you seem to be
> >>> much happier with the results L4S gives you.
> >>>
> >>>> There is room for improvement as regards to the L4S behavior (for
> >>>> instance
> >>> faster ramp-up) and it may well be the case that SCReAM is
> >>> completely scrapped in favor of new designs.
> >>>> But the bottomline, the L4S thresholds and L4S code is not
> >>>> carefully picked to
> >>> show a good performance.
> >>>
> >>> 	[SM] So, you tried different L4S parameters (and found the to have
> >>> similar performance) but for rfc3168 ECN and Codel you only tried
> >>> the default parameters?
> >>>
> >>> Best Regards
> >>> 	Sebastian
> >>>
> >>>> /Ingemar
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: Sebastian Moeller <[e-mail]>
> >>>>> Sent: den 10 mars 2020 11:28
> >>>>> To: Ingemar Johansson S <[e-mail]>
> >>>>> Cc: Ingemar Johansson S
> >>>>> <ingemar.s.johansson=[e-mail]>;
> >>>>> [e-mail]; [e-mail]
> >>>>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
> >>>>>
> >>>>> Hi Ingemar,
> >>>>>
> >>>>>
> >>>>>> On Mar 10, 2020, at 11:07, Ingemar Johansson S
> >>>>> <[e-mail]> wrote:
> >>>>>> Hi
> >>>>>>
> >>>>>> For the future studies we will only focus on L4S as the scope is
> >>>>>> to study the
> >>>>> performance gain that L4S give for instance for AR/VR, gaming and
> >>>>> remote control applications.
> >>>>>
> >>>>> 	[SM] How are you going to "study the performance gain that L4S
> >>>>> give[s]" if you do not compare it with the best of class
> >>>>> alternatives? I am truly puzzled.
> >>>>>
> >>>>>> Flow aware AQMs with RTT estimates as metadata in the packets is
> >>>>>> outside
> >>>>> the scope as it would require packet inspection, which is not
> >>>>> feasible if queues build up on the RLC layer in the 3GPP stack.
> >>>>>
> >>>>> 	[SM] Fair enough. What is this comparison intended to show us then?
> >>>>>
> >>>>> As far as I can see you paired an application designed for
> >>>>> 1/p-type congestion feed-back with an 1/sqrt(p)-type AQM that was
> >>>>> also set for sub-optimal RTT and latency target for the test path.
> >>>>> And lo and behold, the application does "better*" for the 1/p-type
> >>>>> AQM (with lower latency target; I assume that  L4S ramp-marker
> >>>>> (Th_low=2ms,
> >>>>> Th_high=10ms) was carefully selected to match what SCReAM expects).
> >>>>> IMHO that simply demonstrates, that in communication it pays if
> >>>>> sender and receiver of a symbol (CE here) assign the same meaning to it.
> >>>>>
> >>>>> But that can not be it, sohat am I missing here?
> >>>>>
> >>>>> Best Regards
> >>>>> 	Sebastian
> >>>>>
> >>>>>
> >>>>> *) Assuming one buys into your definition of better, in which
> >>>>> (instantaneous) queueing delay is valued over video quality. From
> >>>>> a network operators perspective that seems a valid position
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>> /Ingemar
> >>>>>>
> >>>>>>> -----Original Message-----
> >>>>>>> From: Sebastian Moeller <[e-mail]>
> >>>>>>> Sent: den 10 mars 2020 10:45
> >>>>>>> To: Ingemar Johansson S
> >>>>>>> <ingemar.s.johansson=[e-mail]>
> >>>>>>> Cc: [e-mail]; Ingemar Johansson S
> >>>>>>> <[e-mail]>; [e-mail]
> >>>>>>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
> >>>>>>>
> >>>>>>> Hi Ingemar,
> >>>>>>>
> >>>>>>> thanks for posting this interesting piece of data!
> >>>>>>>
> >>>>>>>> On Mar 10, 2020, at 09:02, Ingemar Johansson S
> >>>>>>> <ingemar.s.johansson=[e-mail]> wrote:
> >>>>>>>> Hi
> >>>>>>>>
> >>>>>>>> I recently updated the readme on the SCReAM github with a
> >>>>>>>> comparison with
> >>>>>>> SCReAM in three different settings
> >>>>>>>> 	• No ECN
> >>>>>>>> 	• CoDel ECN
> >>>>>>>> 	• L4S
> >>>>>>>> https://protect2.fireeye.com/v1/url?k=63019d27-3f884737-6301ddb
> >>>>>>>> c-0
> >>>>>>>> cc
> >>>>>>>> 47
> >>>>>>>> ad93e2a-489fa99c3277fb8a&q=1&e=5aab95a7-4aab-4a64-99a5-
> >>>>>>> 5b55606e303b&u=
> >>>>>>>>
> https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream%23ecn-
> >>>>> explicit-
> >>>>>>> co
> >>>>>>>> ngestion-notification
> >>>>>>>>
> >>>>>>>> Even though it is more than a magnitude difference in queue
> >>>>>>>> delay between CoDel-ECN and L4S,
> >>>>>>>
> >>>>>>> 	[SM] So, in this simulations of a 20ms path, SCReAM over L4S
> >>>>>>> gives
> >>>>>>> ~10 times less queueing delay, but also only ~2 less bandwidth
> >>>>>>> compared to SCReAM over codel. You describe this as "L4S reduces
> >>>>>>> the delay considerably more" and "L4S gives a somewhat lower
> >>>>>>> media
> >>> rate".
> >>>>>>> I wonder how many end-users would tradeoff these 25ms in
> >>>>>>> queueing delay against the decrease in video quality from halving the
> bitrate?
> >>>>>>> Could you repeat the Codel test with interval set to 20 and
> >>>>>>> target to 1ms, please?
> >>>>>>>
> >>>>>>> If that improves things considerably it would argue for
> >>>>>>> embedding the current best RTT estimate into SCReAM packets, so
> >>>>>>> an AQM could tailor its signaling better to individual flow
> >>>>>>> properties (and yes, that will require a flow-aware AQM).
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> it is fair to say that these simple simulations should of
> >>>>>>>> course be seen as just a
> >>>>>>> snapshot.
> >>>>>>>
> >>>>>>> 	[SM] Fair enough.
> >>>>>>>
> >>>>>>>> We hope to present some more simulations with 5G access, and
> >>>>>>>> not just
> >>>>>>> simple bottlenecks with one flow, after the summer.
> >>>>>>>
> >>>>>>> 	[Looking] forward to that.
> >>>>>>>
> >>>>>>>> Meanwhile, the SCReAM code on github is freely available for
> >>>>>>>> anyone who
> >>>>>>> wish to make more experiments.
> >>>>>>>> /Ingemar
> >>>>>>>> ================================ Ingemar Johansson  M.Sc.
> >>>>>>>> Master Researcher
> >>>>>>>>
> >>>>>>>> Ericsson Research
> >>>>>>>> RESEARCHER
> >>>>>>>> GFTL ER NAP NCM Netw Proto & E2E Perf Labratoriegränd 11
> >>>>>>>> 971 28, Luleå, Sweden
> >>>>>>>> Phone [phone]
> >>>>>>>> SMS/MMS [phone]
> >>>>>>>> [e-mail] www.ericsson.com
> >>>>>>>>
> >>>>>>>> Reality, is the only thing… That’s real!
> >>>>>>>>     James Halliday, Ready Player One
> >>>>>>>> =================================
> >
> > --
> >
> ________________________________________________________________
> > Bob Briscoe                               http://bobbriscoe.net/