Re: [tsvwg] [iccrg] SCReAM (RFC8298) with CoDel-ECN and L4S

Bob Briscoe <ietf@bobbriscoe.net> Tue, 17 March 2020 01:01 UTC

Return-Path: <ietf@bobbriscoe.net>
X-Original-To: tsvwg@ietfa.amsl.com
Delivered-To: tsvwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E97223A14B8 for <tsvwg@ietfa.amsl.com>; Mon, 16 Mar 2020 18:01:34 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.098
X-Spam-Level:
X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=bobbriscoe.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3ZKJWNZySSgi for <tsvwg@ietfa.amsl.com>; Mon, 16 Mar 2020 18:01:32 -0700 (PDT)
Received: from cl3.bcs-hosting.net (cl3.bcs-hosting.net [3.11.37.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 785693A14B5 for <tsvwg@ietf.org>; Mon, 16 Mar 2020 18:01:32 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bobbriscoe.net; s=default; h=Content-Type:In-Reply-To:MIME-Version:Date: Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=6slbkPFwXgUNifszGy9Ga0Q5RLv27ytdTcjjOzfObIo=; b=fDPbdKB7x+rgXqSU4BuoyRSA4 qC94KdbJ2YE2v9r1SWkq8tP0R+zqUH1XzILx7H6sihAYhBW+7PudnS5ktAvRXGDV04uZE4Uy/JQ7H V0kckfzgkQggUs74mH10VO9JhDnGK4LUEMBlWQNGO9/nxufImkVw7traJrBy2jNyOVjrFmTftuMh4 XvOUnwtr2PC8IuZufJjsCPw8SpIuhI7sqS9lUznOUQDl8rsmaDqNNLkOCoD5zWbJWOZzNx+HM2Vjs fTj4pKXjciV05YKM/iqWYrazwW07iYZ4DbOkpq+9jXDUw4wKnAS4EMiZoaL1xQx2bmKYk2pxhaDmC NTQcwEzpw==;
Received: from host-79-78-166-168.static.as9105.net ([79.78.166.168]:35352 helo=[192.168.2.5]) by cl3.bcs-hosting.net with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.93) (envelope-from <ietf@bobbriscoe.net>) id 1jE0bo-008itO-Lp; Tue, 17 Mar 2020 01:01:29 +0000
To: Ingemar Johansson S <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>
Cc: Sebastian Moeller <moeller0@gmx.de>, Ingemar Johansson S <ingemar.s.johansson@ericsson.com>, "iccrg@irtf.org" <iccrg@irtf.org>, "tsvwg@ietf.org" <tsvwg@ietf.org>
References: =?utf-8?q?=3CHE1PR07MB44251B019947CDB6602B30B2C2FF0=40HE1PR07MB4?= =?utf-8?q?425=2Eeurprd07=2Eprod=2Eoutlook=2Ecom=3E?= =?utf-8?q?=3CA2300F8D-5F87-461E-AD94-8D7B22A6CDF3=40gmx=2Ede=3E_=3CHE1PR07M?= =?utf-8?q?B4425B105AFF56D1566164900C2FF0=40HE1PR07MB4425=2Eeurprd07=2Eprod?= =?utf-8?q?=2Eoutlook=2Ecom=3E?= <1C969A05-A4B7-43E9-B694-3195A2FC086A@gmx.de> =?utf-8?q?=3CHE1PR07MB44255CED94938F9C38515FD6C2FF0=40HE1PR07MB4425=2Eeurpr?= =?utf-8?q?d07=2Eprod=2Eoutlook=2Ecom=3E?=
From: Bob Briscoe <ietf@bobbriscoe.net>
Message-ID: <3c373357-0cdf-be13-7042-4568661e7f16@bobbriscoe.net>
Date: Tue, 17 Mar 2020 01:01:27 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1
MIME-Version: 1.0
In-Reply-To: =?utf-8?q?=3CHE1PR07MB44255CED94938F9C38515FD6C2FF0=40HE1PR07MB?= =?utf-8?q?4425=2Eeurprd07=2Eprod=2Eoutlook=2Ecom=3E?=
Content-Type: multipart/alternative; boundary="------------CC303045E5104DB43778B554"
Content-Language: en-GB
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - cl3.bcs-hosting.net
X-AntiAbuse: Original Domain - ietf.org
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - bobbriscoe.net
X-Get-Message-Sender-Via: cl3.bcs-hosting.net: authenticated_id: in@bobbriscoe.net
X-Authenticated-Sender: cl3.bcs-hosting.net: in@bobbriscoe.net
X-Source:
X-Source-Args:
X-Source-Dir:
Archived-At: <https://mailarchive.ietf.org/arch/msg/tsvwg/oBopOlE_5D2vDAa9bvS_QPRJftU>
Subject: Re: [tsvwg] [iccrg] SCReAM (RFC8298) with CoDel-ECN and L4S
X-BeenThere: tsvwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <tsvwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tsvwg/>
List-Post: <mailto:tsvwg@ietf.org>
List-Help: <mailto:tsvwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tsvwg>, <mailto:tsvwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 17 Mar 2020 01:01:35 -0000

Ingemar,

On 10/03/2020 12:37, Ingemar Johansson S wrote:
> Hi
>
> The SCReAM code is freely available on https://github.com/EricssonResearch/scream for anybody interested to run their own experiment with whatever AQM/ECN configuration.
> Please note that SCReAM is configured in an L4S mode when the network AQM does L4S marking (mimicking ECT(1)). For CoDel-ECN however, SCReAM runs in normal ECN mode with a beta of 0.8 (=20% reduction on CWND for each congestion event)

[BB] In your previous work, in order to get early L4S ECN marking, you 
used a virtual queue in the network draining at a little less than the 
rate given by the scheduler. Am I right that you're not doing that here?

In the L4S case you show, when the video is running at about half the 
rate in the CoDel case (which is presumably closer to that available 
from the scheduler), one would have thought it would get very little ECN 
marking. But I notice it's getting frequent little spikes of marking. Is 
that due to I-frames causing bursts or something?

What if it wasn't getting those spikes of marks (e.g. using sthg like 
x264 to spread the I-frames over time)?

A random thought: Could the video use the Classic queue as a scavenger 
service to fill in extra coding layer(s) whenever bandwidth is available?


Bob


>
> I tried also with other different ramp markers (1ms/10ms), (2ms/10ms),(5ms/15ms). There are slight variations  in throughput and latency but not dramatic.
> And truth to be told, the ECN behavior is better tuned in the code than the L4S behavior.
> There is room for improvement as regards to the L4S behavior (for instance faster ramp-up) and it may well be the case that SCReAM is completely scrapped in favor of new designs.
> But the bottomline, the L4S thresholds and L4S code is not carefully picked to show a good performance.
>
> /Ingemar
>
>> -----Original Message-----
>> From: Sebastian Moeller <moeller0@gmx.de>
>> Sent: den 10 mars 2020 11:28
>> To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
>> Cc: Ingemar Johansson S
>> <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>rg>; tsvwg@ietf.org;
>> iccrg@irtf.org
>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>>
>> Hi Ingemar,
>>
>>
>>> On Mar 10, 2020, at 11:07, Ingemar Johansson S
>> <ingemar.s.johansson@ericsson.com> wrote:
>>> Hi
>>>
>>> For the future studies we will only focus on L4S as the scope is to study the
>> performance gain that L4S give for instance for AR/VR, gaming and remote
>> control applications.
>>
>> 	[SM] How are you going to "study the performance gain that L4S
>> give[s]" if you do not compare it with the best of class alternatives? I am truly
>> puzzled.
>>
>>> Flow aware AQMs with RTT estimates as metadata in the packets is outside
>> the scope as it would require packet inspection, which is not feasible if queues
>> build up on the RLC layer in the 3GPP stack.
>>
>> 	[SM] Fair enough. What is this comparison intended to show us then?
>>
>> As far as I can see you paired an application designed for 1/p-type congestion
>> feed-back with an 1/sqrt(p)-type AQM that was also set for sub-optimal RTT and
>> latency target for the test path. And lo and behold, the application does
>> "better*" for the 1/p-type AQM (with lower latency target; I assume that  L4S
>> ramp-marker (Th_low=2ms, Th_high=10ms) was carefully selected to match
>> what SCReAM expects). IMHO that simply demonstrates, that in communication
>> it pays if sender and receiver of a symbol (CE here) assign the same meaning to
>> it.
>>
>> But that can not be it, sohat am I missing here?
>>
>> Best Regards
>> 	Sebastian
>>
>>
>> *) Assuming one buys into your definition of better, in which (instantaneous)
>> queueing delay is valued over video quality. From a network operators
>> perspective that seems a valid position
>>
>>
>>
>>
>>> /Ingemar
>>>
>>>> -----Original Message-----
>>>> From: Sebastian Moeller <moeller0@gmx.de>
>>>> Sent: den 10 mars 2020 10:45
>>>> To: Ingemar Johansson S
>>>> <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>
>>>> Cc: tsvwg@ietf.org; Ingemar Johansson S
>>>> <ingemar.s.johansson@ericsson.com>om>; iccrg@irtf.org
>>>> Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
>>>>
>>>> Hi Ingemar,
>>>>
>>>> thanks for posting this interesting piece of data!
>>>>
>>>>> On Mar 10, 2020, at 09:02, Ingemar Johansson S
>>>> <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org> wrote:
>>>>> Hi
>>>>>
>>>>> I recently updated the readme on the SCReAM github with a comparison
>>>>> with
>>>> SCReAM in three different settings
>>>>> 	• No ECN
>>>>> 	• CoDel ECN
>>>>> 	• L4S
>>>>> https://protect2.fireeye.com/v1/url?k=63019d27-3f884737-6301ddbc-0cc
>>>>> 47
>>>>> ad93e2a-489fa99c3277fb8a&q=1&e=5aab95a7-4aab-4a64-99a5-
>>>> 5b55606e303b&u=
>>>>> https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream%23ecn-
>> explicit-
>>>> co
>>>>> ngestion-notification
>>>>>
>>>>> Even though it is more than a magnitude difference in queue delay
>>>>> between CoDel-ECN and L4S,
>>>>
>>>> 	[SM] So, in this simulations of a 20ms path, SCReAM over L4S gives
>>>> ~10 times less queueing delay, but also only ~2 less bandwidth
>>>> compared to SCReAM over codel. You describe this as "L4S reduces the
>>>> delay considerably more" and "L4S gives a somewhat lower media rate".
>>>> I wonder how many end-users would tradeoff these 25ms in queueing
>>>> delay against the decrease in video quality from halving the bitrate?
>>>> Could you repeat the Codel test with interval set to 20 and target to
>>>> 1ms, please?
>>>>
>>>> If that improves things considerably it would argue for embedding the
>>>> current best RTT estimate into SCReAM packets, so an AQM could tailor
>>>> its signaling better to individual flow properties (and yes, that
>>>> will require a flow-aware AQM).
>>>>
>>>>
>>>>
>>>>> it is fair to say that these simple simulations should of course be
>>>>> seen as just a
>>>> snapshot.
>>>>
>>>> 	[SM] Fair enough.
>>>>
>>>>> We hope to present some more simulations with 5G access, and not
>>>>> just
>>>> simple bottlenecks with one flow, after the summer.
>>>>
>>>> 	[Looking] forward to that.
>>>>
>>>>> Meanwhile, the SCReAM code on github is freely available for anyone
>>>>> who
>>>> wish to make more experiments.
>>>>> /Ingemar
>>>>> ================================
>>>>> Ingemar Johansson  M.Sc.
>>>>> Master Researcher
>>>>>
>>>>> Ericsson Research
>>>>> RESEARCHER
>>>>> GFTL ER NAP NCM Netw Proto & E2E Perf Labratoriegränd 11
>>>>> 971 28, Luleå, Sweden
>>>>> Phone +46-1071 43042
>>>>> SMS/MMS +46-73 078 3289
>>>>> ingemar.s.johansson@ericsson.com
>>>>> www.ericsson.com
>>>>>
>>>>>   Reality, is the only thing… That’s real!
>>>>>       James Halliday, Ready Player One
>>>>> =================================
>
> _______________________________________________
> iccrg mailing list
> iccrg@irtf.org
> https://www.irtf.org/mailman/listinfo/iccrg

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/