Re: [iccrg] [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S

Ingemar Johansson S <ingemar.s.johansson@ericsson.com> Fri, 13 March 2020 20:33 UTC

Return-Path: <ingemar.s.johansson@ericsson.com>
X-Original-To: iccrg@ietfa.amsl.com
Delivered-To: iccrg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 049503A0E9D for <iccrg@ietfa.amsl.com>; Fri, 13 Mar 2020 13:33:10 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.1
X-Spam-Level:
X-Spam-Status: No, score=-2.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id o2ejn6l31zvD for <iccrg@ietfa.amsl.com>; Fri, 13 Mar 2020 13:33:06 -0700 (PDT)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-eopbgr60087.outbound.protection.outlook.com [40.107.6.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6498B3A0EFA for <iccrg@irtf.org>; Fri, 13 Mar 2020 13:33:05 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HY+Kpveq00E2MV68GUbd3/x4HG79wTsmR4glRYlkIdmLK/nWIZkxG46gFqzyPR1dPygu/U5yoKBu8UjuliBPFPd7xpNGoVU3whs3tmAQVAkzVg6pybqNX/+VFMi2Q+NfQ2d/4cNQI0Ilmmt/wWIr1mv1V0yA7krFwIaAad+1wv3EAIRgzPL/Jaiv/ZbCkBxyUy/iR0FwAlipxkgT/je0a9f+3Qzs/uyfpy6va7YjkdURUrYQj951W4CuYPEwMLrkl1yVOwCkV8GMdTrVqQFJjCqM9Kjkg0i1BirAP2AgLIn/KD8CZrY94lyONk2mKw0LQhS4MZyuLL3c6Eypk07XLA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;bh=/3q1dPYIfSnectQZZDWhW+KzjIw535AxCuS18PVtS+A=; b=Yht4rukzfaI513A79ZC37cgGQ2hKCf5Dsuvp7/7GFnJ2kzOqS4BrmPrHJBPTzsDTCidkLgxSx9huQRZi+zNp0zKred1qcmRJWDLIsYvOeXXVAnCLMkkEx6MQObfWW37Mo6VMkJ0DocNu7ad4KMSWNNkjAda9epRFeq/dfQVJYNBU7T4NIaHn0BoAErccII4MXPbHKDIisuVXFJUeIyOdnauMprXapogKB5GRVUuMkG1qYNEQq2NkOA9HwNUAn+7RLXA7f4dl+rVOedZ3wIXgcev+S6DBUVn1xHoFoa65iZaoszBWf2ua8UYscGG6eBEcFXJGU0xaVyx+Dj4l8Fv3oA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;bh=/3q1dPYIfSnectQZZDWhW+KzjIw535AxCuS18PVtS+A=; b=c6EQ1UggnY0/sMYC0PR03cU4k3sxCcgAynpSj8X8tPtC1VG7Rlaed4Mkz2xkWpD+GLiOj9hWxtirlduF+aTRvl0KdC4biPTciZRyYub5V1CTh/nJYHRk+xEK/Yr7Z4psXJFCMe6grBRVTgTvEHNZTrgt6v1ZGZJFp9U+Bb7swy8=
Received: from HE1PR07MB4425.eurprd07.prod.outlook.com (20.176.162.29) by HE1PR07MB3147.eurprd07.prod.outlook.com (10.170.241.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2814.10; Fri, 13 Mar 2020 20:33:00 +0000
Received: from HE1PR07MB4425.eurprd07.prod.outlook.com ([fe80::e80a:dc35:1cef:7cb9]) by HE1PR07MB4425.eurprd07.prod.outlook.com ([fe80::e80a:dc35:1cef:7cb9%7]) with mapi id 15.20.2835.003; Fri, 13 Mar 2020 20:33:00 +0000
From: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
To: Sebastian Moeller <moeller0@gmx.de>
CC: Ingemar Johansson S <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>, "tsvwg@ietf.org" <tsvwg@ietf.org>, "iccrg@irtf.org" <iccrg@irtf.org>, Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
Thread-Topic: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
Thread-Index: AdX2sUQTt8TR64exRw6bAaEpFrsNEgAD11cAAACWSIAAAOWNgAAD4iVQAAalBwAACxURwABR57CAAEQlqfA=
Date: Fri, 13 Mar 2020 20:32:59 +0000
Message-ID: <HE1PR07MB44257888079995C04EE6EEDEC2FA0@HE1PR07MB4425.eurprd07.prod.outlook.com>
References: <HE1PR07MB44251B019947CDB6602B30B2C2FF0@HE1PR07MB4425.eurprd07.prod.outlook.com><A2300F8D-5F87-461E-AD94-8D7B22A6CDF3@gmx.de> <HE1PR07MB4425B105AFF56D1566164900C2FF0@HE1PR07MB4425.eurprd07.prod.outlook.com> <1C969A05-A4B7-43E9-B694-3195A2FC086A@gmx.de> <HE1PR07MB44255CED94938F9C38515FD6C2FF0@HE1PR07MB4425.eurprd07.prod.outlook.com><AC10E219-46C2-4345-B98F-71689F788B13@gmx.de> <HE1PR07MB4425AA2A7976C1CCF594D3B2C2FF0@HE1PR07MB4425.eurprd07.prod.outlook.com> <8EBC3700-B93B-4077-A052-43CBF1A8B3E5@gmx.de>
In-Reply-To: <8EBC3700-B93B-4077-A052-43CBF1A8B3E5@gmx.de>
Accept-Language: sv-SE, en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
authentication-results: spf=none (sender IP is ) smtp.mailfrom=ingemar.s.johansson@ericsson.com;
x-originating-ip: [83.227.122.88]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: bdd7d6a7-4b65-4ff7-d3bc-08d7c78db8cc
x-ms-traffictypediagnostic: HE1PR07MB3147:|HE1PR07MB3147:
x-ms-exchange-transport-forked: True
x-microsoft-antispam-prvs: <HE1PR07MB314726C9DF7C93370780DF5DC2FA0@HE1PR07MB3147.eurprd07.prod.outlook.com>
x-ms-oob-tlc-oobclassifiers: OLM:5516;
x-forefront-prvs: 034119E4F6
x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6029001)(4636009)(376002)(396003)(136003)(39860400002)(366004)(346002)(199004)(86362001)(7696005)(4326008)(478600001)(6916009)(9686003)(966005)(66574012)(54906003)(8936002)(66556008)(81166006)(107886003)(66476007)(52536014)(6506007)(33656002)(316002)(2906002)(66446008)(186003)(66616009)(81156014)(55016002)(76116006)(26005)(5660300002)(53546011)(9326002)(64756008)(66946007)(71200400001)(8676002); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR07MB3147; H:HE1PR07MB4425.eurprd07.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1;
received-spf: None (protection.outlook.com: ericsson.com does not designate permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: AhV6Yqe/d9mwtO8pyicWsccSZJeljKCz00s/+suBxyg1C7WExI1ofTUwovgHMUA2E9CJAaTVskIMDCwOCBPw1gr+Clv6N/B79Hzh65ell8kk4lyk5ZbI9jhro1EKsEq4Vkj7tAgCh6wNe1LauUfJq/62q5XdfTy9VDgeHk/mEIzKTpGyaYIowSXJJabJ0MmLbSvOl/b1RXVhl7grpeF0CmNP9lfejgsvncAvcosR85LmvOp6ezdpTFmpn1wdd0YKrBOOAOE7UH8m6k8tdpSnw/aYSatwlfQVf4CGYo82upUU+/p00ipsfm+2OgGErlun5yiJZKFm6OejMW/oQ/WH/IV80xVCMZ7GP+fEwu+S+S8lJT0avadpSBNAvrTCI/DfdVTbW9S1q7Fbo0TOyKYTEOn90vs7/hXueO0eZ344rugunM88NMK0EiyNYTsUMLMUUj2GuEmnJe7iQHhWEUrUMukKujexXiM3pKJtIMecR+Oo9QmwFvwCcm/BzTZo7IvJu/MNRjpKXX9E9Wxlnb8yzQ==
x-ms-exchange-antispam-messagedata: oGsbDR7BCF4o+6L5EdT3gJyxrIcURixNSM8KSjILcwkfos0QXZRM5i453oGeZkB8GIXOAAbgDfWqptNdX8OKmHzeixNDbxqn7507J5W368UMRcEtcfwd58kO5FDFppLi/1YpYoy8sWG8shrkG/fDpw==
Content-Type: multipart/signed; protocol="application/x-pkcs7-signature"; micalg="SHA1"; boundary="----=_NextPart_000_051B_01D5F97E.F6884960"
MIME-Version: 1.0
X-OriginatorOrg: ericsson.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bdd7d6a7-4b65-4ff7-d3bc-08d7c78db8cc
X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Mar 2020 20:32:59.9242 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: NMiT3vEJuZr4+2TEjcF5n9eSjTjnps3pTaeYWw3+EMkr9k4BJeLYDgP5xPgabGsE2wBFmtq8qKn+WcOWdFObH3rOotB0hcU7DV0b1b0QSaaEDAfEsZmt598CyiZgxoOQ
X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR07MB3147
Archived-At: <https://mailarchive.ietf.org/arch/msg/iccrg/I7sazf7JkUIcc0Zu7UQ8umSQYT4>
Subject: Re: [iccrg] [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S
X-BeenThere: iccrg@irtf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussions of Internet Congestion Control Research Group \(ICCRG\)" <iccrg.irtf.org>
List-Unsubscribe: <https://www.irtf.org/mailman/options/iccrg>, <mailto:iccrg-request@irtf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/iccrg/>
List-Post: <mailto:iccrg@irtf.org>
List-Help: <mailto:iccrg-request@irtf.org?subject=help>
List-Subscribe: <https://www.irtf.org/mailman/listinfo/iccrg>, <mailto:iccrg-request@irtf.org?subject=subscribe>
X-List-Received-Date: Fri, 13 Mar 2020 20:33:21 -0000

Hi

 

The recent simulations are with SCReAM integrated in our proprietary 5G system simulator. It is however the same code as that available on github. I believe that the example code and the presentation there will give sufficient guidelines on how to integrate it into NS-3.

 

Regards

/Ingemar 

 

From: Sebastian Moeller <moeller0@gmx.de> 
Sent: den 12 mars 2020 12:52
To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com>
Cc: Ingemar Johansson S <ingemar.s.johansson=40ericsson.com@dmarc.ietf.org>; tsvwg@ietf.org; iccrg@irtf.org
Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S

 

Hi Ingemar,

 

Thanks for following up with interesting data.






On Mar 10, 2020, at 22:04, Ingemar Johansson S <ingemar.s.johansson@ericsson.com <mailto:ingemar.s.johansson@ericsson.com> > wrote:

Hi
Attached is a simulation of the same simple bottleneck case with target=1ms and interval=20ms. 
Two ECN beta values : 0.8 and 0.6. A larger backoff (0.6) does actually not improve things as the CWND reduction leads to that packets from the video coder are queued up in the RTP queue. 
As you can see there is still a way to go to reach the L4S delays. 

 

             [SM] I see. But I also note that the way you describe scream's operation as being a bad match for rfc3168's one mark per RTT operations mode, you really want to track the instantaneous congestion state, so more instantaneous signaling is a better match for your application. 






Perhaps the SCReAM's response to CoDel - ECN marks

 

             [SM] Question, in your test-bed/simulations, is there a linux host with a codel qdisc, or is this using seperate simulation code like ns-3?

 





can be optimized further, don't know, but I already spent a considerable time to try and get where the code is now, and I spent a lot more time on this than I spent on the response to the L4S signal. 

 

             [SM] Yes, as far as I can see scream really wants high resolution (low-magnitude) congestion information and hence seems like a perfect match for the higher signaling rates of 1/p-type congestion signaling, like L4S or (dare I say it SCE).

 





My impression is that it is the fractional congestion signal with L4S that makes it easier to get a good algorithm behavior (and I definitely believe that there is room for improvement)

 

             [SM] I would naively guess that is is both the higher signaling rate and the magnitude that helps, your ECN experiments with 90, 80, and 60% backoff indicate that it does not seem to be the magnitude alone.






And as said earlier, the SCReAM code is freely available to experiment with and occasionally it is quite fun (https://www.youtube.com/watch?v=eU1crtEvMv4 ). 

 

             [SM] So to run tests like yours, what else is needed in addition to the contents of the scream repository (and matlab/octave to generate the plots)?

 

Best Regards

             Sebastian

 







/Ingemar




-----Original Message-----
From: Sebastian Moeller <moeller0@gmx.de <mailto:moeller0@gmx.de> >
Sent: den 10 mars 2020 16:29
To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com <mailto:ingemar.s.johansson@ericsson.com> >
Cc: Ingemar Johansson S
<ingemar.s.johansson=40ericsson.com@dmarc.ietf.org <mailto:ingemar.s.johansson=40ericsson.com@dmarc.ietf.org> >; tsvwg@ietf.org <mailto:tsvwg@ietf.org> ;
iccrg@irtf.org <mailto:iccrg@irtf.org> 
Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S

Hi Ingemar,






On Mar 10, 2020, at 13:37, Ingemar Johansson S

<ingemar.s.johansson@ericsson.com <mailto:ingemar.s.johansson@ericsson.com> > wrote:




Hi

The SCReAM code is freely available on

https://protect2.fireeye.com/v1/url?k=3fa4aff0-6370a99d-3fa4ef6b-
867011091b6c-c4d35656f5c9b268&q=1&e=16e2b451-36c7-4814-af9d-
4acb21ac792b&u=https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream
for anybody interested to run their own experiment with whatever AQM/ECN
configuration.

          [SM] Well, if you are not interested in figuring out what is happening
with with rfc3168 ECN ans SCReAM that is your prerogative...





Please note that SCReAM is configured in an L4S mode when the network
AQM does L4S marking (mimicking ECT(1)). For CoDel-ECN however, SCReAM
runs in normal ECN mode with a beta of 0.8 (=20% reduction on CWND for
each congestion event)


          [SM] Okay, that is a far cry from Reno's 50% though, have you tested
different reduction percentages or how did you arrive at 20%?
But is it really 0.8? ScreamTx.h has the following:

// CWND scale factor upon ECN-CE event
static const float kEcnCeBeta = 0.9f;

I am probably missing something somewhere...





I tried also with other different ramp markers (1ms/10ms),

(2ms/10ms),(5ms/15ms).

          [SM] That means for L4S, or do you mean you tried different values for
Codel's target ans ce_threshold parameters?





There are slight variations  in throughput and latency but not dramatic.


          [SM] Well, these are also pretty slight variations in numerical values, at
least if compared to codel's default interval of 100ms...




And truth to be told, the ECN behavior is better tuned in the code than the L4S

behavior.

          [SM] For a defintion of better that ignores that you seem to be much
happier with the results L4S gives you.




There is room for improvement as regards to the L4S behavior (for instance

faster ramp-up) and it may well be the case that SCReAM is completely scrapped
in favor of new designs.



But the bottomline, the L4S thresholds and L4S code is not carefully picked to

show a good performance.

          [SM] So, you tried different L4S parameters (and found the to have
similar performance) but for rfc3168 ECN and Codel you only tried the default
parameters?

Best Regards
          Sebastian





/Ingemar




-----Original Message-----
From: Sebastian Moeller <moeller0@gmx.de <mailto:moeller0@gmx.de> >
Sent: den 10 mars 2020 11:28
To: Ingemar Johansson S <ingemar.s.johansson@ericsson.com <mailto:ingemar.s.johansson@ericsson.com> >
Cc: Ingemar Johansson S
<ingemar.s.johansson=40ericsson.com@dmarc.ietf.org <mailto:ingemar.s.johansson=40ericsson.com@dmarc.ietf.org> >; tsvwg@ietf.org <mailto:tsvwg@ietf.org> ;
iccrg@irtf.org <mailto:iccrg@irtf.org> 
Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S

Hi Ingemar,





On Mar 10, 2020, at 11:07, Ingemar Johansson S

<ingemar.s.johansson@ericsson.com <mailto:ingemar.s.johansson@ericsson.com> > wrote:




Hi

For the future studies we will only focus on L4S as the scope is to
study the

performance gain that L4S give for instance for AR/VR, gaming and
remote control applications.

          [SM] How are you going to "study the performance gain that L4S
give[s]" if you do not compare it with the best of class
alternatives? I am truly puzzled.




Flow aware AQMs with RTT estimates as metadata in the packets is
outside

the scope as it would require packet inspection, which is not
feasible if queues build up on the RLC layer in the 3GPP stack.

          [SM] Fair enough. What is this comparison intended to show us then?

As far as I can see you paired an application designed for 1/p-type
congestion feed-back with an 1/sqrt(p)-type AQM that was also set for
sub-optimal RTT and latency target for the test path. And lo and
behold, the application does "better*" for the 1/p-type AQM (with
lower latency target; I assume that  L4S ramp-marker (Th_low=2ms,
Th_high=10ms) was carefully selected to match what SCReAM expects).
IMHO that simply demonstrates, that in communication it pays if
sender and receiver of a symbol (CE here) assign the same meaning to it.

But that can not be it, sohat am I missing here?

Best Regards
          Sebastian


*) Assuming one buys into your definition of better, in which
(instantaneous) queueing delay is valued over video quality. From a
network operators perspective that seems a valid position








/Ingemar




-----Original Message-----
From: Sebastian Moeller <moeller0@gmx.de <mailto:moeller0@gmx.de> >
Sent: den 10 mars 2020 10:45
To: Ingemar Johansson S
<ingemar.s.johansson=40ericsson.com@dmarc.ietf.org <mailto:ingemar.s.johansson=40ericsson.com@dmarc.ietf.org> >
Cc: tsvwg@ietf.org <mailto:tsvwg@ietf.org> ; Ingemar Johansson S
<ingemar.s.johansson@ericsson.com <mailto:ingemar.s.johansson@ericsson.com> >; iccrg@irtf.org <mailto:iccrg@irtf.org> 
Subject: Re: [tsvwg] SCReAM (RFC8298) with CoDel-ECN and L4S

Hi Ingemar,

thanks for posting this interesting piece of data!




On Mar 10, 2020, at 09:02, Ingemar Johansson S

<ingemar.s.johansson=40ericsson.com@dmarc.ietf.org <mailto:ingemar.s.johansson=40ericsson.com@dmarc.ietf.org> > wrote:




Hi

I recently updated the readme on the SCReAM github with a
comparison with

SCReAM in three different settings



• No ECN
• CoDel ECN
• L4S
https://protect2.fireeye.com/v1/url?k=63019d27-3f884737-6301ddbc-0
cc
47
ad93e2a-489fa99c3277fb8a&q=1&e=5aab95a7-4aab-4a64-99a5-

5b55606e303b&u=



https%3A%2F%2Fgithub.com%2FEricssonResearch%2Fscream%23ecn-

explicit-



co



ngestion-notification

Even though it is more than a magnitude difference in queue delay
between CoDel-ECN and L4S,



[SM] So, in this simulations of a 20ms path, SCReAM over L4S gives
~10 times less queueing delay, but also only ~2 less bandwidth
compared to SCReAM over codel. You describe this as "L4S reduces
the delay considerably more" and "L4S gives a somewhat lower media

rate".



I wonder how many end-users would tradeoff these 25ms in queueing
delay against the decrease in video quality from halving the bitrate?
Could you repeat the Codel test with interval set to 20 and target
to 1ms, please?

If that improves things considerably it would argue for embedding
the current best RTT estimate into SCReAM packets, so an AQM could
tailor its signaling better to individual flow properties (and yes,
that will require a flow-aware AQM).






it is fair to say that these simple simulations should of course
be seen as just a

snapshot.

[SM] Fair enough.




We hope to present some more simulations with 5G access, and not
just

simple bottlenecks with one flow, after the summer.

[Looking] forward to that.





Meanwhile, the SCReAM code on github is freely available for
anyone who

wish to make more experiments.




/Ingemar
================================
Ingemar Johansson  M.Sc.
Master Researcher

Ericsson Research
RESEARCHER
GFTL ER NAP NCM Netw Proto & E2E Perf Labratoriegränd 11
971 28, Luleå, Sweden
Phone +46-1071 43042
SMS/MMS +46-73 078 3289
ingemar.s.johansson@ericsson.com <mailto:ingemar.s.johansson@ericsson.com> 
www.ericsson.com <http://www.ericsson.com> 

Reality, is the only thing… That’s real!
   James Halliday, Ready Player One
=================================