Re: [tsvwg] Switch testing at 25G with ECN --SCE Draft

Dave Taht <> Sat, 10 August 2019 16:32 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id C40A9120111 for <>; Sat, 10 Aug 2019 09:32:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id Y5cn8oYeA72R for <>; Sat, 10 Aug 2019 09:32:28 -0700 (PDT)
Received: from ( [IPv6:2a01:7e00::f03c:91ff:feae:7028]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id E0F53120046 for <>; Sat, 10 Aug 2019 09:32:27 -0700 (PDT)
Received: from (unknown []) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by (Postfix) with ESMTPSA id C2A2D21425; Sat, 10 Aug 2019 16:32:23 +0000 (UTC)
From: Dave Taht <>
To: "Scaglione\, Giuseppe" <>
Cc: Greg White <>, Jonathan Morton <>, "tsvwg\" <>
References: <> <> <AT5PR8401MB07070C672C9F519C05D2D3F599D60@AT5PR8401MB0707.NAMPRD84.PROD.OUTLOOK.COM> <> <AT5PR8401MB070700703FBF2318808238CF99D60@AT5PR8401MB0707.NAMPRD84.PROD.OUTLOOK.COM> <> <> <AT5PR8401MB07074B665CF7612FAD66E13899D60@AT5PR8401MB0707.NAMPRD84.PROD.OUTLOOK.COM>
Date: Sat, 10 Aug 2019 09:32:07 -0700
In-Reply-To: <AT5PR8401MB07074B665CF7612FAD66E13899D60@AT5PR8401MB0707.NAMPRD84.PROD.OUTLOOK.COM> (Giuseppe Scaglione's message of "Fri, 9 Aug 2019 21:42:58 +0000")
Message-ID: <>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Archived-At: <>
Subject: Re: [tsvwg] Switch testing at 25G with ECN --SCE Draft
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sat, 10 Aug 2019 16:32:30 -0000

"Scaglione, Giuseppe" <> writes:

> Greg,
> We are working -- and in beta -- with having the switch natively set SCE bits
> instead of CE and removing the iptable configuration on the target server.  Yet,

Very cool.

> I do not expect any different result since the TCP-SCE stack would react the
> same.

In looking over the data I see that there were tcp "retries" which I
take to be "retransmits", even when there was no loss noted at the

This implies there was packet loss at the client or server (most likely
client, linux not being a R/T OS, getting stuff off the receive ring
rapidly enough is a PITA with napi and other processes interfering)

My guess is that the dctcp version under test did not do anything to
note the loss there and reduce cwnd and had better throughput
sometimes?, and the reno-sce version did?

Regardless tracking down the source of the loss and retransmit and
fixing it would help for a direct comparison. Increasing the rx ring
size, pinning stuff to cpus, and other methods to find where the
mising packets went will lead to more comparable results.

> Regards,
> Giuseppe Scaglione
> -----Original Message-----
> From: Greg White [] 
> Sent: Friday, August 9, 2019 2:36 PM
> To: Jonathan Morton <>om>; Scaglione, Giuseppe <>
> Cc:;
> Subject: Re: [tsvwg] Switch testing at 25G with ECN --SCE Draft
> Right.  Per the SCE method, the switch would mark ECT(1) using a ramp, and then
> when the ramp would exceed 100% marking, it would change to using CE.
> The implementation discussed here marks CE using a ramp, and when the ramp would
> exceed 100% marking, it changes to packet drop.  This, as you said, is simply
> RFC3168 ECN marking, not SCE ECN marking.
> I was just trying to set the record straight.  There was a claim made that a
> switch vendor had implemented SCE-style packet marking in hardware at 25Gbps,
> which wasn't accurate.
> -Greg
> On 8/9/19, 2:58 PM, "Jonathan Morton" <> wrote:
>     > On 9 Aug, 2019, at 11:44 pm, Scaglione, Giuseppe <> wrote:
>     > 
>     >>> Just to be super clear, this isn't a hardware implementation of SCE running at 25Gbps.
>     > 
>     > I am not sure I follow. The Test setup section of the paper clearly describes the hardware used -- severs, switch, cables. 
>     > What I cannot disclose at this point is the exact model and characteristics of the HPE Aruba Switch used. Yet, it is a "real" Ethernet Switch, providing 25Gbps connectivity, configured to do bridging across the four ports and implementing RFC3168 with the ECN remarking configuration described in my previous email and on the paper.
>     I think the issue is with the way the switch itself marks with CE, not ECT(1).  It's a limitation I think is worth acknowledging and, hopefully, finding a way to remove.
>      - Jonathan Morton