Re: [nwcrg] [LOOPS] IETF106: LOOPS side meeting on Tuesday 08:30, Orchard Room

"Zhouxingwang (Joe)" <> Mon, 06 January 2020 08:47 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 94DDF12001E for <>; Mon, 6 Jan 2020 00:47:25 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -4.2
X-Spam-Status: No, score=-4.2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 2Npysadl9UYn for <>; Mon, 6 Jan 2020 00:47:22 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id B79B012009C for <>; Mon, 6 Jan 2020 00:47:21 -0800 (PST)
Received: from (unknown []) by Forcepoint Email with ESMTP id 4268A27C3CF346369A5C; Mon, 6 Jan 2020 16:47:18 +0800 (CST)
Received: from ( by ( with Microsoft SMTP Server (TLS) id 14.3.439.0; Mon, 6 Jan 2020 16:47:17 +0800
Received: from ( by ( with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1713.5; Mon, 6 Jan 2020 16:47:17 +0800
Received: from ([]) by ([]) with mapi id 15.01.1713.004; Mon, 6 Jan 2020 16:47:17 +0800
From: "Zhouxingwang (Joe)" <>
To: Stuart William Card <>, Carsten Bormann <>
CC: "" <>, "" <>, "" <>
Thread-Topic: [nwcrg] [LOOPS] IETF106: LOOPS side meeting on Tuesday 08:30, Orchard Room
Thread-Index: AQHVxA5wn08ZkzTEl06o0CselVnRRafcV/gAgAD4cVA=
Date: Mon, 6 Jan 2020 08:47:17 +0000
Message-ID: <>
References: <> <> <> <> <> <> <> <> <>
In-Reply-To: <>
Accept-Language: zh-CN, en-US
Content-Language: zh-CN
x-originating-ip: []
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-CFilter-Loop: Reflected
Archived-At: <>
Subject: Re: [nwcrg] [LOOPS] IETF106: LOOPS side meeting on Tuesday 08:30, Orchard Room
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IRTF Network Coding Research Group discussion list <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 06 Jan 2020 08:47:25 -0000


It seems Linux TCP is much more out-of-order tolerant than we think.

1) TCP can dynamically adjust flow reordering level, not stick to 3. (
tcp_reordering - INTEGER Initial reordering level of packets in a TCP stream.
	TCP stack can then dynamically adjust flow reordering level
	between this initial value and tcp_max_reordering
	Default: 3
tcp_max_reordering - INTEGER Maximal reordering level of packets in a TCP stream. 300 is a fairly conservative value, but you might increase it
	if paths are using per packet load balancing (like bonding rr mode) Default: 300

2) RACK has been enabled by default in Linux for loss detection since kernel4.4. ( 
tcp_recovery - INTEGER
	This value is a bitmap to enable various experimental loss recovery

	RACK: 0x1 enables the RACK loss detection for fast detection of lost
	      retransmissions and tail drops. It also subsumes and disables
	      RFC6675 recovery for SACK connections.
	RACK: 0x2 makes RACK's reordering window static (min_rtt/4).
	RACK: 0x4 disables RACK's DUPACK threshold heuristic

	Default: 0x1

The retransmission conflict between TCP and other layer will not be a big issue.


-----Original Message-----
From: nwcrg [] On Behalf Of Stuart William Card
Sent: Monday, January 06, 2020 9:46 AM
To: Carsten Bormann <>
Subject: Re: [nwcrg] [LOOPS] IETF106: LOOPS side meeting on Tuesday 08:30, Orchard Room

Carsten --

Thanks for your thoughtful response. I'll try to focus my inputs. :-)

> ...
> (1) ... more intricate (and timely) knowledge of the critical path
> (2) techniques such as TCP splitting allowed mitigating some...
> of the latency induced performance issues end-to-end protocols exhibited...
> With increasing use of encryption... (2) is going away...
> (1) will stay relevant... where that knowledge can be provided...
> What LOOPS is trying to do is providing a protocol that can help even 
> in the absence of both (1) and (2)...
> recovering packets can lead to reordering...
> applications will need to be part of the picture...
> LOOPS itself will need to focus on a more narrow set of problems and 
> opportunities, and this is the time to discuss whether we got that 
> narrowing right in the current charter proposal...

Agreed. Further, not only transport but also network layer encryption kills simple approaches to (2), leaving AFAIK only multipath or trading increased mean latency to reduce latency variance.

Many of us have suffered from suboptimal and sometimes unanticipated interactions between link and transport layer behaviors (retransmission in both, flow control in the data link, congestion control in the transport).
Application layer adaptive behaviors (e.g. video streaming that switches dynamically during a session among different source coding rates based upon end to end performance as it varies) further complicate this. LOOPS will be yet another interacting control loop. In the worst case, a non-congestion loss of a link layer frame could lead to:
- data link retransmission
- LOOPS retransmission
- transport retransmission
- transport backoff
- application rate downshift
All at once! Presumably we can't prevent this altogether but want to minimize its occurrence.

The charter looks good. I suggest broadening its scope of consideration of cross-layer interactions: not just w/transport congestion control, but also w/retransmission, misordering & resequencing at all layers.

Stuart W. Card, VP & Chief Scientist
Critical Technologies Inc. (CTI)

nwcrg mailing list