Re: [tcpm] Questions about TLP

Yi Huang <> Mon, 06 May 2019 22:20 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id B2E2A1200A4 for <>; Mon, 6 May 2019 15:20:48 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: 1.602
X-Spam-Level: *
X-Spam-Status: No, score=1.602 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=1.989, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, T_DKIMWL_WL_HIGH=-0.01, URIBL_BLOCKED=0.001, URI_HEX=1.122, URI_NOVOWEL=0.5] autolearn=no autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id IzD8PaeJwuV2 for <>; Mon, 6 May 2019 15:20:45 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 52498120098 for <>; Mon, 6 May 2019 15:20:45 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=testarcselector01;; cv=none; b=OGkTBb0K/3J/QXgJKZIBXkG/xX2sTE+B0SBO2c946sawQYCpEBqZFieXl/nUpCQGFsz5ZcQKx581gZ6r9qXFpNVaV2Ac6OEH5nupSkN/1WDS4A50dyl9PQTj0L7/2WupqbpW2vbIRO5zmKCInf53NlOfNuygMLIPMOKj9cc6FFA=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed;; s=testarcselector01; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GoZPZU82DNHS7ojUOT9sTjFSM3qVJkvWRqUWajSFD54=; b=Gd7NmEsgUCoCGoo4lIeWvsq3/JmCKBpGfG8dP3R0kpEkgNHr77mGVLWN3ZKAoWUxyA75JfezMjWBaurQYtPf9D/CnVzHuRmOj9GLOZz+XmFIzD78lzYJypTfiL6tC+Vmuo7kGgJ43NupFpOAnbbrZtX0davoMNc46ZZp4TsDS8o=
ARC-Authentication-Results: i=1; 1;spf=none;dmarc=none;dkim=none;arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GoZPZU82DNHS7ojUOT9sTjFSM3qVJkvWRqUWajSFD54=; b=GeBAFVZwspy0x5AOfT06rfd/BzDebC/fN60hrKOQa+OIQ1IFqnHO10sa0qTCYZWhscC1as4GpTmHN5MjqXWedxQoyQzb2cETruAHpgmQUlC4Nf7uEVf+lxSVrx97Gk1wuOKO8nimL+l2o+5cuKUVRnJbRkIcvMV9VaWbL8cCVII=
Received: from ( by ( with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1900.4; Mon, 6 May 2019 22:20:41 +0000
Received: from ([fe80::2056:cdb6:e2aa:e45a]) by ([fe80::2056:cdb6:e2aa:e45a%9]) with mapi id 15.20.1900.002; Mon, 6 May 2019 22:20:41 +0000
From: Yi Huang <>
To: Yuchung Cheng <>
CC: Yuchung Cheng <>, Matt Olson <>, "" <>, Neal Cardwell <>
Thread-Topic: [tcpm] Questions about TLP
Date: Mon, 6 May 2019 22:20:41 +0000
Message-ID: <>
References: <> <> <> <> <> <>
In-Reply-To: <>
Accept-Language: en-US
Content-Language: en-US
msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=True; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47;; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2019-05-06T22:20:39.4904300Z; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=General; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Application=Microsoft Azure Information Protection; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=b31c7480-4fa1-413c-80ea-3f7503265918; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Extended_MSFT_Method=Automatic
authentication-results: spf=none (sender IP is );
x-originating-ip: [2001:4898:80e8:2:c52:a508:8c4c:b31e]
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: 4efacf08-8588-449a-be00-08d6d2711342
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(4618075)(2017052603328)(7193020); SRVR:BL0PR2101MB0881;
x-ms-traffictypediagnostic: BL0PR2101MB0881:
x-ms-exchange-purlcount: 6
x-microsoft-antispam-prvs: <>
x-ms-oob-tlc-oobclassifiers: OLM:9508;
x-forefront-prvs: 0029F17A3F
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(396003)(39860400002)(376002)(346002)(136003)(366004)(13464003)(199004)(189003)(51914003)(476003)(66556008)(11346002)(66446008)(66946007)(73956011)(446003)(66476007)(64756008)(486006)(606006)(76116006)(46003)(33656002)(236005)(186003)(5660300002)(52536014)(25786009)(6116002)(10090500001)(2906002)(10290500003)(790700001)(86612001)(86362001)(54906003)(256004)(14444005)(6436002)(966005)(8676002)(55016002)(81166006)(81156014)(7736002)(71200400001)(71190400001)(478600001)(6306002)(54896002)(4326008)(9686003)(68736007)(6246003)(22452003)(76176011)(229853002)(99286004)(102836004)(6506007)(53546011)(53936002)(316002)(8990500004)(8936002)(7696005)(74316002)(14454004); DIR:OUT; SFP:1102; SCL:1; SRVR:BL0PR2101MB0881;; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1;
received-spf: None ( does not designate permitted sender hosts)
x-ms-exchange-senderadcheck: 1
x-microsoft-antispam-message-info: x/SwDOKmhMLosb2RLRT46E0zc4KSv8UhviLjl9gq/nrkUvA34nCQnBhNX7x07DzEH8UoET/Y3sSfHeZtaFTQwafftHnXJ1unFYwmzmbrL4DlHud5Ve3lXKQEOqNL8SedBcG3nNYA7nx6c+QKJENiUxcVBAGzQE60y7fcldQkxK7+VxLaWJEiN4ELdQ9M0fubxxEKyhCdP4Bba5tw2G2g8yZZhFP4RRKqTvzxVYMWRsAS54NiXEfcAsTO2tJlIJ7VrMiUsOMq6M8VbeQjlMvy7twi2zf/+Ff46ajAkKlQ/8jv7IeZtO2ONs+QH7okdZaqxp21yEh4UZXZEVPRBX/xBOqeZwfP7fZ3LmqtoYGdslH8QjFRHfGbf+FB/WQvRls051zc8wvsuR8jhcNLFy8w0yEBys3NeZdvsHJZiFHrO04=
Content-Type: multipart/alternative; boundary="_000_BL0PR2101MB1043893F691DED20889B3C45C3300BL0PR2101MB1043_"
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 4efacf08-8588-449a-be00-08d6d2711342
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 May 2019 22:20:41.3253 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR2101MB0881
Archived-At: <>
Subject: Re: [tcpm] Questions about TLP
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Mon, 06 May 2019 22:20:49 -0000

Thanks for the clarification. The proposed rev of the statement is much easier to understand.


From: Yuchung Cheng <>;
Sent: Monday, May 6, 2019 2:28 PM
To: Yi Huang <>;
Cc: Yuchung Cheng <>;; Matt Olson <>;;; Neal Cardwell <>;
Subject: Re: [tcpm] Questions about TLP

Sorry for the late reply. Forget to hit send after drafting the reply...

Thanks for spotting that. TCP_send_probe() should always re-arm the
RTO regardless of a TLP is sent or not, if there's unacknowledged
data. Proposed rev in

"Note that the sender MUST arm an RTO timer and not the PTO timer at the end of TLP_send_probe() as long as FlightSize is not zero, regardless of sending a probe or not. This ensures that the sender does not send repeated, back-to-back TLP probes. Also checking TLPRxtOut prior to sending the loss probe is important to avoid TLP loops if an application writes periodically at an interval less than PTO."

From: Yi Huang <<>>
Date: Thu, May 2, 2019 at 11:47 AM
To: Yuchung Cheng, Matt Olson
If TLPRxtOut is checked in TLP_send_probe(), a new PTO can cancel an already running RTO timer armed by sending TLP previously. If TLP_send_probe() decides not to send the probe for the new PTO, what should the sender do next? Rearm RTO or treat it as RTO timeout? The draft states we must arm an RTO after transmitting TLP but in this case, no probe will be sent.

Also, in page 17, the draft says "This is important to avoid TLP loops if an application writes periodically at an interval less than PTO." but if the app writes periodically at an interval less than PTO, PTO will just be pushed out and not fire until the sender cannot send anything due to window limit. Then in this case, TLP loops do not seem to exist if TLP loops means back-to-back TLP probes. Am I missing anything here?



-----Original Message-----
From: Yuchung Cheng <<>>
Sent: Thursday, May 2, 2019 10:25 AM
To: Matt Olson <<>>
Cc: Yi Huang <<>>;<>
Subject: Re: [tcpm] Questions about TLP

On Thu, May 2, 2019 at 10:09 AM Matt Olson <<>> wrote:
> Also, section 6.5.1 Step 1 is presented as a complete set of conditions for scheduling a PTO, but it is missing a bullet point for TLPRxtOut.
Thanks for checking: but checking TLPRxtOut is not necessary and is not preferred when scheduling a probe -- the goal is to prevent more than one probe inflight, so checking right before TLPRxtOut sending the next one allows the best coverage of application-limited writes (burst-idle-burst-....).

> -----Original Message-----
> From: Yuchung Cheng <<>>
> Sent: Wednesday, May 1, 2019 11:33 PM
> To: Yi Huang <<>>
> Cc:<>; Matt Olson <<>>
> Subject: Re: [tcpm] Questions about TLP
> On Wed, May 1, 2019 at 10:48 PM Yi Huang <<>> wrote:
> >
> > Hi RACK/TLP authors,
> >
> >
> >
> > I have the following questions (page numbers refer to draft-ietf-tcpm-rack-05):
> >
> >
> >
> > 1.In page 16, it says “Finally, if the time at which an RTO would fire (here denoted "TCP_RTO_expire") is sooner than the computed time for the PTO, then a probe is scheduled to be sent at that earlier time.” What does this TCP_RTO_expire actually mean? Does it mean there is another RTO timer (possibly started some time in the past) running along with PTO or just RTT + 4*RTTVar + Now()? Also, why would another probe be sent at RTO expiration time instead of treating it like RTO and collapsing the cwnd?
> TCP_RTO_expire() should return a timeout value for regular RTO, i.e.
> SRTT + 4*RTTVAR + Now()
> We use TCP_RTO_expire() because many implementations including Linux
> do not use the exact RFC formula
> The reason it's not treated as a regular RTO is to avoid resetting congestion window to 1. The rationale is if the probe was sent within min(PTO, RTO) and was delivered successfully, there's no need to reset congestion window and re-start slow-start, similar to the rationale of Reno's fast recovery reducing window to ssthresh instead of 1. We have found this strategy benefits wireless connections, as the chance of spurious RTO is high due to the delay variation.
> >
> >
> >
> > 2.In page 18 section 6.6, a new variable TLPRxtOut is defined and it is stated that TLPRxtOut is used to guarantee that there is only one outstanding TLP retransmission.. However, it is not clear to me when TLPRxtOut should be really used. Should it be used in 6.5.1 Step. 1 (checking whether we should and can schedule a TLP) or in TLP_send_probe()?
> This is in
> 6.6.2.  Recording loss probe states
>    Senders MUST only send a TLP loss probe retransmission if TLPRxtOut
>    is false.  This ensures that at any given time a connection has at
>    most one outstanding TLP retransmission.  This allows the sender to
> but I agree it'd be more clear to include this in TLP_send_probe() pseudo code. I'd incorporate in the next rev.
> >
> >
> >
> > Thanks,
> >
> >
> >
> > Yi
> >
> >
> >
> > _______________________________________________
> > tcpm mailing list
> ><>
> >
> ><>%2Fmailman%2Flistinfo%2Ftcpm&amp;data=01%7C01%7Cmaolson%40mi
> > cr
> ><>%7C66db44672e3d4101d72908d6cec82001%7C72f988bf86f141af91ab2
> > d7
> > cd011db47%7C1&amp;sdata=n8rp5Emowm459iSKolhIP1hFXBs8pZjsGG2iy3OBAno%
> > 3D
> > &amp;reserved=0
tcpm mailing list<><>