Re: [tcpm] Thank you for the QUIC session in tcpm

Praveen Balasubramanian <pravb@microsoft.com> Mon, 12 November 2018 17:38 UTC

Return-Path: <pravb@microsoft.com>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4662B130E89 for <tcpm@ietfa.amsl.com>; Mon, 12 Nov 2018 09:38:32 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.47
X-Spam-Level:
X-Spam-Status: No, score=-2.47 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.47, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=microsoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id G9ubyQUD0vyG for <tcpm@ietfa.amsl.com>; Mon, 12 Nov 2018 09:38:28 -0800 (PST)
Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-eopbgr780094.outbound.protection.outlook.com [40.107.78.94]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 53CAB130E6F for <tcpm@ietf.org>; Mon, 12 Nov 2018 09:38:28 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nVENmkYbdtifDa+ooWUwx38Gt0OcW80Zt2Xy/rk80AA=; b=H27GpB/IvnOQko7EnM6FeLpl+4Mnxnof+nXtre6jMHy+JBO7sOsILUwnnSzAcRNkxv108tE8CFKHLy82SM6bcyrs0IMTkSzsJsf5yCA7gJA5aRjGsBNDu90zBYaFqBGh5rtGPqGX0oRlJqWTXfc1MXc+bJbvvMQRB5y20lNR9kc=
Received: from MW2PR2101MB1049.namprd21.prod.outlook.com (52.132.149.13) by MW2PR2101MB1113.namprd21.prod.outlook.com (52.132.149.30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1361.6; Mon, 12 Nov 2018 17:35:34 +0000
Received: from MW2PR2101MB1049.namprd21.prod.outlook.com ([fe80::89e4:5371:41cc:d24a]) by MW2PR2101MB1049.namprd21.prod.outlook.com ([fe80::89e4:5371:41cc:d24a%4]) with mapi id 15.20.1361.004; Mon, 12 Nov 2018 17:35:34 +0000
From: Praveen Balasubramanian <pravb@microsoft.com>
To: Yuchung Cheng <ycheng@google.com>, Jana Iyengar <jri.ietf@gmail.com>
CC: Bob Briscoe <ietf@bobbriscoe.net>, Ian Swett <ianswett@google.com>, "tcpm@ietf.org Extensions" <tcpm@ietf.org>
Thread-Topic: [tcpm] Thank you for the QUIC session in tcpm
Thread-Index: AQHUd1RO08g6biwnTkKIBqYAsYD8EKVGmkCAgAAPe9CABWndAIAASaGAgAAPTgA=
Date: Mon, 12 Nov 2018 17:35:34 +0000
Message-ID: <MW2PR2101MB1049420039F3E1C96A885A9EB6C10@MW2PR2101MB1049.namprd21.prod.outlook.com>
References: <dddc426c-b7e0-8446-d236-71bdba4010fe@bobbriscoe.net> <CAK6E8=eEQM++TqAS+wLWCwFbXuNcbRZV6Nnewz1+6nWhnfAuQQ@mail.gmail.com> <MW2PR2101MB1049AD006A7311CBB7D9D072B6C60@MW2PR2101MB1049.namprd21.prod.outlook.com> <CACpbDcfcxZBo6A_9yjefhedUWTFe0Ce2eZxyFFa8zvscTRtAKg@mail.gmail.com> <CAK6E8=eAfBMNrCsCLexg2fWxb2dOVOSeFPV81pFPaz+3TdyHsA@mail.gmail.com>
In-Reply-To: <CAK6E8=eAfBMNrCsCLexg2fWxb2dOVOSeFPV81pFPaz+3TdyHsA@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [2001:4898:80e8:f:6ce0:726b:c77c:8d6d]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; MW2PR2101MB1113; 6:D1+eWiXN2d+xvZC+e0gAUJDNMhN0w4LgXRwtaww7sRNXvgf/V8+krq+xH2fB5T1d/7O0WHBXHoL+fQ9DdrnQznSkEoSuLJ7Wp+KAv9S8M3kt68Scn10MmOgjJS3nU1ILpq+hKmVaGsem9E7G1wjnOHwyBe+JWRQFU5D4sNpE7/J7VYoiwMUKI2+n+eTt5i/dWg7TxgUIKIjsUEX9awvwfWVWK6wnulK0l3WHZlo4bXPWCrXbqIpSmEA8r2tzSH4QuK05T/SlqrS3Uwn8d+cjy7iBlIl7va4lASCLAbHPt3bdBMAS56cW2Jl5aHo+QED+T+Cb/QxZmcT5C6FAGeM4nLz9KAqbdhhIogw4UkMFEGortEfbhOYftS4CBaWBObQMt/1oorgYysHOPlOwj5zmpb/QoB3MS6heVjU4sfjr0yQ9Cle/ZotGHF76kh4Xuu01Y0YR+mhh8YQXMRWQ4ml5Ng==; 5:MPFg6RX81bVEV7NVvZ5ySutAxlXwngAh+YTEpUg/do2qL7fOoDvly3J33Yw0dbyPstdQkrL7diSZDq5xl9PwmgkKQ4VRUMqfI7bfhQsD5WgNdCFuEQe18U+JDQIjCE4WsvBY8NgulRIdmQf6dZONEX2JMgQ4e2AWFbwyR77MDBY=; 7:L253Mboq6ad0dt0tUOo49ixH6cYawpO110ZFiV6BfqRGimoTRaLzhc7iJ/zQZPvj4VewRuAHVppTJQH49qYn8xFKHCFC5P31xXcvNHlOOmKKRsBAydWPp+n1URkyVGJRAfL0UyDIQBruLf9Ie1rGng==
x-ms-exchange-antispam-srfa-diagnostics: SOS;
x-ms-office365-filtering-correlation-id: 1595d258-4200-46a6-7da7-08d648c54050
x-ms-office365-filtering-ht: Tenant
x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390040)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(4618075)(2017052603328)(7193020); SRVR:MW2PR2101MB1113;
x-ms-traffictypediagnostic: MW2PR2101MB1113:
x-ms-exchange-purlcount: 4
authentication-results: spf=none (sender IP is ) smtp.mailfrom=pravb@microsoft.com;
x-microsoft-antispam-prvs: <MW2PR2101MB11137525B828A8D1FD7F74A9B6C10@MW2PR2101MB1113.namprd21.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(8220035)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(3002001)(3231402)(944501410)(4982022)(2018427008)(6055026)(148016)(149066)(150057)(6041310)(20161123558120)(20161123564045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(201708071742011)(7699051)(76991095); SRVR:MW2PR2101MB1113; BCL:0; PCL:0; RULEID:; SRVR:MW2PR2101MB1113;
x-forefront-prvs: 0854128AF0
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(136003)(39860400002)(376002)(396003)(346002)(366004)(189003)(199004)(13464003)(478600001)(966005)(33656002)(6436002)(14454004)(10290500003)(25786009)(105586002)(22452003)(54906003)(4326008)(256004)(14444005)(110136005)(71190400001)(71200400001)(39060400002)(93886005)(2900100001)(316002)(229853002)(53936002)(6306002)(9686003)(6246003)(55016002)(7736002)(345774005)(97736004)(8990500004)(68736007)(5660300001)(46003)(11346002)(305945005)(486006)(74316002)(86612001)(86362001)(446003)(476003)(6506007)(102836004)(53546011)(76176011)(99286004)(7696005)(106356001)(81166006)(6116002)(186003)(10090500001)(8676002)(8936002)(81156014)(2906002); DIR:OUT; SFP:1102; SCL:1; SRVR:MW2PR2101MB1113; H:MW2PR2101MB1049.namprd21.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1;
received-spf: None (protection.outlook.com: microsoft.com does not designate permitted sender hosts)
x-microsoft-antispam-message-info: vLg6eZaMBga6aBQsAWjxQs65rrF+6AX7cf22UGQ3LFl+Pz2k6B/pYAxgSrBr/ULvBVyyU9dvPGs3kLDR4YXXTEgVh29sdRvDqfzNB7UYSMHMcMDkOKaOb62e1ySFqO2uzNQLddg8coiLQ6CwYgCHSAK6sH1gS2wxIrcZylJIeGUd1U/IEBnyaY7E7U4M/AJGuVp8Nb3p25/3jP0kQkhhXZHy4ofjkVwlFiyxaLcd0XX8isHpIyKDoJVObNcg/FaJWeblVQGbu7kqOaINE4EsNwtplXUJTpaoxmKK52nSlnVHhvV61TgYvDA7Yy6ePDnxhjXjkgYLBFjwl+IXHuZZC+vngHXkkipQBztRmkvBNMs=
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: microsoft.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1595d258-4200-46a6-7da7-08d648c54050
X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Nov 2018 17:35:34.1706 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW2PR2101MB1113
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/1fUtm7VLkuQ2DOKi-4iCtMGkRIk>
Subject: Re: [tcpm] Thank you for the QUIC session in tcpm
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 12 Nov 2018 17:38:39 -0000

Yuchung this might be worth experimenting with and getting more data on. Maybe there should be some reaction to an unconfirmed RTO even if it is not reduction all the way to LW? I like Jana's suggestion that the implementation SHOULD drop the cwnd even for unconfirmed RTO if not pacing. 

I checked the latest draft and pacing is a RECOMMENDED. I am noticing that transport algorithms in Linux and in QUIC are evolving assuming pacing which is a function usually implemented and configured outside of the transport. So the transport is potentially making an assumption which may result in safety issues. For example BBR congestion control may not work very well if pacing was disabled or misconfigured? I don’t know of any robust solution to this other than build pacing into the transport itself. Pacing is also very challenging for low latency flows because of lack of support for fine grained timers.

-----Original Message-----
From: Yuchung Cheng <ycheng@google.com> 
Sent: Monday, November 12, 2018 8:37 AM
To: Jana Iyengar <jri.ietf@gmail.com>
Cc: Praveen Balasubramanian <pravb@microsoft.com>om>; Bob Briscoe <ietf@bobbriscoe.net>et>; Ian Swett <ianswett@google.com>om>; tcpm@ietf.org Extensions <tcpm@ietf.org>
Subject: Re: [tcpm] Thank you for the QUIC session in tcpm

On Mon, Nov 12, 2018 at 4:13 AM, Jana Iyengar <jri.ietf@gmail.com> wrote:
> Praveen,
>
> The point you're raising -- that we've lost the ack clock after an RTO 
> -- is a reasonable point. My argument is that with pacing, cwnd 
> reduction is unwarranted because in the extreme case, this collapses 
> to restart after idle, where sending at cwnd with pacing is reasonable
I agree w/ Jana as well - the most generic way to treat burst due to prior_inflight << cwnd is pacing. With QUIC's approach, when RTO fires, cwnd remains unchanged until the ACK of the first retransmission (i.e. a probe packet) comes back. Therefore the delay is one RTT and the potential damage is an additional cwnd-worth of burst.

Yes the worst case is more aggressive, and it can be too much for DC incast case if pacing isn't supported - one idea is to selective enable that if RTT variance is very large vs RTT.

>
> The draft does not say that a sender should reduce the cwnd if it is 
> not pacing, which we should add. Does that make sense?
>
> - jana
>
> On Fri, Nov 9, 2018 at 8:34 AM Praveen Balasubramanian 
> <pravb=40microsoft.com@dmarc.ietf.org> wrote:
>>
>> Yuchung I brought that difference up in an email to the quic wg 
>> earlier this week.
>>
>> In app send limited case, inflight could be very small compared to cwnd.
>> So in QUIC there is potential to send a burst out after a long idle 
>> period (with outstanding data) where TCP wouldn't. The draft claims 
>> this is okay to do because RTO may have been a result of RTT increase 
>> instead of loss. Is there data to suggest on which side we should 
>> err? i.e. data on what are the chances that an RTO is due to an RTT increase versus loss.
>>
>> Do you see any safety concerns with delayed reduction of cwnd in case 
>> where the RTO is not spurious?
>>
>> -----Original Message-----
>> From: tcpm <tcpm-bounces@ietf.org> On Behalf Of Yuchung Cheng
>> Sent: Thursday, November 8, 2018 4:38 PM
>> To: Bob Briscoe <ietf@bobbriscoe.net>
>> Cc: Ian Swett <ianswett@google.com>om>; tcpm IETF list <tcpm@ietf.org>
>> Subject: Re: [tcpm] Thank you for the QUIC session in tcpm
>>
>> On Thu, Nov 8, 2018 at 3:14 AM, Bob Briscoe <ietf@bobbriscoe.net> wrote:
>> >
>> > I just wanted to thank Jana for explaining QUIC loss recovery to us 
>> > (and QUIC CC as far as it goes).
>> > And thank you Jana, Ian, the chairs of both WGs (and anyone else
>> > involved) for setting it up.
>> >
>> > If one is not full-time on QUIC, it's very difficult to keep up 
>> > with all the changes. But now we have a checkpoint to start from, I 
>> > feel I will not be wasting people's time if I try to get involved - 
>> > at least I only might say something un-QUIC occasionally, rather 
>> > than nearly always. This has allowed people who understand how TCP 
>> > cold be improved to help with QUIC, when working on QUIC isn't their day job.
>> >
>> > Again, Thank you.
>> I like particularly that QUIC only reduces cwnd to 1 after the loss 
>> is confirmed not upon RTO fires. It should be very feasible for TCP 
>> (at least
>> Linux) w/ TCP timestamps. It'll save a lot of spurious cwnd reductions!
>>
>> Also IMHO TCP w/ quality timestamps are almost as good as QUIC pkt-ids.
>> Google internally uses usec. We wish we could upstream it but RFC 
>> needs to be updated.
>>
>> >
>> >
>> > Bob
>> >
>> >
>> > --
>> > ________________________________________________________________
>> > Bob Briscoe
>> > https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fbob
>> > briscoe.net%2F&amp;data=02%7C01%7Cpravb%40microsoft.com%7C3c4d22866
>> > 8444691c16808d648bd2ce9%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%
>> > 7C636776374678321324&amp;sdata=T%2BWTEc42VC%2Bz%2BsmMFyjlm37hmAwfee
>> > buPcuMYlVhgDY%3D&amp;reserved=0
>> >
>> > _______________________________________________
>> > tcpm mailing list
>> > tcpm@ietf.org
>> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fww
>> > w.i 
>> > etf.org%2Fmailman%2Flistinfo%2Ftcpm&amp;data=02%7C01%7Cpravb%40micr
>> > oso
>> > ft.com%7Cf0911eeb74d7446f424508d645dbb779%7C72f988bf86f141af91ab2d7
>> > cd0 
>> > 11db47%7C1%7C0%7C636773207316711149&amp;sdata=K667a3IQG4rarQ%2FOfAl
>> > yhK
>> > QQ05Cea421rgb64DlEMvs%3D&amp;reserved=0
>>
>> _______________________________________________
>> tcpm mailing list
>> tcpm@ietf.org
>>
>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
>> ietf.org%2Fmailman%2Flistinfo%2Ftcpm&amp;data=02%7C01%7Cpravb%40micro
>> soft.com%7C3c4d228668444691c16808d648bd2ce9%7C72f988bf86f141af91ab2d7
>> cd011db47%7C1%7C0%7C636776374678321324&amp;sdata=33WxKh5c6qL4ln5jerpx
>> Rhytfrj1iTK284pEqv5fWKY%3D&amp;reserved=0
>>
>> _______________________________________________
>> tcpm mailing list
>> tcpm@ietf.org
>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
>> ietf.org%2Fmailman%2Flistinfo%2Ftcpm&amp;data=02%7C01%7Cpravb%40micro
>> soft.com%7C3c4d228668444691c16808d648bd2ce9%7C72f988bf86f141af91ab2d7
>> cd011db47%7C1%7C0%7C636776374678321324&amp;sdata=33WxKh5c6qL4ln5jerpx
>> Rhytfrj1iTK284pEqv5fWKY%3D&amp;reserved=0