Re: FIN + RST behavior

Subodh Iyengar <subodh@fb.com> Thu, 23 March 2017 04:10 UTC

Return-Path: <prvs=4255290284=subodh@fb.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CD113129411 for <quic@ietfa.amsl.com>; Wed, 22 Mar 2017 21:10:29 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.719
X-Spam-Level:
X-Spam-Status: No, score=-2.719 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=fb.com header.b=Hs8iqh/R; dkim=pass (1024-bit key) header.d=fb.onmicrosoft.com header.b=CJSSTARj
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KdR9CvfeQAAV for <quic@ietfa.amsl.com>; Wed, 22 Mar 2017 21:10:27 -0700 (PDT)
Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B1E7E1200C5 for <quic@ietf.org>; Wed, 22 Mar 2017 21:10:27 -0700 (PDT)
Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v2N47xtH017456; Wed, 22 Mar 2017 21:10:25 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : mime-version; s=facebook; bh=rhHGz9qJHod316Eq3+V+eklb5hSLAHNjDp65jvP5qkY=; b=Hs8iqh/RWl5p3YlIZG2Oo2w6ccKsD3sDnefzhS/MSx1m1cSgPNovUbnKwAKe7Qb0vRR6 xhSbp2K7PVZ0b0foGmIDGomv9gNUox3/Mg2tpd4Il2snY8O8vZbqIUPW/rkVCuTssCAh z5pb4LkaoS7MojuO2YrZPcbPl7DtwduIKHk=
Received: from mail.thefacebook.com ([199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 29byxf9a1d-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 22 Mar 2017 21:10:25 -0700
Received: from NAM03-CO1-obe.outbound.protection.outlook.com (192.168.54.28) by o365-in.thefacebook.com (192.168.16.14) with Microsoft SMTP Server (TLS) id 14.3.319.2; Wed, 22 Mar 2017 21:10:23 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=rhHGz9qJHod316Eq3+V+eklb5hSLAHNjDp65jvP5qkY=; b=CJSSTARjo3dsPXHmd6DTaWfZzIq2QcOCmDYbOn6F4WdBgwGF7IzF6CPx3LtyY+nnkkuExOGDt6IYozseEGmtygxBMtYckuw/qdWlG2Sz4ZafTmEwJlS9tnhYVCdCClOw/XKHbMDOR9tEbiuyTJbYgs+/yWLYxs1anKoLt8QL818=
Received: from MWHPR15MB1455.namprd15.prod.outlook.com (10.173.234.145) by MWHPR15MB1456.namprd15.prod.outlook.com (10.173.234.146) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.977.11; Thu, 23 Mar 2017 04:10:21 +0000
Received: from MWHPR15MB1455.namprd15.prod.outlook.com ([10.173.234.145]) by MWHPR15MB1455.namprd15.prod.outlook.com ([10.173.234.145]) with mapi id 15.01.0977.020; Thu, 23 Mar 2017 04:10:21 +0000
From: Subodh Iyengar <subodh@fb.com>
To: Martin Thomson <martin.thomson@gmail.com>
CC: "quic@ietf.org" <quic@ietf.org>
Subject: Re: FIN + RST behavior
Thread-Topic: FIN + RST behavior
Thread-Index: AQHSo1zPcZTZN/tlZkuJR+NHZURtjKGhltKAgAAu7CA=
Date: Thu, 23 Mar 2017 04:10:21 +0000
Message-ID: <MWHPR15MB145521D729498455F923679EB63F0@MWHPR15MB1455.namprd15.prod.outlook.com>
References: <MWHPR15MB1455B5E6D4F4FAC8411927E4B63C0@MWHPR15MB1455.namprd15.prod.outlook.com>, <CABkgnnVRo_mT7sPO=xvmMdS-6wAJoSpARp+FG1R8i-RQWqE8Lg@mail.gmail.com>
In-Reply-To: <CABkgnnVRo_mT7sPO=xvmMdS-6wAJoSpARp+FG1R8i-RQWqE8Lg@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-Mentions: martin.thomson@gmail.com
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
authentication-results: gmail.com; dkim=none (message not signed) header.d=none;gmail.com; dmarc=none action=none header.from=fb.com;
x-originating-ip: [25.172.112.132]
x-microsoft-exchange-diagnostics: 1; MWHPR15MB1456; 7:CDUcstSyiPON5+X/LtLOKil61X7skd6MmX7ZzQEWAXlSCqdxLGOG0oL3TOrIBTe37tymX2bGCYCoEh42bcCCE6YGoBUdTZuMfVg7ki9+w4pSX4c+pPizVo6jmjiXBrpG1WrQmOFhAQvM+sakfq12P4RhLdQOJFAcFWZadIb+lOR+ESn9Z2zEK+gUjH59KB3LaKk3jXqhClSnyGS0mRqzDh7PYoYiGG5yZY/q5ddAWln0BJUJ3c2TxIWzp7Ik6jOWtjZNCSFRebdIUEgzmagla86stCWw0avhZzji8vhDcXb0j3M8qthMoRLaC3DB/QGZYGQTGZiiyXa/xrgq5wjavA==; 20:nLcb+DqR0HKKq0BRXTx0p/qcmYqCFr6AbTK7YMXI1Wp2Pw15y4PsXtCwSLCfdGdAdj2NBdfybwYDDLYlWvHmyiwgGZALNzPVV6WR6cfd3lRwmZxEvx7rGPN3gl8c+ppq3Jg4yyOe1BaHstMDdWbrZTCoTHAwYS/WtfWaoDe22MU=
x-ms-office365-filtering-correlation-id: 808944f4-26e0-43be-859c-08d471a28633
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254075); SRVR:MWHPR15MB1456;
x-microsoft-antispam-prvs: <MWHPR15MB145665E6576C046AC8B98300B63F0@MWHPR15MB1456.namprd15.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(158342451672863)(166708455590820)(67672495146484);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040375)(601004)(2401047)(8121501046)(5005006)(10201501046)(3002001)(6041248)(20161123558025)(20161123564025)(20161123555025)(20161123562025)(20161123560025)(6072148); SRVR:MWHPR15MB1456; BCL:0; PCL:0; RULEID:; SRVR:MWHPR15MB1456;
x-forefront-prvs: 0255DF69B9
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(39450400003)(39830400002)(39410400002)(377454003)(24454002)(74316002)(3846002)(102836003)(6116002)(8676002)(6436002)(50986999)(122556002)(7906003)(7736002)(81166006)(66066001)(53546009)(33656002)(54356999)(8936002)(3280700002)(25786009)(76176999)(5660300001)(7696004)(2950100002)(86362001)(9686003)(77096006)(55016002)(6306002)(54896002)(99286003)(53936002)(606005)(6916009)(229853002)(236005)(110136004)(6246003)(6506006)(4326008)(38730400002)(39060400002)(2900100001)(2906002)(189998001)(3660700001); DIR:OUT; SFP:1102; SCL:1; SRVR:MWHPR15MB1456; H:MWHPR15MB1455.namprd15.prod.outlook.com; FPR:; SPF:None; MLV:ovrnspm; PTR:InfoNoRecords; LANG:en;
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_MWHPR15MB145521D729498455F923679EB63F0MWHPR15MB1455namp_"
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Mar 2017 04:10:21.3404 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR15MB1456
X-OriginatorOrg: fb.com
X-Proofpoint-Spam-Reason: safe
X-FB-Internal: Safe
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-03-23_03:, , signatures=0
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/7EjUL_z39-n5RBbNM1uQEysLpO8>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 23 Mar 2017 04:10:30 -0000

@Mike Bishop<mailto:Michael.Bishop@microsoft.com>
> I don’t see these as in conflict.  If the client receives a RST when it has already sent a FIN, it's not subject to the MUST.  It may, of course, opt to send a RST and abort retransmission.  Or, it may choose to just let the sent data be, but with the obligation to retransmit it.  Its obligation is to deliver the final offset, and all data up to that offset if the offset is in a FIN.

To confirm that I'm interpreting you correctly I'm going to try and rephrase that a bit: The intent of the draft is that if the client receives the RST stream and it has already sent a FIN, it should ignore the RST, and perform retransmission of all the packets as it usually would, even if the server is no longer interested in it. If it wanted to not retransmit the packets, it should send a RST of it's own.

@Martin Thomson<mailto:martin.thomson@gmail.com>
>  A sender needs to wait for acknowledgment before transitioning to new
states.  Receivers just process and acknowledge.
Ya that's definitely a given. The issue here I was asking about was what if there would never be an ACK for the FIN because the client decided to abort the retransmissions.

>  Both endpoints need to end their stream in one
way or other.
That's definitely true but it seems like closing a stream which is not initiated by you has a different behavior with respect to closing streams initiated by you because you need to account for
stream concurrency limit and connection flow control window differently.

> Here's the short version:
Ya this is definitely a very simplistic description of what needs to be done when a connection needs to be closed and doesn't account for how to keep the client + server in sync for the concurrency limit
and connection flow control window which is also very important. My question was actually about the latter.

This stuff gets hairy fast.

For example in a clean close:
1) client sends FIN to server, sever gets FIN
2) Server sends ACK for the client's FIN (client is now half closed local)
2) server sends data and FIN (Server doesn't change state, it waits for the ACK of all the bytes + FIN)
3) client gets all data and FIN and then ACKs all of it.
4) the client ACK for the server FIN gets lost

>From the client's perspective it sent out all the ACKs for the FIN and doesn't know whether they got lost.
How does the client know when the server has released a slot in it's concurrency limit? The server can't immediately release this when it sends out the FIN because it still must keep state until it receives all the
ACKs for the stream. I think the client need to keep track of the ACK of the ACKs of all the server sent bytes in the stream, which would be a bit sad.

Subodh


________________________________
From: Martin Thomson <martin.thomson@gmail.com>
Sent: Wednesday, March 22, 2017 5:45:31 PM
To: Subodh Iyengar
Cc: quic@ietf.org
Subject: Re: FIN + RST behavior

This is just an error.  Both endpoints need to end their stream in one
way or other.  The text about not sending a RST_STREAM needs to take
into account the fact that any FIN or RST_STREAM that was previously
sent ALSO needs to be acknowledged.

A sender needs to wait for acknowledgment before transitioning to new
states.  Receivers just process and acknowledge.

I agree that this is poorly documented.  Here's the short version:

Each endpoint has to end the stream one way or other.  This
establishes the final data offset for the stream, which is necessary
for connection flow control accounting.  A stream can be ended by
either sending a STREAM frame with the FIN flag set, or a RST_STREAM
frame.  A stream isn't ended in the sending direction until a packet
containing one of those two frames has been acknowledged.  An endpoint
MAY choose to switch from sending STREAM frames to sending RST_STREAM
frames at any time.

For the long version, you will have to wait until we get around to
fixing https://github.com/quicwg/base-drafts/issues/413

On 23 March 2017 at 09:37, Subodh Iyengar <subodh@fb.com> wrote:
> There might be some strange cases around RSTs and FINs which I would love
> more clarity on.
>
> Let's say that the following actions happen:
> 1) Client initiates a stream to the server and sends some data
> 2) Server sends a FIN to the client which the client receives. The client
> and server are both half closed (remote and local respectively).
> 3) After some time the client sends a FIN to the server. So it's waiting for
> the FIN and the ack of all the data before closing the stream.
> 4) Server sends a RST which races with the client sending the FIN.
> 5) Client gets a RST
>
> From the draft I think it's ambiguous on what the client is supposed to do
>
> "An endpoint that receives a RST_STREAM frame (and which has not sent a FIN
> or a RST_STREAM) MUST immediately respond with a RST_STREAM frame, and MUST
> NOT send any more data on the stream"
>
> So clearly a client should not send a RST stream in response
>
> However:
> "When an endpoint sends a RST_STREAM frame, data outstanding on that stream
> SHOULD NOT be retransmitted, since subsequent data on this stream is
> expected to not be delivered by the receiver."
>
> If the client decides to honor both of these requirements I believe it will
> not retransmit the FIN. So if the FIN gets lost the client will have no idea
> whether the server has seen the FIN and reduced its
> number of streams counting towards the concurrency limit so that the client
> can open up a new stream. It would need to track the ACK of the ACK of the
> RST stream that the server sends back.
>
> Am I missing another requirement that prevents this from being an issue or
> could "not sent a FIN" mean that "not sent a FIN that has been ACKed".
>
> Subodh
>