Re: Partially Reliable Message Stream

Roberto Peon <fenix@fb.com> Thu, 31 May 2018 01:03 UTC

Return-Path: <prvs=968909f698=fenix@fb.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2B48A12D870 for <quic@ietfa.amsl.com>; Wed, 30 May 2018 18:03:26 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.719
X-Spam-Level:
X-Spam-Status: No, score=-2.719 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_DKIMWL_WL_HIGH=-0.01, T_DKIMWL_WL_MED=-0.01, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=fb.com header.b=Z6gGa+dN; dkim=pass (1024-bit key) header.d=fb.onmicrosoft.com header.b=BEXsAlRt
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bv8N3Jv43ojZ for <quic@ietfa.amsl.com>; Wed, 30 May 2018 18:03:23 -0700 (PDT)
Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 81B5912DA15 for <quic@ietf.org>; Wed, 30 May 2018 18:03:23 -0700 (PDT)
Received: from pps.filterd (m0001255.ppops.net [127.0.0.1]) by mx0b-00082601.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w4V0v392017377; Wed, 30 May 2018 18:03:18 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : mime-version; s=facebook; bh=amDVm5B/8H94R/Ir+COvnrRlerLOrP0SUyovYyvn3pk=; b=Z6gGa+dNT75ypHxjp6H+smDTeE/cEUCp84xTDXTDU5aYL5wS2cSP0OnUjUHzH7I/tUQs LkR5Ntr7W5FTZdVaA7wlNoTyNSiWQK9PMdjqnbHmlli4815y39zszxgo87lYrRgwGA16 PMeU3JajwSv19Q84ok2lqrqmSUWXclnmnk0=
Received: from maileast.thefacebook.com ([199.201.65.23]) by mx0b-00082601.pphosted.com with ESMTP id 2ja2msgp8w-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 30 May 2018 18:03:18 -0700
Received: from NAM02-CY1-obe.outbound.protection.outlook.com (192.168.183.28) by o365-in.thefacebook.com (192.168.177.29) with Microsoft SMTP Server (TLS) id 14.3.361.1; Wed, 30 May 2018 21:03:16 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=amDVm5B/8H94R/Ir+COvnrRlerLOrP0SUyovYyvn3pk=; b=BEXsAlRtONw0fCvOjOTiYpfhrM3cn/CpHfEj1FZ/+sa6gH9B18NYt2NYCtBmhNj3qnywa98c9XkWxjwCwBALqHq/LNktgLoRX1jqTTDDxYnD9pZ4cQRt7FTI7g9oc1sapDw0pPu/WaXX7CJNl68wCXqxqFudAVWv98fG0jJE9jE=
Received: from BYAPR15MB2312.namprd15.prod.outlook.com (52.135.197.146) by BYAPR15MB2408.namprd15.prod.outlook.com (52.135.198.148) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.797.11; Thu, 31 May 2018 01:03:15 +0000
Received: from BYAPR15MB2312.namprd15.prod.outlook.com ([fe80::949a:9cbd:277c:96e0]) by BYAPR15MB2312.namprd15.prod.outlook.com ([fe80::949a:9cbd:277c:96e0%4]) with mapi id 15.20.0797.020; Thu, 31 May 2018 01:03:15 +0000
From: Roberto Peon <fenix@fb.com>
To: Mike Bishop <mbishop@evequefou.be>, Martin Thomson <martin.thomson@gmail.com>
CC: QUIC WG <quic@ietf.org>, "Lubashev, Igor" <ilubashe=40akamai.com@dmarc.ietf.org>
Subject: Re: Partially Reliable Message Stream
Thread-Topic: Partially Reliable Message Stream
Thread-Index: AdP4Mo3KTicRHsMuS7e4Jrowbk1vdQAMuR2gAAT/n4AAAG2XyQ==
Date: Thu, 31 May 2018 01:03:14 +0000
Message-ID: <BYAPR15MB2312A20C0FFF67119059D2B3CD630@BYAPR15MB2312.namprd15.prod.outlook.com>
References: <d263c36bc5264842a65f04fc3b017538@ustx2ex-dag1mb6.msg.corp.akamai.com> <SN1PR08MB18541BF8B27EA5484FA6A437DA6C0@SN1PR08MB1854.namprd08.prod.outlook.com>, <CABkgnnWUpaFvz3d0SJJ99JkhdQfwdaxYosKwj3dK9BVZUtZ_yw@mail.gmail.com>
In-Reply-To: <CABkgnnWUpaFvz3d0SJJ99JkhdQfwdaxYosKwj3dK9BVZUtZ_yw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [40.67.186.83]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; BYAPR15MB2408; 7:7oqAVlH2zcNYYF+VF382ZjGiiUkHj9EgAdG9LbddZRlNye0ZZk7puBTQGjBdwR14CsrXcT3axf1wKzbrec5ZoAr43LoVBKDzvLYAx5mZJouizDpoi70q7sS3aS8EwDOLBCC2X/2LUFPQKxVPfK6PS0OUopVtY9kN6SSXe5I9skTvxoKH8dbDLIvNEXPYFw02OimyMyr/JBydicFUR2bPVDLug5kpHG1LXQs4uDa3kMHYAPl8Ye694yXMYjN7TyZm; 20:rp6nmXDNxpYBfw0sPJxIMSvLBMVhbbIAe2gBuubI1B+1yO9uUTYae9Bz/NzzDZ8Nvimf4PDgZO/oofZ+aDR/qibXbmdVf3W4Ba71x/PKvGbtNQdm76KkTmG4CnKQSFz9O9qqY7i+CYWRSKZRRW6bVMRaMhzvLQuAOmyvOCJbBNY=
x-ms-exchange-antispam-srfa-diagnostics: SOS;
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:BYAPR15MB2408;
x-ms-traffictypediagnostic: BYAPR15MB2408:
x-microsoft-antispam-prvs: <BYAPR15MB2408E379D30B34D2F9F029D1CD630@BYAPR15MB2408.namprd15.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(190756311086443)(158342451672863)(10436049006162)(85827821059158)(265313219721884);
x-ms-exchange-senderadcheck: 1
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(3231254)(11241501184)(944501410)(52105095)(3002001)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123564045)(20161123558120)(20161123562045)(6072148)(201708071742011)(7699016); SRVR:BYAPR15MB2408; BCL:0; PCL:0; RULEID:; SRVR:BYAPR15MB2408;
x-forefront-prvs: 06891E23FB
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(396003)(346002)(376002)(366004)(39860400002)(39380400002)(189003)(199004)(110136005)(99286004)(54906003)(478600001)(6246003)(3480700004)(105586002)(229853002)(2906002)(45080400002)(106356001)(102836004)(53936002)(66066001)(59450400001)(2900100001)(316002)(81156014)(8676002)(26005)(55016002)(15650500001)(81166006)(186003)(6436002)(86362001)(575784001)(68736007)(486006)(3660700001)(97736004)(5660300001)(3280700002)(476003)(5250100002)(74316002)(6116002)(3846002)(6306002)(9686003)(54896002)(236005)(53546011)(76176011)(8936002)(14454004)(6506007)(25786009)(7696005)(446003)(39060400002)(11346002)(4326008)(606006)(33656002)(7736002)(42262002); DIR:OUT; SFP:1102; SCL:1; SRVR:BYAPR15MB2408; H:BYAPR15MB2312.namprd15.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1;
received-spf: None (protection.outlook.com: fb.com does not designate permitted sender hosts)
x-microsoft-antispam-message-info: +GzY6irqwJLvz0U9j9trvvDH/T9mzVU7EBxUpKTSbFXa0qkh545woM9ZhaSQdpLyeaJBRWM5S1Q3MG+kKovBb+KAq1tTbAaqgh0zA7e4Tel8OYo1uQV3xvYSli/aMquTN3kK82WgOuGLUDxid5081Qd1RiYhcpurTFJIFmiC3EcutzwJWa3NQv51Zd1CSeLZ
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_BYAPR15MB2312A20C0FFF67119059D2B3CD630BYAPR15MB2312namp_"
MIME-Version: 1.0
X-MS-Office365-Filtering-Correlation-Id: 9ddb7c32-163e-40ea-2b10-08d5c69249f2
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ddb7c32-163e-40ea-2b10-08d5c69249f2
X-MS-Exchange-CrossTenant-originalarrivaltime: 31 May 2018 01:03:14.9519 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR15MB2408
X-OriginatorOrg: fb.com
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-30_10:, , signatures=0
X-Proofpoint-Spam-Reason: safe
X-FB-Internal: Safe
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/ryyoVvOj4MaAenT7-DaLXatOHtM>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 31 May 2018 01:03:26 -0000

I find it confusing as well.  I far prefer keeping the offsets monotonically increasing as it allows for real-world use of a partially reliable stream (where you may not recieve that data but you can at least figure out what you did get and interpret it).

I dont know how one wohud resolve the race even within the sending application -- one thread sends, another thread sends while the first says to forget about stuff. I dont know what offset stuff ends up at in such cases, and so I cannot frame data in a way that is non-synchronous. That defeats some of the main reasons to do partial reliability (i.e. to reduce jitter).

Draft 02 seemed to be going in a better direction w.r.t the usecases i can conceive of. I think rewinding to that and then potentially figuring out how to simplify it would be fun!

I'm excited to have this being discussed, since there is pent up demand to use it already.
-=R

Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: QUIC <quic-bounces@ietf.org> on behalf of Martin Thomson <martin.thomson@gmail.com>
Sent: Wednesday, May 30, 2018 5:50:59 PM
To: Mike Bishop
Cc: QUIC WG; Lubashev, Igor
Subject: Re: Partially Reliable Message Stream

Like Mike, I find this gap thing confusing.  And Mike's message
helped, but not a lot.

It took a while to realize that forcing a gap creates a new starting
point for the next message that forces the recipient to recognize and
deal with.  However, the logic for identifying that is just hard and
it's an unnecessary imposition in my view.

Say I send a message with a length of 10 and only two octets make it
onto the wire (as with your example), then it is immediately
abandoned.  Your solution would have the EXPIRED_STREAM_DATA force a
gap after the second octet, so that the next message could start at
offset 3 rather than offset 10.  That saves the remaining 7 octets of
flow control credit.  (Use bigger numbers and perhaps it seems more
motivating).

If the octets that cover the gap were sent, then you have a problem
because now you have sent two versions of the same octets, so you can
only do this if the octets were never sent.  It starts to get closer
and closer to the point where a new stream is just easier.

Worst, I think is that the receiver has to have complicated logic for
identifying that a gap exists and dealing with it.  It has to read up
to what it has, observe that the data has expired, then learn where
the gap ends, then resynchronize based on 1+gap_end for the next
message.

Better to eat the flow control cost in my view.  Or send RST_STREAM
and make another stream.
On Thu, May 31, 2018 at 8:48 AM Mike Bishop <mbishop@evequefou.be> wrote:
>
> I find the one octet gap a bit confusing, but as I think about it, I see why you need it.  If all the stream data arrived successfully (but hadn’t been acknowledged yet due to loss of the ACK, delay, etc.) and the EXPIRED_STREAM_DATA gets lost, the receiver can only retroactively realize there was a jump.  Having an octet that is never transmitted ensures the receiver actually sees a gap, which means that an in-order API will not proceed until it has received either the missing octet or an EXPIRED_STREAM_DATA informing it the octet will never arrive.
>
>
>
> This simplification (versus -02) comes at a price:  If an API were exposing stream data out-of-order, then in your example, the receiver knows that a message always begins on a ten-byte boundary.  A receiver can no longer find ten-byte boundaries, because the offset on the read side doesn’t match the offset on the send side.  I agree with you that this seems like a reasonable trade-off for the simpler flow control.
>
>
>
> One conflict I see in the doc:
>
> ·         Section 3 says:  Receipt of an EXPIRED_STREAM_DATA does not advance the largest received offset for the stream.
>
> Section 5 says:  A receiver SHOULD discard any stream data received for an offset smaller than the new smallest receive offset, possibly advancing the largest received offset for the stream.
>
>
>
> Other minor nits are better done via PR.
>
> From: QUIC <quic-bounces@ietf.org> On Behalf Of Lubashev, Igor
> Sent: Wednesday, May 30, 2018 9:45 AM
> To: QUIC WG <quic@ietf.org>
> Subject: Partially Reliable Message Stream
>
>
>
> I’ve just uploaded a new draft for partially reliable QUIC streams.  Note: this feature is likely not in scope for V1, but it can be an extension for V1 and/or a part of V2.
>
>
>
> The new version 03 (https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dlubashev-2Dquic-2Dpartial-2Dreliability-2D03&d=DwIFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=C0sUo-LFNBaYfyoaCsf6TA&m=yyRpbe14kTYgY413gjVkOalWE_UC3xYerKpJqS8HiQo&s=XZ295xiZsN11yP8GLcdMs1jf7z0b2_WhiIxMLLorZXo&e=) no longer needs complex flow control changes and removes the need to transmit multiple frames in the same packet.
>
>
>
> Igor
>
>
>
> P.S.
>
>   There is also a new version 02, which includes a more complex algorithm with more features and different trade-offs.  But I think version 03 is a better match for the needs so far.