RE: Partially Reliable Message Stream

"Lubashev, Igor" <ilubashe@akamai.com> Thu, 31 May 2018 23:26 UTC

Return-Path: <ilubashe@akamai.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3516A126BF3 for <quic@ietfa.amsl.com>; Thu, 31 May 2018 16:26:23 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.71
X-Spam-Level:
X-Spam-Status: No, score=-2.71 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_DKIMWL_WL_HIGH=-0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=akamai.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kcYhKuOBO-4E for <quic@ietfa.amsl.com>; Thu, 31 May 2018 16:26:21 -0700 (PDT)
Received: from mx0a-00190b01.pphosted.com (mx0a-00190b01.pphosted.com [IPv6:2620:100:9001:583::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 392F2124D37 for <quic@ietf.org>; Thu, 31 May 2018 16:26:21 -0700 (PDT)
Received: from pps.filterd (m0122332.ppops.net [127.0.0.1]) by mx0a-00190b01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w4VNMDfa021872; Fri, 1 Jun 2018 00:26:16 +0100
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akamai.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=jan2016.eng; bh=IZLv24rb4bOWuG93n2pMq0KcdKT84CbzOwYoxqvsp4U=; b=LyC0ZX6Nv7Iec9cNZSu8kNndeLpVxzGH8H9MgEsnmruOfwfoqxKVkLmWuX2UD8zsF6bh J1K6tyP9fIP7bXxqDp4jsPvuU3KZTH6Y/20TK1TY5DiCye5cHdafJqBlPqtYqvboARBh 2iaT7JYOzLgDSztYtypqA376SPVl0R5dV2DTX+GYSHR8hb3pSiTF6EtFRKLJXctT9FAG Pq6lUaGIpaQm63XVG1CoirteK4d12LFHuMtQJU8+8ViTWZ0qp4JiK+nUXcMSqR4MkymN kjp5/SbTRR+EN4624YPvv6cgtO7WlezCCX/spAMfqQ3NUH04fWczdfEF7lc2zGldYDCC hg==
Received: from prod-mail-ppoint4 (a96-6-114-87.deploy.static.akamaitechnologies.com [96.6.114.87] (may be forged)) by mx0a-00190b01.pphosted.com with ESMTP id 2ja5h1vnee-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 01 Jun 2018 00:26:16 +0100
Received: from pps.filterd (prod-mail-ppoint4.akamai.com [127.0.0.1]) by prod-mail-ppoint4.akamai.com (8.16.0.21/8.16.0.21) with SMTP id w4VNQ7RL001861; Thu, 31 May 2018 19:26:15 -0400
Received: from email.msg.corp.akamai.com ([172.27.27.21]) by prod-mail-ppoint4.akamai.com with ESMTP id 2j9cvvc3a1-9 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Thu, 31 May 2018 19:26:15 -0400
Received: from ustx2ex-dag1mb6.msg.corp.akamai.com (172.27.27.107) by ustx2ex-dag1mb3.msg.corp.akamai.com (172.27.27.103) with Microsoft SMTP Server (TLS) id 15.0.1365.1; Thu, 31 May 2018 18:22:33 -0500
Received: from ustx2ex-dag1mb6.msg.corp.akamai.com ([172.27.27.107]) by ustx2ex-dag1mb6.msg.corp.akamai.com ([172.27.27.107]) with mapi id 15.00.1365.000; Thu, 31 May 2018 16:22:33 -0700
From: "Lubashev, Igor" <ilubashe@akamai.com>
To: Martin Thomson <martin.thomson@gmail.com>, Mike Bishop <mbishop@evequefou.be>
CC: QUIC WG <quic@ietf.org>
Subject: RE: Partially Reliable Message Stream
Thread-Topic: Partially Reliable Message Stream
Thread-Index: AdP4Mo3KTicRHsMuS7e4Jrowbk1vdQAMuR2gABOqt4AAIBzWMA==
Date: Thu, 31 May 2018 23:22:32 +0000
Message-ID: <0c68d4cc013743d789b4fff092a7b68a@ustx2ex-dag1mb6.msg.corp.akamai.com>
References: <d263c36bc5264842a65f04fc3b017538@ustx2ex-dag1mb6.msg.corp.akamai.com> <SN1PR08MB18541BF8B27EA5484FA6A437DA6C0@SN1PR08MB1854.namprd08.prod.outlook.com> <CABkgnnWUpaFvz3d0SJJ99JkhdQfwdaxYosKwj3dK9BVZUtZ_yw@mail.gmail.com>
In-Reply-To: <CABkgnnWUpaFvz3d0SJJ99JkhdQfwdaxYosKwj3dK9BVZUtZ_yw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [172.19.32.104]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-31_13:, , signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1805220000 definitions=main-1805310259
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-05-31_13:, , signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1805220000 definitions=main-1805310258
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/o0DmPiWH8VWKSxe43cT4byAaT5Q>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 31 May 2018 23:26:23 -0000

Thanks for the review, Martin!

Yes, all the shenanigans are to limit the _connection_ flow control wastage to a single octet per stream [*].  If you allow for unbounded wastage, you have to actively deal with this (that's what version 02 of the draft does, but it is necessarily more complex for it).

> If the octets that cover the gap were sent, then you have a problem because now you have sent two versions of the same octets

You would not create a gap, if octets right before the "expiration point" were sent.  Because the application only expires on message boundaries, there is no problem if all octets before the "expiration point" (aka. "start of a new message") are actually received.  It would just mean that the receiver application got the expired message; it would not get confused about where a new message starts.

> [...] identifying a gap [...]

For the receiver, identifying a gap (to notify the application of the gap) is actually easy.  Got EXPIRED_STREAM_DATA that covers some offsets you never received?  You've got a gap to notify the application about!



-----Original Message-----
From: Martin Thomson [mailto:martin.thomson@gmail.com] 
Sent: Wednesday, May 30, 2018 8:51 PM
To: Mike Bishop <mbishop@evequefou.be>
Cc: Lubashev, Igor <ilubashe@akamai.com>om>; QUIC WG <quic@ietf.org>
Subject: Re: Partially Reliable Message Stream

Like Mike, I find this gap thing confusing.  And Mike's message helped, but not a lot.

It took a while to realize that forcing a gap creates a new starting point for the next message that forces the recipient to recognize and deal with.  However, the logic for identifying that is just hard and it's an unnecessary imposition in my view.

Say I send a message with a length of 10 and only two octets make it onto the wire (as with your example), then it is immediately abandoned.  Your solution would have the EXPIRED_STREAM_DATA force a gap after the second octet, so that the next message could start at offset 3 rather than offset 10.  That saves the remaining 7 octets of flow control credit.  (Use bigger numbers and perhaps it seems more motivating).

If the octets that cover the gap were sent, then you have a problem because now you have sent two versions of the same octets, so you can only do this if the octets were never sent.  It starts to get closer and closer to the point where a new stream is just easier.

Worst, I think is that the receiver has to have complicated logic for identifying that a gap exists and dealing with it.  It has to read up to what it has, observe that the data has expired, then learn where the gap ends, then resynchronize based on 1+gap_end for the next message.

Better to eat the flow control cost in my view.  Or send RST_STREAM and make another stream.
On Thu, May 31, 2018 at 8:48 AM Mike Bishop <mbishop@evequefou.be> wrote:
>
> I find the one octet gap a bit confusing, but as I think about it, I see why you need it.  If all the stream data arrived successfully (but hadn’t been acknowledged yet due to loss of the ACK, delay, etc.) and the EXPIRED_STREAM_DATA gets lost, the receiver can only retroactively realize there was a jump.  Having an octet that is never transmitted ensures the receiver actually sees a gap, which means that an in-order API will not proceed until it has received either the missing octet or an EXPIRED_STREAM_DATA informing it the octet will never arrive.
>
>
>
> This simplification (versus -02) comes at a price:  If an API were exposing stream data out-of-order, then in your example, the receiver knows that a message always begins on a ten-byte boundary.  A receiver can no longer find ten-byte boundaries, because the offset on the read side doesn’t match the offset on the send side.  I agree with you that this seems like a reasonable trade-off for the simpler flow control.
>
>
>
> One conflict I see in the doc:
>
> ·         Section 3 says:  Receipt of an EXPIRED_STREAM_DATA does not advance the largest received offset for the stream.
>
> Section 5 says:  A receiver SHOULD discard any stream data received for an offset smaller than the new smallest receive offset, possibly advancing the largest received offset for the stream.
>
>
>
> Other minor nits are better done via PR.
>
> From: QUIC <quic-bounces@ietf.org> On Behalf Of Lubashev, Igor
> Sent: Wednesday, May 30, 2018 9:45 AM
> To: QUIC WG <quic@ietf.org>
> Subject: Partially Reliable Message Stream
>
>
>
> I’ve just uploaded a new draft for partially reliable QUIC streams.  Note: this feature is likely not in scope for V1, but it can be an extension for V1 and/or a part of V2.
>
>
>
> The new version 03 (https://tools.ietf.org/html/draft-lubashev-quic-partial-reliability-03) no longer needs complex flow control changes and removes the need to transmit multiple frames in the same packet.
>
>
>
> Igor
>
>
>
> P.S.
>
>   There is also a new version 02, which includes a more complex algorithm with more features and different trade-offs.  But I think version 03 is a better match for the needs so far.