Re: flow control and DATAGRAM

Roberto Peon <fenix@fb.com> Mon, 29 October 2018 23:06 UTC

Return-Path: <prvs=484051c544=fenix@fb.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id EB316131028 for <quic@ietfa.amsl.com>; Mon, 29 Oct 2018 16:06:52 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: 3.828
X-Spam-Level: ***
X-Spam-Status: No, score=3.828 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.47, DKIMWL_WL_MED=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, GB_SUMOF=5, KHOP_DYNAMIC=1.999, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=fb.com header.b=DyLW0bdW; dkim=pass (1024-bit key) header.d=fb.onmicrosoft.com header.b=b0QxPXUN
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id D9oiMLtQRwQG for <quic@ietfa.amsl.com>; Mon, 29 Oct 2018 16:06:50 -0700 (PDT)
Received: from mx0a-00082601.pphosted.com (mx0b-00082601.pphosted.com [67.231.153.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 78E68131021 for <quic@ietf.org>; Mon, 29 Oct 2018 16:06:50 -0700 (PDT)
Received: from pps.filterd (m0001255.ppops.net [127.0.0.1]) by mx0b-00082601.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w9TN2LMs027518; Mon, 29 Oct 2018 16:06:46 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=facebook; bh=kFDixJGpPF9KKFTKIm8L6dnA3fEKxa5LxZym/tOeqIw=; b=DyLW0bdWkiSCfhAOLwNRcSSIR0ydKjPFHRtRdtXUzbp9UWIrwXZ//MpFlsTUiD0uFu40 HuJDJlpnoUT0xW+qqvI1tJ1WuM9Bk3COb/xGMp9w8xWktefNvzb/XC+WbjKqmOiZgIY/ xrIGu2XW7Eps88/R0H3pVvJKIaJg539ZRrk=
Received: from mail.thefacebook.com ([199.201.64.23]) by mx0b-00082601.pphosted.com with ESMTP id 2ne8u7gndq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 29 Oct 2018 16:06:46 -0700
Received: from prn-hub03.TheFacebook.com (2620:10d:c081:35::127) by prn-hub03.TheFacebook.com (2620:10d:c081:35::127) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.1.1531.3; Mon, 29 Oct 2018 16:06:44 -0700
Received: from PRN-CHUB08.TheFacebook.com (2620:10d:c081:35::17) by prn-hub03.TheFacebook.com (2620:10d:c081:35::127) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA) id 15.1.1531.3 via Frontend Transport; Mon, 29 Oct 2018 16:06:44 -0700
Received: from NAM04-SN1-obe.outbound.protection.outlook.com (192.168.54.28) by o365-in.thefacebook.com (192.168.16.18) with Microsoft SMTP Server (TLS) id 14.3.361.1; Mon, 29 Oct 2018 16:06:44 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kFDixJGpPF9KKFTKIm8L6dnA3fEKxa5LxZym/tOeqIw=; b=b0QxPXUNHwiDBbK1W7UmCXRmUenpOiF0lx4Hax68elZJXwHyOnO4s5VS0uV1Mb5Gf90JlKzPduFvHvfwGkrK8hukV8YQhOoweZlxmtK4UKZezIrrfRaGQsZdyhypxj/JfCC/JpYoK5MlpDHzItnXoTgbj31BgBgk+OFUSm/FIiE=
Received: from BYAPR15MB2312.namprd15.prod.outlook.com (52.135.197.146) by BYAPR15MB2471.namprd15.prod.outlook.com (52.135.200.25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1273.26; Mon, 29 Oct 2018 23:06:42 +0000
Received: from BYAPR15MB2312.namprd15.prod.outlook.com ([fe80::11c0:d70e:972b:7d51]) by BYAPR15MB2312.namprd15.prod.outlook.com ([fe80::11c0:d70e:972b:7d51%3]) with mapi id 15.20.1273.027; Mon, 29 Oct 2018 23:06:42 +0000
From: Roberto Peon <fenix@fb.com>
To: Martin Thomson <martin.thomson@gmail.com>, Tommy Pauly <tpauly@apple.com>
CC: Jana Iyengar <jri.ietf@gmail.com>, Ian Swett <ianswett@google.com>, QUIC WG <quic@ietf.org>
Subject: Re: flow control and DATAGRAM
Thread-Topic: flow control and DATAGRAM
Thread-Index: AQHUb0aT1DuXDxXj8U2a4gtuRLXs9aU2KFYAgAAvQoCAAEeKAIAABpeAgAAvvwD//469gA==
Date: Mon, 29 Oct 2018 23:06:42 +0000
Message-ID: <71FCFE79-D9FA-4240-B215-E19695C8BA1F@fb.com>
References: <CABkgnnU26aYD=TybuD0FZGYtfEa6np-Sk3Jo6t7LRp0wzKh3Lg@mail.gmail.com> <CAKcm_gMDOyJC0AwOo2knN6AMxVbySjGsrLvjcC9A8UA9xcvcWw@mail.gmail.com> <19D3595B-8845-45B6-A60D-9E934DD49FAC@apple.com> <CACpbDcfk+GXb3aL5LG0wQM87thRGO5Y4Q+9cbXf5YuW1=jWCig@mail.gmail.com> <B7BE2454-2A4D-4323-B0A5-0D73BD4B819F@apple.com> <CABkgnnVsgNU8dm08-UGQZqyMP49KtXyV5bSDrWAto2t3x9VOcw@mail.gmail.com>
In-Reply-To: <CABkgnnVsgNU8dm08-UGQZqyMP49KtXyV5bSDrWAto2t3x9VOcw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
user-agent: Microsoft-MacOutlook/10.12.0.181014
x-originating-ip: [2620:10d:c090:200::6:e9de]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; BYAPR15MB2471; 20:cmbOahFd2f7rXMPxgVAaXM5MbVQSzR9rX35P7MYvxe9V0YyRYoqFvgHF48ICc400uZutHiZY9NDu07LGRyereux3Xny/nRtrwSEJGSfctvfQmeEdfIOPlUoL7hpddl8nC6GppHVIYRyy0qP8sIeArJJSit1LRibuHo3njCiIUo0=
x-ms-exchange-antispam-srfa-diagnostics: SOS;
x-ms-office365-filtering-correlation-id: 51d8e136-703b-43bb-9f68-08d63df330ea
x-microsoft-antispam: BCL:0; PCL:0; RULEID:(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(2017052603328)(7153060)(7193020); SRVR:BYAPR15MB2471;
x-ms-traffictypediagnostic: BYAPR15MB2471:
x-microsoft-antispam-prvs: <BYAPR15MB24713E6FA3033460D175C400CDF30@BYAPR15MB2471.namprd15.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:(85827821059158)(31960201722614)(80524489315369);
x-ms-exchange-senderadcheck: 1
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(3002001)(3231382)(11241501184)(944501410)(52105095)(148016)(149066)(150057)(6041310)(20161123564045)(20161123560045)(20161123562045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699051)(76991095); SRVR:BYAPR15MB2471; BCL:0; PCL:0; RULEID:; SRVR:BYAPR15MB2471;
x-forefront-prvs: 084080FC15
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(136003)(366004)(396003)(39860400002)(376002)(346002)(189003)(199004)(7736002)(53936002)(6512007)(81156014)(5250100002)(8676002)(81166006)(6246003)(6116002)(305945005)(54906003)(68736007)(4326008)(229853002)(25786009)(256004)(2906002)(97736004)(110136005)(14444005)(8936002)(551984002)(3480700004)(5660300001)(6436002)(76176011)(33656002)(14454004)(446003)(105586002)(36756003)(86362001)(106356001)(99286004)(6506007)(476003)(53546011)(11346002)(102836004)(316002)(71200400001)(2900100001)(83716004)(39060400002)(46003)(186003)(71190400001)(58126008)(93886005)(478600001)(82746002)(2616005)(6486002)(486006)(42262002); DIR:OUT; SFP:1102; SCL:1; SRVR:BYAPR15MB2471; H:BYAPR15MB2312.namprd15.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1;
received-spf: None (protection.outlook.com: fb.com does not designate permitted sender hosts)
x-microsoft-antispam-message-info: bzhCfsQYhrMzZgKK/bQuxHcJTqr+dYJe+NSDXx+Zb8p6snF55KuQ2hU4zWqSO0aXE6tMDb/KxzqN1iiodto9lb+pxfnjgSUCaohlgh4+1vdyoagQ31KD4YwwUq3qf0IK3WcZXJ/LQjjetz4I7BKB6fbDgQjl1ZLVmEHktsyPpqK5a+NrXP9FX5MMySAFQl0Y6SJ6NgKRNO4mPqCTXc/LJVAOGkF6uy2b/yJAt282z+BI2cIfAvhA+RzcGEbFU9DN8SGn1TSftTr/mU/v6NvyIMX9m9iIi7PnO1FAv9d+PdGStnHOFPn4H612bY1RH2Hh/CvEH15gv75AcK9GiL1LKPjlqyxKlwab5laqfhdURqw=
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="utf-8"
Content-ID: <97ED8E478A6D184383C34C407BA65C44@namprd15.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 51d8e136-703b-43bb-9f68-08d63df330ea
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Oct 2018 23:06:42.5283 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR15MB2471
X-OriginatorOrg: fb.com
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-10-29_13:, , signatures=0
X-Proofpoint-Spam-Reason: safe
X-FB-Internal: Safe
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/vQXEd1apfb5oOTVf7zz19wFxanY>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 29 Oct 2018 23:06:53 -0000

Such per-frame tracking could be a single bit if the protocol using DATAGRAM is willing to do windowed updates (i.e. 1 in flight at once) of such things.
-=R

On 10/29/18, 3:52 PM, "QUIC on behalf of Martin Thomson" <quic-bounces@ietf.org on behalf of martin.thomson@gmail.com> wrote:

    On Tue, Oct 30, 2018 at 7:01 AM Tommy Pauly <tpauly@apple.com> wrote:
    > I'm not suggesting any changes to when the packet gets ack'ed—specifically, I was responding to Martin's hypothetical of having an ACK to a DATAGRAM frame meaning that the application had processed the frame. My impression is that the ACK to a packet containing a DATAGRAM frame means:
    > - The packet made it across the network to the QUIC endpoint on the other side
    > - The QUIC implementation will deliver the DATAGRAM frame to application (it won't drop it locally)
    
    That works.  If you allow for the possibility that an application that
    is unable to process the DATAGRAM will be indistinguishable from QUIC
    dropping it.  That is, if the application can't take the upcall
    because it is too busy, then QUIC might not bother waiting.
    
    > Having a separate outstanding data limit for the DATAGRAM "stream" is an interesting solution to the space. It would then have the nice property of not looking like traditional flow control. It could even be measured in number of frames, rather than bytes (depending on what the limiting factors are).
    
    This is something I alluded to in my email.  I don't think that it
    improves things much, other than perhaps insulating streams from the
    excesses of DATAGRAM frames and vice versa.
    
    If you have a limit, then you need to ensure that both endpoints agree
    on its value eventually.  With DATAGRAM, you might think that it's
    possible to add a new frame that expresses how much data was sent
    (that is the sum of sizes of DATAGRAM frames).  That would run afoul
    of reordering problems, where the limit is exceeded when delayed
    DATAGRAM frames that are already counted, arrive after the accounting
    frame.
    
    You could limit the count to just lost DATAGRAM frames.  That would
    allow a receiver to repair its view of how much was used.  However
    that runs afoul of two problems: acknowledgments aren't reliable, and
    packets can be declared lost and still delivered.  In both cases, the
    sender can believe that frames didn't arrive and so increase their
    report of "lost bytes" too much.  The receiver ends up with an
    inflated view of what was consumed (as opposed to the value being
    short).
    
    If you wanted to solve this problem, you would probably need to have
    per-frame tracking, which makes this a lot less lightweight than I
    think we want.