Re: RE: [External] New Version Notification for draft-chen-quic-quicu-01.txt
Yongyi Yu <yuyongyi.yyy@bytedance.com> Tue, 11 October 2022 08:37 UTC
Return-Path: <yuyongyi.yyy@bytedance.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 53DDAC159498 for <quic@ietfa.amsl.com>; Tue, 11 Oct 2022 01:37:38 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.795
X-Spam-Level:
X-Spam-Status: No, score=-1.795 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_FONT_LOW_CONTRAST=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, TRACKER_ID=0.1, T_KAM_HTML_FONT_INVALID=0.01, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=bytedance-com.20210112.gappssmtp.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id z3bNKxIeT7ER for <quic@ietfa.amsl.com>; Tue, 11 Oct 2022 01:37:34 -0700 (PDT)
Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 89202C157B37 for <quic@ietf.org>; Tue, 11 Oct 2022 01:37:34 -0700 (PDT)
Received: by mail-pl1-x632.google.com with SMTP id 10so12647141pli.0 for <quic@ietf.org>; Tue, 11 Oct 2022 01:37:34 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=cc:to:subject:message-id:date:in-reply-to:from:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=RilaH+nDPdJTVv04xEwlIWLZMDRD8FWvkX0i/f7Sxt4=; b=cr4B2rq9q7F0pna2i2Pm1AhJRbX8LBAivOn7vkcGtQs1XRnwRpI06hUNpf40UvrQbH wVmq2TLBHbyNzCoLtiB5iXHRbn43L0hHcbJUK46I5G2xXuDeVJA2oHIoGP6BG84MsCXH EWGYG9HHfjWdW0YnKXtktu1ENxQAzn+4rsCHjd1ARLD6GMWykyHCcF3L4OhaqkxMAP0L TZ8V4bx3eGo39xcZ8jqQUzWro1RFjqlar4RtxZ3T+qvwPlmTbGz7h8t4ukpHphX7dRJW VntH00Ni2mvG2BkueiUL426LAprmopE55otAcG/cEUVnrq+xaIRkr0uuSBF+XvTGZIN0 xPYA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:in-reply-to:from:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RilaH+nDPdJTVv04xEwlIWLZMDRD8FWvkX0i/f7Sxt4=; b=iYhQsUg9L/n++7qp9t1z/1SJgbEGzgXZmyAq7eGnQxg3NtnuxGlD2H1e5yTbz+vPr8 NN2b+DaUs7ALiRlL3IjWEZE/44UNBZ8ch/17fckdP17qHw8GvbZV7Q5ZkWnmKLX/3ZS2 xplkeLyZzb1zXlUXTdFnbVm3taj98w8KAQ5Ey6PWFfxKZ+O/crvx8qsfEbsKlLuX1RgZ wEHcr3nAGaCN7vFWUNUDhDq71Y2bsKAmlgeks9te9LqmL342XmKKXnQla09r0BVNgend JLQq8Za7qIq3xJ7I6tBbYLgdbFAAHWhyAQJnntLEYo+sxVA9UN7ZB/95L9Cf1A2eKHBq 1Iww==
X-Gm-Message-State: ACrzQf2W2cjZCTBVDUOJaayRjxBmdEkoWkIbhTmSG1+gAa01l5PHYmg8 6/P8jk7d9e0DFZ4hNbhdgMgfdFotMF9x3Jd+cdTCBg==
X-Google-Smtp-Source: AMsMyM4ZPvFqU81pkOE3G7FcnGtWG3HBhOxBkYqtenpF6ChUBi3knFGWYMU2P2usbVc3xYxXLzFQJeQiPuoelvrQkMw=
X-Received: by 2002:a17:90a:8044:b0:20a:6412:3b8c with SMTP id e4-20020a17090a804400b0020a64123b8cmr25454006pjw.139.1665477453307; Tue, 11 Oct 2022 01:37:33 -0700 (PDT)
Received: from 349319672217 named unknown by gmailapi.google.com with HTTPREST; Tue, 11 Oct 2022 01:37:32 -0700
Mime-Version: 1.0
References: <166359205152.6422.10161941398731313578@ietfa.amsl.com> <CAGAnozYePCwo5S-EUTN4X=-EixNFVgF+1MwiD60-GDogw7ojnA@mail.gmail.com> <CAJ_4DfS8n5sZXEk0=Eojq_DEK6QGvgG6-KjOiVnHv8etxzCSkQ@mail.gmail.com> <AE5493E8-DFAD-42F2-A41C-4B924B8602EF@apple.com> <CAGAnozZHFChVbK2MNkxVm5XbF3msGyHuRy1er7OjAGPzE_EMLQ@mail.gmail.com> <d0f85f1306c94e8d92ad4ad706e9f48f@akamai.com> <CAGAnozaGaYNPFFcv3SXM9wqWJ+NWUh_+bTKFcp_D0_fuQUd0Aw@mail.gmail.com> <5f9517bfd0a34301b33a80c2dc90a984@akamai.com> <MW5PR15MB5145872722F2482A298767C2D4559@MW5PR15MB5145.namprd15.prod.outlook.com>
From: Yongyi Yu <yuyongyi.yyy@bytedance.com>
In-Reply-To: <MW5PR15MB5145872722F2482A298767C2D4559@MW5PR15MB5145.namprd15.prod.outlook.com>
Date: Tue, 11 Oct 2022 01:37:32 -0700
Message-ID: <CAGAnozanMfhtzxz0PAPH6hu_7hAHTwfQFSzhi7RcZ8aCo2bUrA@mail.gmail.com>
Subject: Re: RE: [External] New Version Notification for draft-chen-quic-quicu-01.txt
To: Roberto Peon <fenix@meta.com>
Cc: "Lubashev, Igor" <ilubashe=40akamai.com@dmarc.ietf.org>, Tommy Pauly <tpauly@apple.com>, Ryan Hamilton <rch=40google.com@dmarc.ietf.org>, "quic@ietf.org" <quic@ietf.org>, 陈鉴平 <chenjianping.ireader@bytedance.com>, 景君羡 <jingjunxian@bytedance.com>, 刘天一 <liutianyi.lty@bytedance.com>
Content-Type: multipart/alternative; boundary="000000000000fc7a7f05eabe2f94"
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/iMpi57yOTAHbH57cWR4RIl__5r0>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 11 Oct 2022 08:37:38 -0000
Hi Roberto, I agree that there are many flaws about flow-control in our draft, which would cause underutilizing the bandwidth or other issues, as you and Igor have mentioned. We really miss out those cases and will try to fix them first. Also, could you please clarify the use case about "atoms" you mentioned? We are eager to see what we can do in that scenario. Thanks, Yongyi. From: "Roberto Peon"<fenix@meta.com> Date: Fri, Sep 30, 2022, 23:26 Subject: Re: RE: [External] New Version Notification for draft-chen-quic-quicu-01.txt To: "Lubashev, Igor"<ilubashe=40akamai.com@dmarc.ietf.org>, "Yongyi Yu"< yuyongyi.yyy@bytedance.com> Cc: "Tommy Pauly"<tpauly@apple.com>, "Ryan Hamilton"< rch=40google.com@dmarc.ietf.org>, "quic@ietf.org"<quic@ietf.org>, "陈鉴平"< chenjianping.ireader@bytedance.com>, "景君羡"<jingjunxian@bytedance.com>, "刘天一"<liutianyi.lty@bytedance.com> W.r.t. past implementation of PR stuff: there was a partial implementation of partial reliability which didn’t get used because there were other pressing problems (like getting H3 working/standardized), and there was a separate implementation for video which predated H3 (and many things QUIC). I’d like to echo Igor’s suggestion to focus on flow-control. This is, without a doubt, the hardest part. This gets interesting when you think about connection-level flow control vs stream flow control, and ensuring that you can reasonably use the channel bandwidth. An approach where you don’t change flow control at all (and instead fake acks or similar) can work, but may end up with a bunch of operational complexity as you debug why the bandwidth isn’t being used like you think it should be. (Thankfully, this won’t be worse than just using the reliable stuff). I do like the breakdown of a stream into data-blocks. In prior conversations near the dawn of the quic working-group, we (or maybe I) called these “atoms”. At the time, I was imagining that this would be done with two, associated, streams. One which has the data, and another which points to the data stream, and has fixed-size records which describe atom (data-block) boundaries. Another way to think about this is that you de-mux the video container into a metadata-stream, and media streams which have “raw bitstream” payloads. In my explorations, it was interesting to see that this actually reduced the on-the-wire size for the video while allowing for re-sync. I’ll be following this with great interest! -=R *From: *QUIC <quic-bounces@ietf.org> on behalf of Lubashev, Igor < ilubashe=40akamai.com@dmarc.ietf.org> *Date: *Tuesday, September 27, 2022 at 9:13 AM *To: *Yongyi Yu <yuyongyi.yyy@bytedance.com> *Cc: *Tommy Pauly <tpauly@apple.com>, Ryan Hamilton < rch=40google.com@dmarc.ietf.org>, quic@ietf.org <quic@ietf.org>, 陈鉴平 < chenjianping.ireader@bytedance.com>, 景君羡 <jingjunxian@bytedance.com>, 刘天一 < liutianyi.lty@bytedance.com> *Subject: *RE: RE: [External] New Version Notification for draft-chen-quic-quicu-01.txt Yongyi, Without going into the question of whether partially reliable streams are necessary (as Matt said, Meta implemented them but is not using that implementation in production), here are a few comments I have. ZjQcmQRYFpfptBannerStart *This Message Is From an External Sender * ZjQcmQRYFpfptBannerEnd Yongyi, Without going into the question of whether partially reliable streams are necessary (as Matt said, Meta implemented them but is not using that implementation in production), here are a few comments I have. 1. Have you thought about the interaction of you draft with stream and connection flow control? 95% of the time I spent on my design was spent on thinking of ways to augment flow control to avoid extra round trips just to open a flow control window, while still ensuring that the resulting protocol is resilient against an adversarial endpoint forcing the peer to commit an unreasonable amount of resources (memory) to the connection. - 1. Our proposal splits a QUIC stream into many data blocks by Boundary Frame. The receiver would provide data to the application layer only when a data block is completely received. Therefore, the application layer needn't be aware of the gap at the expiration point or discard data already read. - 2. Because the stream is split into blocks, the sender could send or expire data by data blocks. Then the transport layer could track the message boundaries instead of the application layer. 1. Section 4.4* Data Receiving* of your draft is saying the same. It seems wrong to have the transport protocol specify the behavior of the QUIC library APIs so precisely. It is ok to try to enable a particular library API, but the transport protocol’s specification ought to limit itself to the semantics of bytes on the wire. Very best, - Igor *From:* Yongyi Yu <yuyongyi.yyy@bytedance.com> *Sent:* Tuesday, September 27, 2022 11:04 AM Hi Igor, Thanks for your replying. Our proposal indeed shares similar ideas with yours, but there are also some difference. We'd like to update our draft to clarify the difference in detail, and to briefly summary here, there are three major points. 1. Our proposal splits a QUIC stream into many data blocks by Boundary Frame. The receiver would provide data to the application layer only when a data block is completely received. Therefore, the application layer needn't be aware of the gap at the expiration point or discard data already read. 2. Because the stream is split into blocks, the sender could send or expire data by data blocks. Then the transport layer could track the message boundaries instead of the application layer. 3. With the introduction of Correlation Frame, we can combine multiple QUIC streams into single data stream while avoiding head of line blocking. This feature could be quite useful in many scenarios, e.g., transmitting both video and audio data through single HTTP response. Please do let me know if you have any further advice, and please do not hesitate to point out if I make any mistakes. Thanks, Yongyi. From: "Lubashev, Igor"< <ilubashe@akamai.com>ilubashe@akamai.com> Date: Mon, Sep 26, 2022, 00:29 Subject: RE: [External] New Version Notification for draft-chen-quic-quicu-01.txt To: "Yongyi Yu"< <yuyongyi.yyy@bytedance.com>yuyongyi.yyy@bytedance.com>, "Tommy Pauly"< <tpauly@apple.com>tpauly@apple.com> Cc: "Ryan Hamilton"< <rch=40google.com@dmarc.ietf.org> rch=40google.com@dmarc.ietf.org>, " <quic@ietf.org>quic@ietf.org"< <quic@ietf.org>quic@ietf.org>, "陈鉴平"< <chenjianping.ireader@bytedance.com> chenjianping.ireader@bytedance.com>, "景君羡"< <jingjunxian@bytedance.com> jingjunxian@bytedance.com>, "刘天一"< <liutianyi.lty@bytedance.com> liutianyi.lty@bytedance.com> Are you trying to implement something like https://datatracker.ietf.org/doc/html/draft-lubashev-quic-partial-reliability-02 or https://datatracker.ietf.org/doc/html/draft-lubashev-quic-partial-reliability-03? (The -02 and -3 versions have somewhat different properties, and some people prefer -02 version.) - Igor *From:* Yongyi Yu <yuyongyi.yyy@bytedance.com> *Sent:* Friday, September 23, 2022 1:11 AM *To:* Tommy Pauly <tpauly@apple.com> *Cc:* Ryan Hamilton <rch=40google.com@dmarc.ietf.org>; quic@ietf.org; 陈鉴平 < chenjianping.ireader@bytedance.com>; 景君羡 <jingjunxian@bytedance.com>; 刘天一 < liutianyi.lty@bytedance.com> *Subject:* Re: [External] New Version Notification for draft-chen-quic-quicu-01.txt Thanks for your replying. I agree "partial reliability" fits our draft better, since we do retransmit lost data and provide in-order transmission. And, our proposal could be compatible with H3, should we clarify the behavior in the draft? Please do let me know if you have any further advice. Thanks, Yongyi. From: "Tommy Pauly"<tpauly@apple.com> Date: Thu, Sep 22, 2022, 11:32 Subject: Re: [External] New Version Notification for draft-chen-quic-quicu-01.txt To: "Ryan Hamilton"<rch=40google.com@dmarc.ietf.org> Cc: "于涌溢"<yuyongyi.yyy@bytedance.com>, "quic@ietf.org"<quic@ietf.org>, "陈鉴平 "<chenjianping.ireader@bytedance.com>, "景君羡"<jingjunxian@bytedance.com>, " 刘天一"<liutianyi.lty@bytedance.com> I noticed that this draft does reference RFC 9221, and DATAGRAM frames. If I’m understanding correctly, it seems like this is trying to define an approach for what’s more commonly referred to as “partial reliability”, which was something DATAGRAM explicitly didn’t try to include. If this is the case, I think it would be useful to reframe the document in terms of that, since the heavy use of “unreliable” is quite confusing. It also isn’t clear to me how this proposal would work with specific application protocols on top of QUIC — is this something that’s meant to be compatible with H3, or is it for entirely separate use cases? Best, Tommy On Sep 21, 2022, at 3:17 PM, Ryan Hamilton <rch=40google.com@dmarc.ietf.org> wrote: QUIC has support for unreliable data via the DATAGRAM frame, RFC 9221 <https://www.rfc-editor.org/info/rfc9221>. It seems that this new proposal attempts to add unreliable data support not to QUIC, but to QUIC *streams*. QUIC streams, of course, offer a reliable, in-order, sequence of bytes. Did you consider starting with DATAGRAMs, and building from there instead? Cheers, Ryan On Wed, Sep 21, 2022 at 1:14 AM 于涌溢 <yuyongyi.yyy@bytedance.com> wrote: Deal all, We would like to introduce an extension of the QUIC protocol for unreliable data transmission called QUIC Unreliable (QUICU) . The following draft is submitted for your consideration and comments. We would also like to present this at IETF 115 to discuss further. To summarize on this draft: it describes an extension of the QUIC protocol for unreliable data transmission called QUIC Unreliable (QUICU), which mainly through the definition and use of three new types of frames, namely the Expire Offset Frame, Boundary Frame, and Correlation Frame. The main purpose of this extension is to reduce data delivery delay. Through controlling the timing and scope of packet losses, QUICU reduces useless transmission to improve transmission efficiency, reduces head-of-line blocking to improve transmission timeliness, which are beneficial to delay-sensitive applications, especially audio and video applications. This document describes the motivation, the operations, and the wire formats of the three new types of frames. Link to html: https://datatracker.ietf.org/doc/html/draft-chen-quic-quicu-01 Please feel free to reach us for any comments or questions on this. Thanks, Yongyi. ---------- Forwarded message --------- From: <internet-drafts@ietf.org>internet-drafts@ietf.org Date: Mon, Sep 19, 2022, 20:54 Subject: [External] New Version Notification for draft-chen-quic-quicu-01.txt To: "Jianping Chen"< <chenjianping.ireader@bytedance.com> chenjianping.ireader@bytedance.com>, "Junxian Jing"< jingjunxian@bytedance.com>, "Tianyi Liu"<liutianyi.lty@bytedance.com>, "Yongyi Yu"<yuyongyi.yyy@bytedance.com> A new version of I-D, draft-chen-quic-quicu-01.txt has been successfully submitted by Yongyi Yu and posted to the IETF repository. Name: draft-chen-quic-quicu Revision: 01 Title: An Unreliable Extension to QUIC Document date: 2022-09-19 Group: Individual Submission Pages: 10 URL: https://www.ietf.org/archive/id/draft-chen-quic-quicu-01.txt Status: https://datatracker.ietf.org/doc/draft-chen-quic-quicu/ Htmlized: https://datatracker.ietf.org/doc/html/draft-chen-quic-quicu Diff: https://www.ietf.org/rfcdiff?url2=draft-chen-quic-quicu-01 Abstract: QUIC Unreliable (QUICU) is an extension of the QUIC protocol for unreliable data transmission, mainly through the definition and use of three new types of frames, namely the Expire Offset Frame, Boundary Frame, and Correlation Frame. The main purpose of this extension is to reduce data delivery delay. Through controlling the timing and scope of packet losses, QUICU reduces useless transmission to improve transmission efficiency, reduces head-of-line blocking to improve transmission timeliness, which are beneficial to delay- sensitive applications, especially audio and video applications. This document describes the motivation, the operations, and the wire formats of the three new types of frames. The IETF Secretariat
- Fw: [External] New Version Notification for draft… 于涌溢
- Re: Fw: [External] New Version Notification for d… Ryan Hamilton
- Re: [External] New Version Notification for draft… Tommy Pauly
- Re: [External] New Version Notification for draft… Yongyi Yu
- RE: [External] New Version Notification for draft… Lubashev, Igor
- Re: [External] New Version Notification for draft… Matt Joras
- Re: [External] New Version Notification for draft… Matt Joras
- Re: [External] New Version Notification for draft… Yongyi Yu
- Re: RE: [External] New Version Notification for d… Yongyi Yu
- Re: RE: [External] New Version Notification for d… Yongyi Yu
- RE: RE: [External] New Version Notification for d… Lubashev, Igor
- Re: [External] New Version Notification for draft… Konstantin Tsoy
- Re: [External] New Version Notification for draft… Ryan Hamilton
- RE: [External] New Version Notification for draft… Konstantin Tsoy
- Re: [External] New Version Notification for draft… Yongyi Yu
- Re: RE: RE: [External] New Version Notification f… Yongyi Yu
- RE: [External] New Version Notification for draft… Shihang(Vincent)
- Re: RE: [External] New Version Notification for d… Roberto Peon
- RE: RE: RE: [External] New Version Notification f… Lubashev, Igor
- Re: RE: [External] New Version Notification for d… Yongyi Yu
- RE: [External] New Version Notification for draft… Shihang(Vincent)