Re: [Moq] [External] New Version Notification for draft-chen-quic-quicu-01.txt

[Yongyi Yu <[e-mail]>]

Hi Matt,

Thanks for your replying. It's a great idea to send messages through
streams and control their lifetime by cancelling streams, which leverages
existing capability of QUIC. But, if I understand correctly, this mechanism
may not prevent data out-of-order without additional information, or may
face head-of-line blocking if receiving data according stream ID. Also,
since streams in QUIC are independent of each other, additional information
may also be needed to achieve session multiplexing within single QUIC
connection.

Indeed, we want to achieve low latency transmission base on QUIC with
"partial reliability" and that's the reason we make this proposal. We
believe this mechanism could be compatiable with HTTP protocol, but it may
not only be used with HTTP protocol. In a secnario that the messages to be
sent have clear boundaries and need to be delivered in-order, like media
transmission, this proposal should work well. And, since the "partial
reliability" is within a QUIC stream, it can still benefit from the stream
multiplexing feature of QUIC. Would it be helpful if we clarify the details
about the usecase of our proposal in media transmission scenarios in the
draft?

Please do not hesitate to point out if I make any mistakes, and please do
let me know if you have any further advice.

Thanks,
Yongyi.

From: "Matt Joras"<[e-mail]>
Date: Tue, Sep 27, 2022, 01:31
Subject: Re: [External] New Version Notification for
draft-chen-quic-quicu-01.txt
To: "Shihang(Vincent)"<[e-mail]>
Cc: "Lubashev, Igor"<ilubashe=[e-mail]>, "Yongyi Yu"<
[e-mail]>, "Tommy Pauly"<[e-mail]>, "Ryan
Hamilton"<rch=[e-mail]>, "[e-mail]"<[e-mail]>,
"陈鉴平"<[e-mail]>, "景君羡"<[e-mail]>,
"刘天一"<[e-mail]>, "[e-mail]"<
[e-mail]>, <[e-mail]>
+ moq mailing list to this discussion, since I think this thread is highly
relevant to their work as well.

On Mon, Sep 26, 2022 at 12:01 AM Shihang(Vincent) <[e-mail]>
wrote:

> The draft implements partial reliability by splitting one stream into
> multiple blocks and drop stale blocks till a certain offset. In DTP[1], we
> take another approach by mapping the block to stream 1 to 1. In this way
> dropping a block can leverage existing QUIC stream management mechanism
> such as RESET_STREAM, similar to RUSH protocol[2]. I wonder the reason
> behind your design choice. Why mapping multiple blocks into one stream?
>
>
>
> [1] https://datatracker.ietf.org/doc/draft-shi-quic-dtp/
>
> [2] https://datatracker.ietf.org/doc/draft-kpugin-rush/01/
>
>
>
> Best Regards,
>
> Hang Shi
>
>
>
> *From:* QUIC <[e-mail]> *On Behalf Of *Matt Joras
> *Sent:* Monday, September 26, 2022 1:29 PM
> *To:* Lubashev, Igor <ilubashe=[e-mail]>
> *Cc:* Yongyi Yu <[e-mail]>; Tommy Pauly <
> [e-mail]>; Ryan Hamilton <rch=[e-mail]>;
> [e-mail]; 陈鉴平 <[e-mail]>; 景君羡 <
> [e-mail]>; 刘天一 <[e-mail]>
> *Subject:* Re: [External] New Version Notification for
> draft-chen-quic-quicu-01.txt
>
>
>
> First off, chair hat off.
>
> Thanks for taking the time to write up this draft Yongyi et al. I agree
> with others that the more common language we've been using for a scheme
> like this is "partial reliability" as applied to QUIC streams. And indeed,
> as Igor noted, exploring this area is not new for the working group. If you
> had asked me a few years ago, I would have agreed that partial reliability
> within a stream, (i.e. the ability to ensure delivery of parts of a stream)
> was a pressing problem and extremely useful semantic to have in QUIC.
> However my views on the matter have evolved in the intervening years. The
> original driving problem for this capability, and indeed I suspect the
> continued interest, is "low latency" HTTP-based video playback.
>
>
>
> Low latency delivery of data and full reliability of delivery are, in any
> scheme requiring occasional retransmission of data, at odds. However, video
> and audio encodings can playback acceptably for humans using "partial"
> data, and so a integrated video player could potentially deliver an
> acceptable experience at lower latency if it chooses not to wait for the
> entirety of the data. This naturally leads one to exploring a solution
> involving partial delivery, hence "partial reliability". However, the
> existing methods of serving video on the Internet largely consists of
> HTTP-based CDN infrastructure, which for video pretty much always involves
> serving segments of encoded video in HTTP responses, which are then
> sequentially fed into a player. HTTP today always maps to an underlying
> notion of a reliable sequential byte stream. Thus, if we are to reuse the
> existing HTTP CDN infrastructure, it follows that we must make it so that a
> byte stream is partially reliable and figure out a way to map such concepts
> to HTTP.
>
>
>
> We (Meta, formerly Facebook) explored this notion for real ultra low
> latency video playback, implementing what amounted to a variant of the
> drafts Igor linked, as well as the accompanying components at the HTTP
> layer. This effort though ultimately has not been used in production for
> video playback for various reasons, some of which relate to the complexity
> of the resulting systems. What has seen production success for a subset of
> low latency video, and has emerging promising low-latency
> not-strictly-video Internet usecases, is a rather different scheme that I
> think bears repeating.
>
>
>
> While QUIC lacks a natural semantic for partial reliability within a
> stream, streams themselves offer the aspiring application the ability to a
> achieve independent data segmentation of arbitrary size. This fundamentally
> amounts to treating a QUIC stream as a sort of "message" semantic rather
> than the more familiar mapping of a "socket"ish byte stream. An application
> can leverage this to cancel arbitrary segments of data and thus effectively
> achieve partial reliability. This is the core of the idea we have
> successfully leveraged at Meta (and was indeed our first shipped usecase of
> QUIC on the Internet) for video ingest via the RUSH protocol[1]. Concepts
> similar to this have also been explored similarly in the Warp protocol[2].
> These are somewhat in contrast to what has been proposed by the design of
> the QuicR protocol, though it also does not attempt to use partial
> reliability within a QUIC stream either[3]. All three of which, and this
> space in general, are going to be concepts discussed at end by the
> newly-formed MOQ working group[4].
>
> All of this is to say, I don't think that the value of reusing HTTP
> infrastructure has necessarily changed for HTTP playback, but it does seem
> as though there are greater challenges in achieving that than some may
> initially have thought.
>
>
>
> Best,
>
> Matt Joras
>
>
>
> [1] https://datatracker.ietf.org/doc/draft-kpugin-rush/01/
>
> [2] https://datatracker.ietf.org/doc/draft-lcurley-warp/
> [3] https://datatracker.ietf.org/doc/draft-jennings-moq-quicr-proto/
> [4] https://datatracker.ietf.org/wg/moq/about/
>
>
>
> On Sun, Sep 25, 2022 at 9:29 AM Lubashev, Igor <ilubashe=
> [e-mail]> wrote:
>
> Are you trying to implement something like
> https://datatracker.ietf.org/doc/html/draft-lubashev-quic-partial-reliability-02
> or
> https://datatracker.ietf.org/doc/html/draft-lubashev-quic-partial-reliability-03?
> (The -02 and -3 versions have somewhat different properties, and some
> people prefer -02 version.)
>
>
>
>    - Igor
>
>
>
> *From:* Yongyi Yu <[e-mail]>
> *Sent:* Friday, September 23, 2022 1:11 AM
> *To:* Tommy Pauly <[e-mail]>
> *Cc:* Ryan Hamilton <rch=[e-mail]>; [e-mail]; 陈鉴平
> <[e-mail]>; 景君羡 <[e-mail]>; 刘天一
> <[e-mail]>
> *Subject:* Re: [External] New Version Notification for
> draft-chen-quic-quicu-01.txt
>
>
>
> Thanks for your replying. I agree "partial reliability" fits our draft
> better, since we do retransmit lost data and provide in-order transmission.
> And, our proposal could be compatible with H3, should we clarify the
> behavior in the draft? Please do let me know if you have any further
> advice.
>
>
>
> Thanks,
>
> Yongyi.
>
> From: "Tommy Pauly"< <[e-mail]>[e-mail]>
>
> Date: Thu, Sep 22, 2022, 11:32
>
> Subject: Re: [External] New Version Notification for
> draft-chen-quic-quicu-01.txt
>
> To: "Ryan Hamilton"< <rch=[e-mail]>
> rch=[e-mail]>
>
> Cc: "于涌溢"< <[e-mail]>[e-mail]>, "
> [e-mail]"< <[e-mail]>[e-mail]>, "陈鉴平"<
> <[e-mail]>[e-mail]>, "
> 景君羡"< <[e-mail]>[e-mail]>, "刘天一"<
> <[e-mail]>[e-mail]>
>
> I noticed that this draft does reference RFC 9221, and DATAGRAM frames.
>
>
>
> If I’m understanding correctly, it seems like this is trying to define an
> approach for what’s more commonly referred to as “partial reliability”,
> which was something DATAGRAM explicitly didn’t try to include.
>
>
>
> If this is the case, I think it would be useful to reframe the document in
> terms of that, since the heavy use of “unreliable” is quite confusing. It
> also isn’t clear to me how this proposal would work with specific
> application protocols on top of QUIC — is this something that’s meant to be
> compatible with H3, or is it for entirely separate use cases?
>
>
>
> Best,
>
> Tommy
>
>
>
> On Sep 21, 2022, at 3:17 PM, Ryan Hamilton <
> rch=[e-mail]> wrote:
>
>
>
> QUIC has support for unreliable data via the DATAGRAM frame, RFC 9221
> <https://urldefense.com/v3/__https:/www.rfc-editor.org/info/rfc9221__;!!GjvTz_vk!VHC6upHgCOojEiYhln21JerQBPYz1cbPLifo7gevl5sFTM-5qEoTkN-TxpvuKG1MQfnSBiZsXrrQ2OC2qjWu1aws$>.
> It seems that this new proposal attempts to add unreliable data support not
> to QUIC, but to QUIC *streams*. QUIC streams, of course, offer a reliable,
> in-order, sequence of bytes. Did you consider starting with DATAGRAMs, and
> building from there instead?
>
>
>
> Cheers,
>
>
> Ryan
>
>
>
> On Wed, Sep 21, 2022 at 1:14 AM 于涌溢 <[e-mail]> wrote:
>
> Deal all,
>
>
>
> We would like to introduce an extension of the QUIC protocol for
> unreliable data transmission called QUIC Unreliable (QUICU) . The following
> draft is submitted for your consideration and comments. We would also like
> to present this at IETF 115 to discuss further.
>
>
>
> To summarize on this draft: it describes an extension of the QUIC protocol
> for unreliable data transmission called QUIC Unreliable (QUICU), which
> mainly through the definition and use of three new types of frames, namely
> the Expire Offset Frame, Boundary Frame, and Correlation Frame. The main
> purpose of this extension is to reduce data delivery delay. Through
> controlling the timing and scope of packet losses, QUICU reduces useless
> transmission to improve transmission efficiency, reduces head-of-line
> blocking to improve transmission timeliness, which are beneficial to
> delay-sensitive applications, especially audio and video applications. This
> document describes the motivation, the operations, and the wire formats of
> the three new types of frames.
>
>
>
> Link to html:
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-chen-quic-quicu-01__;!!GjvTz_vk!VHC6upHgCOojEiYhln21JerQBPYz1cbPLifo7gevl5sFTM-5qEoTkN-TxpvuKG1MQfnSBiZsXrrQ2OC2qol99skU$>
> https://datatracker.ietf.org/doc/html/draft-chen-quic-quicu-01
>
>
>
> Please feel free to reach us for any comments or questions on this.
>
>
>
> Thanks,
>
> Yongyi.
>
>
>
> ---------- Forwarded message ---------
>
> From: <[e-mail]>[e-mail]
>
> Date: Mon, Sep 19, 2022, 20:54
>
> Subject: [External] New Version Notification for
> draft-chen-quic-quicu-01.txt
>
> To: "Jianping Chen"< <[e-mail]>
> [e-mail]>, "Junxian Jing"<
> <[e-mail]>[e-mail]>, "Tianyi Liu"<
> <[e-mail]>[e-mail]>, "Yongyi Yu"<
> <[e-mail]>[e-mail]>
>
> A new version of I-D, draft-chen-quic-quicu-01.txt
>
> has been successfully submitted by Yongyi Yu and posted to the
>
> IETF repository.
>
>
>
> Name:                draft-chen-quic-quicu
>
> Revision:        01
>
> Title:                An Unreliable Extension to QUIC
>
> Document date:        2022-09-19
>
> Group:                Individual Submission
>
> Pages:                10
>
> URL:
> <https://urldefense.com/v3/__https:/www.ietf.org/archive/id/draft-chen-quic-quicu-01.txt__;!!GjvTz_vk!VHC6upHgCOojEiYhln21JerQBPYz1cbPLifo7gevl5sFTM-5qEoTkN-TxpvuKG1MQfnSBiZsXrrQ2OC2qgIVZwR1$>
> https://www.ietf.org/archive/id/draft-chen-quic-quicu-01.txt
>
> Status:
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-chen-quic-quicu/__;!!GjvTz_vk!VHC6upHgCOojEiYhln21JerQBPYz1cbPLifo7gevl5sFTM-5qEoTkN-TxpvuKG1MQfnSBiZsXrrQ2OC2qqEw9hmD$>
> https://datatracker.ietf.org/doc/draft-chen-quic-quicu/
>
> Htmlized:
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-chen-quic-quicu__;!!GjvTz_vk!VHC6upHgCOojEiYhln21JerQBPYz1cbPLifo7gevl5sFTM-5qEoTkN-TxpvuKG1MQfnSBiZsXrrQ2OC2qk9rs9H6$>
> https://datatracker.ietf.org/doc/html/draft-chen-quic-quicu
>
> Diff:
> <https://urldefense.com/v3/__https:/www.ietf.org/rfcdiff?url2=draft-chen-quic-quicu-01__;!!GjvTz_vk!VHC6upHgCOojEiYhln21JerQBPYz1cbPLifo7gevl5sFTM-5qEoTkN-TxpvuKG1MQfnSBiZsXrrQ2OC2qv4nuLVS$>
> https://www.ietf.org/rfcdiff?url2=draft-chen-quic-quicu-01
>
>
>
> Abstract:
>
>   QUIC Unreliable (QUICU) is an extension of the QUIC protocol for
>
>   unreliable data transmission, mainly through the definition and use
>
>   of three new types of frames, namely the Expire Offset Frame,
>
>   Boundary Frame, and Correlation Frame. The main purpose of this
>
>   extension is to reduce data delivery delay. Through controlling the
>
>   timing and scope of packet losses, QUICU reduces useless transmission
>
>   to improve transmission efficiency, reduces head-of-line blocking to
>
>   improve transmission timeliness, which are beneficial to delay-
>
>   sensitive applications, especially audio and video applications.
>
>
>
>   This document describes the motivation, the operations, and the wire
>
>   formats of the three new types of frames.
>
>
>
>
>
>
>
>
>
> The IETF Secretariat
>
>
>
>