Re: [External] New Version Notification for draft-chen-quic-quicu-01.txt

[Matt Joras <[e-mail]>]

First off, chair hat off.

Thanks for taking the time to write up this draft Yongyi et al. I agree
with others that the more common language we've been using for a scheme
like this is "partial reliability" as applied to QUIC streams. And indeed,
as Igor noted, exploring this area is not new for the working group. If you
had asked me a few years ago, I would have agreed that partial reliability
within a stream, (i.e. the ability to ensure delivery of parts of a stream)
was a pressing problem and extremely useful semantic to have in QUIC.
However my views on the matter have evolved in the intervening years. The
original driving problem for this capability, and indeed I suspect the
continued interest, is "low latency" HTTP-based video playback.

Low latency delivery of data and full reliability of delivery are, in any
scheme requiring occasional retransmission of data, at odds. However, video
and audio encodings can playback acceptably for humans using "partial"
data, and so a integrated video player could potentially deliver an
acceptable experience at lower latency if it chooses not to wait for the
entirety of the data. This naturally leads one to exploring a solution
involving partial delivery, hence "partial reliability". However, the
existing methods of serving video on the Internet largely consists of
HTTP-based CDN infrastructure, which for video pretty much always involves
serving segments of encoded video in HTTP responses, which are then
sequentially fed into a player. HTTP today always maps to an underlying
notion of a reliable sequential byte stream. Thus, if we are to reuse the
existing HTTP CDN infrastructure, it follows that we must make it so that a
byte stream is partially reliable and figure out a way to map such concepts
to HTTP.

We (Meta, formerly Facebook) explored this notion for real ultra low
latency video playback, implementing what amounted to a variant of the
drafts Igor linked, as well as the accompanying components at the HTTP
layer. This effort though ultimately has not been used in production for
video playback for various reasons, some of which relate to the complexity
of the resulting systems. What has seen production success for a subset of
low latency video, and has emerging promising low-latency
not-strictly-video Internet usecases, is a rather different scheme that I
think bears repeating.

While QUIC lacks a natural semantic for partial reliability within a
stream, streams themselves offer the aspiring application the ability to a
achieve independent data segmentation of arbitrary size. This fundamentally
amounts to treating a QUIC stream as a sort of "message" semantic rather
than the more familiar mapping of a "socket"ish byte stream. An application
can leverage this to cancel arbitrary segments of data and thus effectively
achieve partial reliability. This is the core of the idea we have
successfully leveraged at Meta (and was indeed our first shipped usecase of
QUIC on the Internet) for video ingest via the RUSH protocol[1]. Concepts
similar to this have also been explored similarly in the Warp protocol[2].
These are somewhat in contrast to what has been proposed by the design of
the QuicR protocol, though it also does not attempt to use partial
reliability within a QUIC stream either[3]. All three of which, and this
space in general, are going to be concepts discussed at end by the
newly-formed MOQ working group[4].

All of this is to say, I don't think that the value of reusing HTTP
infrastructure has necessarily changed for HTTP playback, but it does seem
as though there are greater challenges in achieving that than some may
initially have thought.

Best,
Matt Joras

[1] https://datatracker.ietf.org/doc/draft-kpugin-rush/01/
[2] https://datatracker.ietf.org/doc/draft-lcurley-warp/
[3] https://datatracker.ietf.org/doc/draft-jennings-moq-quicr-proto/
[4] https://datatracker.ietf.org/wg/moq/about/

On Sun, Sep 25, 2022 at 9:29 AM Lubashev, Igor <ilubashe=
[e-mail]> wrote:

> Are you trying to implement something like
> https://datatracker.ietf.org/doc/html/draft-lubashev-quic-partial-reliability-02
> or
> https://datatracker.ietf.org/doc/html/draft-lubashev-quic-partial-reliability-03?
> (The -02 and -3 versions have somewhat different properties, and some
> people prefer -02 version.)
>
>
>
>    - Igor
>
>
>
> *From:* Yongyi Yu <[e-mail]>
> *Sent:* Friday, September 23, 2022 1:11 AM
> *To:* Tommy Pauly <[e-mail]>
> *Cc:* Ryan Hamilton <rch=[e-mail]>; [e-mail]; 陈鉴平
> <[e-mail]>; 景君羡 <[e-mail]>; 刘天一
> <[e-mail]>
> *Subject:* Re: [External] New Version Notification for
> draft-chen-quic-quicu-01.txt
>
>
>
> Thanks for your replying. I agree "partial reliability" fits our draft
> better, since we do retransmit lost data and provide in-order transmission.
> And, our proposal could be compatible with H3, should we clarify the
> behavior in the draft? Please do let me know if you have any further
> advice.
>
>
>
> Thanks,
>
> Yongyi.
>
> From: "Tommy Pauly"<[e-mail]>
>
> Date: Thu, Sep 22, 2022, 11:32
>
> Subject: Re: [External] New Version Notification for
> draft-chen-quic-quicu-01.txt
>
> To: "Ryan Hamilton"<rch=[e-mail]>
>
> Cc: "于涌溢"<[e-mail]>, "[e-mail]"<[e-mail]>, "
> 陈鉴平"<[e-mail]>, "景君羡"<[e-mail]>,
> "刘天一"<[e-mail]>
>
> I noticed that this draft does reference RFC 9221, and DATAGRAM frames.
>
>
>
> If I’m understanding correctly, it seems like this is trying to define an
> approach for what’s more commonly referred to as “partial reliability”,
> which was something DATAGRAM explicitly didn’t try to include.
>
>
>
> If this is the case, I think it would be useful to reframe the document in
> terms of that, since the heavy use of “unreliable” is quite confusing. It
> also isn’t clear to me how this proposal would work with specific
> application protocols on top of QUIC — is this something that’s meant to be
> compatible with H3, or is it for entirely separate use cases?
>
>
>
> Best,
>
> Tommy
>
>
>
> On Sep 21, 2022, at 3:17 PM, Ryan Hamilton <
> rch=[e-mail]> wrote:
>
>
>
> QUIC has support for unreliable data via the DATAGRAM frame, RFC 9221
> <https://urldefense.com/v3/__https:/www.rfc-editor.org/info/rfc9221__;!!GjvTz_vk!VHC6upHgCOojEiYhln21JerQBPYz1cbPLifo7gevl5sFTM-5qEoTkN-TxpvuKG1MQfnSBiZsXrrQ2OC2qjWu1aws$>.
> It seems that this new proposal attempts to add unreliable data support not
> to QUIC, but to QUIC *streams*. QUIC streams, of course, offer a reliable,
> in-order, sequence of bytes. Did you consider starting with DATAGRAMs, and
> building from there instead?
>
>
>
> Cheers,
>
>
> Ryan
>
>
>
> On Wed, Sep 21, 2022 at 1:14 AM 于涌溢 <[e-mail]> wrote:
>
> Deal all,
>
>
>
> We would like to introduce an extension of the QUIC protocol for
> unreliable data transmission called QUIC Unreliable (QUICU) . The following
> draft is submitted for your consideration and comments. We would also like
> to present this at IETF 115 to discuss further.
>
>
>
> To summarize on this draft: it describes an extension of the QUIC protocol
> for unreliable data transmission called QUIC Unreliable (QUICU), which
> mainly through the definition and use of three new types of frames, namely
> the Expire Offset Frame, Boundary Frame, and Correlation Frame. The main
> purpose of this extension is to reduce data delivery delay. Through
> controlling the timing and scope of packet losses, QUICU reduces useless
> transmission to improve transmission efficiency, reduces head-of-line
> blocking to improve transmission timeliness, which are beneficial to
> delay-sensitive applications, especially audio and video applications. This
> document describes the motivation, the operations, and the wire formats of
> the three new types of frames.
>
>
>
> Link to html:
> https://datatracker.ietf.org/doc/html/draft-chen-quic-quicu-01
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-chen-quic-quicu-01__;!!GjvTz_vk!VHC6upHgCOojEiYhln21JerQBPYz1cbPLifo7gevl5sFTM-5qEoTkN-TxpvuKG1MQfnSBiZsXrrQ2OC2qol99skU$>
>
>
>
> Please feel free to reach us for any comments or questions on this.
>
>
>
> Thanks,
>
> Yongyi.
>
>
>
> ---------- Forwarded message ---------
>
> From: [e-mail]
>
> Date: Mon, Sep 19, 2022, 20:54
>
> Subject: [External] New Version Notification for
> draft-chen-quic-quicu-01.txt
>
> To: "Jianping Chen"<[e-mail]>, "Junxian Jing"<
> [e-mail]>, "Tianyi Liu"<[e-mail]>,
> "Yongyi Yu"<[e-mail]>
>
> A new version of I-D, draft-chen-quic-quicu-01.txt
>
> has been successfully submitted by Yongyi Yu and posted to the
>
> IETF repository.
>
>
>
> Name:                draft-chen-quic-quicu
>
> Revision:        01
>
> Title:                An Unreliable Extension to QUIC
>
> Document date:        2022-09-19
>
> Group:                Individual Submission
>
> Pages:                10
>
> URL:
> https://www.ietf.org/archive/id/draft-chen-quic-quicu-01.txt
> <https://urldefense.com/v3/__https:/www.ietf.org/archive/id/draft-chen-quic-quicu-01.txt__;!!GjvTz_vk!VHC6upHgCOojEiYhln21JerQBPYz1cbPLifo7gevl5sFTM-5qEoTkN-TxpvuKG1MQfnSBiZsXrrQ2OC2qgIVZwR1$>
>
> Status:         https://datatracker.ietf.org/doc/draft-chen-quic-quicu/
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-chen-quic-quicu/__;!!GjvTz_vk!VHC6upHgCOojEiYhln21JerQBPYz1cbPLifo7gevl5sFTM-5qEoTkN-TxpvuKG1MQfnSBiZsXrrQ2OC2qqEw9hmD$>
>
> Htmlized:
> https://datatracker.ietf.org/doc/html/draft-chen-quic-quicu
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-chen-quic-quicu__;!!GjvTz_vk!VHC6upHgCOojEiYhln21JerQBPYz1cbPLifo7gevl5sFTM-5qEoTkN-TxpvuKG1MQfnSBiZsXrrQ2OC2qk9rs9H6$>
>
> Diff:           https://www.ietf.org/rfcdiff?url2=draft-chen-quic-quicu-01
> <https://urldefense.com/v3/__https:/www.ietf.org/rfcdiff?url2=draft-chen-quic-quicu-01__;!!GjvTz_vk!VHC6upHgCOojEiYhln21JerQBPYz1cbPLifo7gevl5sFTM-5qEoTkN-TxpvuKG1MQfnSBiZsXrrQ2OC2qv4nuLVS$>
>
>
>
> Abstract:
>
>   QUIC Unreliable (QUICU) is an extension of the QUIC protocol for
>
>   unreliable data transmission, mainly through the definition and use
>
>   of three new types of frames, namely the Expire Offset Frame,
>
>   Boundary Frame, and Correlation Frame. The main purpose of this
>
>   extension is to reduce data delivery delay. Through controlling the
>
>   timing and scope of packet losses, QUICU reduces useless transmission
>
>   to improve transmission efficiency, reduces head-of-line blocking to
>
>   improve transmission timeliness, which are beneficial to delay-
>
>   sensitive applications, especially audio and video applications.
>
>
>
>   This document describes the motivation, the operations, and the wire
>
>   formats of the three new types of frames.
>
>
>
>
>
>
>
>
>
> The IETF Secretariat
>
>
>