Re: [Webtransport] Measuring the friction of Http3Transport

[David Schinazi <[e-mail]>]

On Mon, Nov 16, 2020 at 6:02 PM Luke Curley <[e-mail]> wrote:

> I think that's a good starting summary David. Some things worth mentioning:
>
> For 1 and 4, Twitch is using QuicTransport for both native applications
> (ex. mobile, desktop, devices) and web applications. It's minor, but we
> benefit from the ability to use any QUIC library (technically we don't
> implement datagrams...) instead only those with HTTP/3 and Http3Transport
> support. I think this context was missing in the previous discussions.
>

Can you clarify which implementations you were considering that support
QUIC+DATAGRAM but not HTTP/3?


> For 3, QuicTransport can support 0-RTT if we declare the path/origin
> handshake to be idempotent
> <https://github.com/vasilvv/webtransport/issues/16>... unless the new
> connection is routed to a different host. Both pooled Http3Transport and a
> new QuicTransport connection could take the same number of round trips.
>

0-RTT state isn't saved across ALPNs, even if it's to the same host. So
Http3Transport would get inline 0-RTT but QuicTransport would not get 0-RTT
unless you've already performed a QuicTransport handshake in recent history.


> For 4 and 5, I'm worried about flow control
> <https://github.com/vasilvv/webtransport/issues/22>. tl;dr a
> Http3Transport session could easily exhaust flow control limits for the
> entire connection. This seems especially negative if pooling is performed
> transparently (ex. a Http3Transport session may silently break HTTP/3 or
> other sessions).
>

Unintentionally hitting the connection-level flow control limit is a
serious implementation bug. I wouldn't expect this to still be a problem
once implementations mature. I wouldn't want to base our protocol choices
based on a bug in non-production-ready implementations.


> On Mon, Nov 16, 2020 at 5:10 PM David Schinazi <[e-mail]>
> wrote:
>
>> I'd like us to take a step back, I think we're conflating many different
>> topics here, and it's making it impossible to reason about this (at least
>> for someone running on not enough sleep like I am this week).
>>
>> Topic 1: the complexity of implementing Http3Transport as opposed to
>> QuicTransport on the client
>>     From my perspective, all browsers interested in WebTransport are
>> implementing HTTP/3, so I don't see much complexity there.
>>     Please reply here if you know of complexities I'm not seeing
>>
>> Topic 2: the complexity of implementing Http3Transport as opposed to
>> QuicTransport on the server
>>     As far as I know, most server-side QUIC stacks also support HTTP/3,
>> and for QuicTransport they already have DATAGRAM support so I think the
>> delta is negligible.
>>     Please reply here if you know of complexities I'm not seeing
>>
>> Topic 3: the benefits of pooling WebTransport
>>     This allows saving round trips to get WebTransport up and running
>>     This allows sharing congestion control state
>>     This allows letting the QUIC implementation prioritize DATAGRAMs vs
>> HTTP requests instead of relying on the kernel or routers which have less
>> information
>>     Overall, I think pooling is beneficial (though not an absolute
>> must-have)
>>     Please reply here if you know of benefits I'm not seeing
>>
>> Topic 4: the difficulty of pooling Http3Transport on the client
>>     It appears that all browsers that are interested in WebTransport are
>> implementing pooling in their HTTP/3 stack so additionally pooling
>> WebTransport should not introduce more complexity
>>     Please reply here if you know of complexities I'm not seeing
>>
>> Topic 5: the difficulty of pooling Http3Transport on the server
>>     This one is trickier. If you have frontends that know how to route
>> HTTP requests to backends, DATAGRAM support is going to be a whole new
>> world.
>>     However, and I think this is really important: implementing this on
>> the server is OPTIONAL.
>>     The server can easily use a different hostname (and different TLS
>> certificates) for HTTP vs WebTransport and clients will not pool the two
>> together.
>>     Once you use different domain names per WebTransport object,
>> Http3Transport has the exact same pooling properties as QuicTransport.
>>     Please reply here if you know of complexities I'm not seeing
>>
>> What am I missing?
>>
>> Thanks,
>> David
>>
>>
>> On Mon, Nov 16, 2020 at 4:45 PM Ian Swett <[e-mail]> wrote:
>>
>>> I'm *very* concerned about any functionality that *requires* pooling.
>>> Pooling is a client-specific optimization and pooling policy is subject to
>>> change(within reason) at any time.  If applications are saying "it'd be
>>> cool if we could share congestion control and loss recovery state between
>>> H3 and WebTransport most of the time", then I can see there being
>>> real(albeit small) benefits, just like there would be for putting
>>> DataChannels and RTP into one QUIC connection for WebRTC.  If people are
>>> intending to rely on pooling so an HTTP(hopefully H3) request can trigger a
>>> WebTransport response, I think we're in a world of pain and
>>> over-optimization.
>>>
>>> This is more obvious in light of the fact that a WebTransport
>>> endpoint is likely not a generic CDN endpoint, as has been clearly stated
>>> by Mark Nottingham.
>>>
>>> Bernard's comments above point to a tighter coupling between HTTP/3 and
>>> WebTransport than I think is practical in production, unless I
>>> misunderstood(which I may have, it's been a long day).
>>>
>>> All of this makes me think we should punt the HTTP based WebTransport
>>> options down the road a year or so and focus on QuicTransport initially.
>>> It's not to say we won't do them, but they're clearly more complex
>>> than QuicTransport and it'd be good to have real deployment experience
>>> prior to starting so we know what the requirements are and where the
>>> dragons are lurking.  Also, the W3C should design APIs that make it easy to
>>> detect failure to establish a QUIC-based WebTransport, and this ensures
>>> those are well exercised.  Maybe we DO require pooling under certain
>>> conditions and describe those, but there are likely privacy concerns under
>>> some circumstances and I'm not enough of an expert to know what would be
>>> acceptable to all clients.
>>>
>>> Some people in the WG may think we have to release a TCP based
>>> WebTransport at the same time as a QUIC based one, but I don't think that's
>>> true and even if we standardized them at the same time, it doesn't mean
>>> both will achieve wide deployment at the same time.  As I said at the mic,
>>> I don't see any reason YouTube would deploy a TCP based WebTransport given
>>> it will have to maintain an HTTP/1.1 option for basically forever and
>>> they've spent a good bit of effort to make sure that works as well as it
>>> can.  It's clear to me there are compelling use cases right now for
>>> QuicTransport, but the driving use cases for the fallback are not clear to
>>> me.  Consistency is intellectually appealing, but doesn't drive deployments.
>>>
>>> Ian
>>>
>>> On Mon, Nov 16, 2020 at 6:36 PM Bernard Aboba <[e-mail]>
>>> wrote:
>>>
>>>> David said:
>>>>
>>>> "Where does this come from? A lot of QUIC servers are implementing QUIC
>>>> datagrams, and once you have those the delta with h3 datagrams is
>>>> negligible. Additionally, multiple h3 server implementers have indicated
>>>> they want to support WebTransport. It's starting to feel like the pushback
>>>> on HTTP3Transport is purely based on a perception of complexity from folks
>>>> who haven't tried to implement it? If that's the case, then we should get
>>>> some implementation experience to confirm or debunk that theory."
>>>>
>>>> [BA] The pooling use cases often involve an interaction between an
>>>> HTTP/3 request/response and WebTransport.  so the HTTP/3 Request doesn't
>>>> just elicit an HTTP/3 Response.  It also causes something else to happen,
>>>> something that isn't specified in the HTTP/3 protocol, such as eliciting a
>>>> datagram flow or bi/unidirectional stream from the server.  This is what is
>>>> contemplated in several of the "Live Event" use cases, where the HTTP/3
>>>> Request/Response would be requesting the desired Event. The major advantage
>>>> over HLS is avoiding Head-of-Line blocking, reducing video freezes and
>>>> stalls.  So developers understand the value that support for datagrams and
>>>> server-initiated streams can provide - the question is whether they'll have
>>>> all the interoperable pieces in place to deploy it.
>>>>
>>>> Just because a generic HTTP/3 server supports QUIC and WebTransport,
>>>> doesn't mean that it will support these use cases.  I've been asked whether
>>>> better interoperability and wider ecosystem support could be achieved by
>>>> extending HTTP/3 in a way more in keeping with the existing protocol design
>>>> - such as by allowing an HTTP/3 Request to specify a resource to be
>>>> returned in a flow of datagrams.
>>>>
>>>> The situation might be more tenable if the node.js framework had HTTP/3
>>>> and QUIC modules on the horizon.
>>>>
>>>> On Mon, Nov 16, 2020 at 1:00 PM David Schinazi <
>>>> [e-mail]> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Mon, Nov 16, 2020 at 12:40 PM Bernard Aboba <
>>>>> [e-mail]> wrote:
>>>>>
>>>>>> Luke said:
>>>>>>
>>>>>> "My primary concern is that Http3Transport requires non-generic
>>>>>> modifications to the HTTP/3 layer mostly to support connection pooling. I'm
>>>>>> worried that this will limit support for Http3Transport and complicates any
>>>>>> HTTP/3 implementation that does support it."
>>>>>>
>>>>>> [BA] I have also heard this concern from developers. The concerns
>>>>>> primarily arise from the pooling of HTTP/3 Request/Responses with
>>>>>> WebTransport. This does enable some interesting use cases, such as a
>>>>>> Request/Response (e.g. gRPC) that can result in a server sending datagrams
>>>>>> or initiating bi-directional or uni-directional streams.  So there is
>>>>>> perceived value, but the question is whether that value will be widely
>>>>>> supported across the HTTP/3 ecosystem.
>>>>>>
>>>>>> At this point, most generic HTTP/3 servers are not expected to
>>>>>> support HTTP/3 datagrams (not part of the HTTP/3 protocol) or
>>>>>> Http3Transport (perceived as a niche protocol), so that pooling would only
>>>>>> be possible when using a framework.  node.js will not have QUIC support for
>>>>>> 12-18 months (held up by OpenSSL) so even framework support is expected to
>>>>>> be limited.
>>>>>>
>>>>>
>>>>> Where does this come from? A lot of QUIC servers are implementing QUIC
>>>>> datagrams, and once you have those the delta with h3 datagrams is
>>>>> negligible. Additionally, multiple h3 server implementers have indicated
>>>>> they want to support WebTransport. It's starting to feel like the pushback
>>>>> on HTTP3Transport is purely based on a perception of complexity from folks
>>>>> who haven't tried to implement it? If that's the case, then we should get
>>>>> some implementation experience to confirm or debunk that theory.
>>>>>
>>>>>
>>>>>> On Mon, Nov 16, 2020 at 11:37 AM David Schinazi <
>>>>>> [e-mail]> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Nov 16, 2020 at 11:34 AM Luke Curley <[e-mail]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> We're on the same page that there's code in HTTP/3 that could be
>>>>>>>> leveraged for QuicTransport in the form of the header parsing. I think
>>>>>>>> there's a path to merge QuicTransport and Http3Transport by leveraging
>>>>>>>> HTTP/3 for the handshake and QUIC for stream delivery.
>>>>>>>>
>>>>>>>> My primary concern is that Http3Transport requires non-generic
>>>>>>>> modifications to the HTTP/3 layer mostly to support connection pooling. I'm
>>>>>>>> worried that this will limit support for Http3Transport and complicates any
>>>>>>>> HTTP/3 implementation that does support it.
>>>>>>>>
>>>>>>>
>>>>>>> Hi Luke,
>>>>>>>
>>>>>>> I'm curious, could you elaborate on what these "non-generic
>>>>>>> modifications to the HTTP/3 layer" are please?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>>> On Mon, Nov 16, 2020 at 4:49 AM Lucas Pardue <
>>>>>>>> [e-mail]> wrote:
>>>>>>>>
>>>>>>>>> Hi Luke,
>>>>>>>>>
>>>>>>>>> On Mon, 16 Nov 2020, 12:26 Luke Curley, <[e-mail]>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Just to clarify, when I say "QuicTransport is simpler" I really
>>>>>>>>>> mean "QuicTransport has greater interoperability". You can take any QUIC
>>>>>>>>>> implementation (+ datagram extension) and add a thin layer to support
>>>>>>>>>> QuicTransport. It's a nice layering of protocols and it's something that I
>>>>>>>>>> want to see in general as QUIC replaces TCP.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks for the clarification. I appreciate the sentiment. But
>>>>>>>>> given the development of QUIC its kind of surprising that people would find
>>>>>>>>> it easier to identify an implementation that supports QUIC and datagram but
>>>>>>>>> does not support HTTP/3.
>>>>>>>>>
>>>>>>>>> Thr situation for a more clean-room implementation is different, I
>>>>>>>>> agree. However, by the time you've done all the hard stuff with QUIC, a
>>>>>>>>> very focused HTTP/3 layer ends up quite thin too. You can ignore dynamic
>>>>>>>>> compression, server push and extensibility points.
>>>>>>>>>
>>>>>>>>> I also see the current design being the thin end of the wedge. The
>>>>>>>>> proposal I linked talks about HTTP-like header parsing. So its "just" a
>>>>>>>>> thin layer to do some parsing. Then folks will probably want some want to
>>>>>>>>> tweak parameter, acontrol channel, a way to do graceful close, a sprinkle
>>>>>>>>> of GREASE. To extrapolate forward, what would be the delta be between some
>>>>>>>>> final product QuicTransport and HTTP/3. Too small a delta and we've wasted
>>>>>>>>> years of effort.
>>>>>>>>>
>>>>>>>>> Cheers
>>>>>>>>> Lucas
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Mon, Nov 16, 2020 at 3:16 AM Lucas Pardue <
>>>>>>>>>> [e-mail]> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi folks,
>>>>>>>>>>>
>>>>>>>>>>> To follow up on one thread of discussion during IETF 109. To
>>>>>>>>>>> over simplify what we've heard a few folks say, they prefer QuicTransport
>>>>>>>>>>> because it simpler and they don't need pooling.
>>>>>>>>>>>
>>>>>>>>>>> I'd like to push more on this, what is the measurable complexity
>>>>>>>>>>> of HTTP/3 over QUIC plus a new application mapping that needs to be
>>>>>>>>>>> defined? Let's isolate pooling as a variable and ignore it.
>>>>>>>>>>>
>>>>>>>>>>> Victor mentioned his proposal for a unified header semantic
>>>>>>>>>>> across all transports, this is
>>>>>>>>>>> https://github.com/ietf-wg-webtrans/draft-ietf-webtrans-overview/pull/4/files.
>>>>>>>>>>> Taking a closer look at it, I have some concerns that I added as comments.
>>>>>>>>>>> HTTP has semantics for a reason. If QuicTransport is going to start
>>>>>>>>>>> borrowing HTTP piecemeal, I really wonder how simple people might feel it
>>>>>>>>>>> is. I think it's important to avoid something that looks like HTTP but
>>>>>>>>>>> behaves differently. Is it really much of a stretch to just do HTTP/3 but
>>>>>>>>>>> give it a different ALPN so that we avoid problems related to
>>>>>>>>>>> cross-protocol pooling?
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Lucas
>>>>>>>>>>> --
>>>>>>>>>>> Webtransport mailing list
>>>>>>>>>>> [e-mail]
>>>>>>>>>>> https://www.ietf.org/mailman/listinfo/webtransport
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>> Webtransport mailing list
>>>>>>>> [e-mail]
>>>>>>>> https://www.ietf.org/mailman/listinfo/webtransport
>>>>>>>>
>>>>>>> --
>>>>>>> Webtransport mailing list
>>>>>>> [e-mail]
>>>>>>> https://www.ietf.org/mailman/listinfo/webtransport
>>>>>>>
>>>>>> --
>>>> Webtransport mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/webtransport
>>>>
>>>