Re: What to do about multipath in QUIC

[Mikkel Fahnøe Jørgensen <[e-mail]>]

I can’t say I disagree with server push og H2 priorities, and yes there is
a risk of going unused. But I doubt it. With 5G, 4G, WiFi and ethernet all
being available at once, I see this as a vast untapped market that just
hasnøt been viable so far. If your propasal was easy, it would be widely
deployed rather than limited to heavyweights end to end controlled cases
like FaceBook.

Building an application on top of dubious low level exposure is at best
lock in to one stack, and worst case not possible because the peer can’t
support it. So MP can have esoteric APIs for specifying available paths,
but all supporting stacks would have some way of configuring it.

Kind Regards,
Mikkel Fahnøe Jørgensen


On 11 November 2020 at 21.39.42, Roberto Peon ([e-mail]) wrote:

What the application can do, or what it can cause to be done depends on the
API.
Does the application get a callback or have opportunity to take action when
data has been received/when you’ve received and ack or nack’d?
Does it have a way to express to the lower level stack what it should do?

In recent memory, I’ve said that H2 priorities in their original form are
dead or should be.
Within the past few days we’re seeing Ian mention Chrome killing server
push.
I believe that we’ve not had the appropriate APIs to make these features
useful in the browsers and sometimes in servers, and that has resulted in
their death on some platforms, because there has been no reasonable way to
use them.

I believe we’re likely to suffer exactly the same thing here—a feature that
has obvious theoretical advantages, but which has tradeoffs in use.
Because it has tradeoffs, it will fail in applications or browsers without
the appropriate API which allows the application to deal with those
tradeoffs and play/innovate in the space.

-=R

*From: *Mikkel Fahnøe Jørgensen <[e-mail]>
*Date: *Wednesday, November 11, 2020 at 7:47 AM
*To: *"[e-mail]" <[e-mail]>, Roberto Peon <[e-mail]>,
Lucas Pardue <[e-mail]>
*Cc: *QUIC WG <[e-mail]>, Olivier Bonaventure <
[e-mail]>, Christian Huitema <[e-mail]>
*Subject: *Re: What to do about multipath in QUIC



You can’t do that at the app layer. You said yourself, wait for ACK on one
path before sending on the other.



I can see a protocol that uses both QUIC and TPC to overcome UDP blocks,
but then you could also argue that QUIC should support TCP as a transport
fallback. I think that would go down the wrong path so to speak. It is
better to let QUIC motivate passing UDP in more cases. So I’d rather have a
super QUIC doing multiple paths.



There is of course WebRTC that uses all sorts of UDP/TCP patterns, but it
is fairly complex to get right in all scenarios. Perhaps it is best to let
WebRTC handle TCP and move forward with new tech for QUIC. Then you can
hack WebRTC to also support QUIC if you can afford it.



On a related note:



I would be useful to be able to connect to an endpoint through multiple
different public access points, i.e. load balancers. The LBs have a
tendency to cut idle connections after a while, so being able to connect
through multiple would avoid downtime on that account. This is sort of
complicated because the connection establishment would see different IPs to
the same endpiont, but I guess that is what multipath is?





Kind Regards,

Mikkel Fahnøe Jørgensen



On 10 November 2020 at 18.33.44, Roberto Peon ([e-mail]) wrote:

I believe it does solve the serialization problem, but let’s talk it out! 😊

My example of TCP+QUIC is the kind of thing you’d do when you were
uncertain about one failing (perhaps UDP is being blocked).

What I’m talking about is a likely combination of two things:
1) Ability to get some new path or connection to a session (likely a single
“server”, but that is the decision of the “server”).
- can also be within the connection, so long as the path is addressable by
the application

- can be external to the app, where different connections are used.
2) Ability of application to schedule data to each path or session (i.e. to
mux/demux onto the paths).
- This could be via a standardized config, or
- It could be up to the application to figure it out

- An API for determining how packets are scheduled seems necessary
regardless of any other multipath implementation path.

An example standardized config could be something like:
  (race s=1 (path A) (delay 0.1 (path B)))
This would say that data should flow on path A by default (it is first),
and if you’ve not gotten an ack in 100ms (0.1 s), try path B.

If one wanted to send data in duplicate on all paths:
  (race s=2 (path A)  (path B))
If one wanted to send the first X bytes on path A, and the rest on path B:

(partition X (path A) (path B))

This kind of config is something we’ve been using for a couple of years for
video things, to good effect.
It is most certainly not perfect, but it does allow quite a bit of
flexibility with a fairly minimal non-Turing complete config.
I’m not claiming we should use it, just showing it absolutely can be done
in a way that supports about any application need without needing to write
code everywhere.

-=R



*From: *Mikkel Fahnøe Jørgensen <[e-mail]>
*Date: *Tuesday, November 10, 2020 at 3:37 AM
*To: *Lucas Pardue <[e-mail]>, Roberto Peon <[e-mail]>,
"[e-mail]" <[e-mail]>
*Cc: *Olivier Bonaventure <[e-mail]>, Christian
Huitema <[e-mail]>, QUIC WG <[e-mail]>
*Subject: *Re: What to do about multipath in QUIC



But this doesn’t solve the serialization of a single stream over multiple
paths.



Also, it doesn’t really make sense to mix a video stream on QUIC on one
path and TCP on another. That would cause all kinds of problems, not to
mention privacy.



I agree that there is a risk of a complex unnecesary feature being poorly
implemented in QUIC. But true multipath cannot really be solved outiside of
QUIC. I’m fine with making it optional or giving it dedicated version.



Someone suggested building multipath on top of multiple QUIC connections. I
think that is viable if it is a newer QUIC version that delagates work to
older QUIC versions. The key point is that externally this happens
transparently, and there are optionas for the QUIC stacks to coordinate
locally or remotely via a signalling path. Not sure which solution is
ultimately the best, but you build upon what you already have.





Kind Regards,

Mikkel Fahnøe Jørgensen



On 9 November 2020 at 18.32.06, Roberto Peon ([e-mail]) wrote:

I’m still concerned that we’re looking at solving this inside the
connection, instead of providing a way for this to be solved irrespective
of the connection.
There is a fundamental routing problem we have here that we could address
(addressing a session), but we’re not addressing with what I’m seeing
discussed (addressing a session within the same connection object).

If we consider this problem as making the session addressable, then
applications can do it the way that makes sense for them, without having to
put everything in every stack everywhere, plus new APIs to actually make
them work.

I’m afraid if we add multipath, it’ll be like what happened with server
push. The lack of appropriate APIs made using it with the browser fraught
with tradeoffs with no reasonable way for an application to fix.

Solve the addressing-of-a-session problem, however, and we make it easier
to solve the likely API problem that will accompany multipath.

Example:
I could have a virtual connection which is composed of a TCP connection on
path A, and a QUIC connection on path B.
.. or maybe I want to try out a new version/extension on QUIC, so I have a
virtual connection with QUIC and QUIC+extension.
I could declare that I’d like for data to flow down QUIC+extension path
unless that is too slow, then duplicate the data onto the QUIC path.

In my mind, the application should establishes the virtual connection, and
provide at least one path, and can optionally add (and remove) subsequent
paths.
This is something we do already in “storage-land”, where diversity and
separable failure domains are important, and where the use-cases are
extremely diverse in latency, data-amounts, and cost.
-=R



*From: *QUIC <[e-mail]> on behalf of Behcet Sarikaya <
[e-mail]>
*Reply-To: *"[e-mail]" <[e-mail]>
*Date: *Monday, November 9, 2020 at 8:58 AM
*To: *Lucas Pardue <[e-mail]>
*Cc: *Christian Huitema <[e-mail]>, Behcet Sarikaya <
[e-mail]>, QUIC WG <[e-mail]>, Olivier Bonaventure <
[e-mail]>, Mikkel Fahnøe Jørgensen <
[e-mail]>
*Subject: *Re: What to do about multipath in QUIC



Hi Lucas, Olivier,





On Mon, Nov 9, 2020 at 10:51 AM Lucas Pardue <[e-mail]>
wrote:

Hey Olivier,



On Mon, Nov 9, 2020 at 4:31 PM Olivier Bonaventure <
[e-mail]> wrote:

Lucas,
>
> On Mon, Nov 9, 2020 at 3:55 PM Behcet Sarikaya <[e-mail]
> <mailto:[e-mail]>> wrote:
>
>     Hi Folks,
>     I agree with Mikkel's points.
>     To Lucas: I meant my short mail sometime ago I think it was before
>     the interim (?) where I explained that connection migration is
>     mobility support which should (from layering point of view) be in IP
>     layer. In fact if IP layer has this support then then no need for
>     connection migration in QUIC, so those procedures in the code do not
>     get executed.
>
>     Multipath is multiple interface support. It seems more and more like
>     multipath probably better belongs in transport layer. Traffic in
>     each interface may go over different networks (in my case on over T
>     Mobile and the other AT&T). I believe a different PN is well
>     justified in multipath as we have it in the base draft because of
>     these traffic conditions (no offense to Christian).
>
>
> I still don't see why the current features of connection migration are
> not in some way a form of multipath.

You are right, connection migration is the weakest form of multipath.



Thanks. We heard use cases that would like stronger forms. I think it will
help continue to move the discussion forward if we can establish some
common ground on terms and capabilities.



This paragraph of RFC6824 then continues as follows :

    However, to the network layer, each MPTCP subflow looks
    like a regular TCP flow whose segments carry a new TCP option type.
    Multipath TCP manages the creation, removal, and utilization of these
    subflows to send data.  The number of subflows that are managed
    within a Multipath TCP connection is not fixed and it can fluctuate
    during the lifetime of the Multipath TCP connection.

This is not really connection migration and MPTCP provides much more
multipath capabilities than connection migration.



Yeah I follow. As someone coming from QUIC, the first sentence is kind of
easily negated (which is a benefit IIUC). I think the remainder of the
paragraph is partially satisfied by QUIC v1 if we consider
PATH_CHALLENGE/PATH_RESPONSE and NEW_CONNECTION_ID/RETIRE_CONNECTION_ID.
But it starts to fall apart when you want to do more complicated things. I
think understanding the gaps in the transport signalling would be useful to
document in isolation to any specific solution.
draft-deconinck-quic-multipath has done some of that work already but it
gets a little too tied up with the solution IMO.







I don't think Olivier would wish to undermine the most important feature of
multipath: multiple paths going over concurrently possible over different
networks.

Then he can not justify many features in draft-deconinck-quic-multipath.



Behcet

Cheers

Lucas