Re: What to do about multipath in QUIC
Mikkel Fahnøe Jørgensen <mikkelfj@gmail.com> Wed, 11 November 2020 15:48 UTC
Return-Path: <mikkelfj@gmail.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 55A0D3A0BAD for <quic@ietfa.amsl.com>; Wed, 11 Nov 2020 07:48:52 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.096
X-Spam-Level:
X-Spam-Status: No, score=-1.096 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, FREEMAIL_REPLY=1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id X6UjoSbssgq2 for <quic@ietfa.amsl.com>; Wed, 11 Nov 2020 07:48:49 -0800 (PST)
Received: from mail-yb1-xb2d.google.com (mail-yb1-xb2d.google.com [IPv6:2607:f8b0:4864:20::b2d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8B7663A0147 for <quic@ietf.org>; Wed, 11 Nov 2020 07:48:49 -0800 (PST)
Received: by mail-yb1-xb2d.google.com with SMTP id k65so2340186ybk.5 for <quic@ietf.org>; Wed, 11 Nov 2020 07:48:49 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:in-reply-to:references:mime-version:date:message-id:subject:to :cc; bh=9kiOS0kS4yKff+UlAeTSaGWN0fGrod1OOxRfSBSdDw8=; b=ukAIU6qbYhZeOgbulJZAU9EtaCHTe17zQiHhwI5/4ytF5V+jDhHD/Gm9AeswCbzHnv 2osgTbuGXY/XgX47OxeIVPzeaHeVLyiCC5ejqLMxQAg/4MMC4bZLfZMzSVXPy3P1XWcK Ldgy9wOsQSdKRzYCctrrHQUWOmY7ISGb4ZHe4diXy6+SJtDQ1K+J9KyfPjve2WwwXvzk aSYtahw3Wbmfay6gwPP8n9K202hMPnZAEuDzW+GRn4+QtczQs7bXeglVqj1WFQ7/Bh6V 8wf3WvdP5CayTAXb6+x56uvENXpUP+cCsalRHGjflDSxfrsGs4+N02NuJ6+qNtvD2Zkt RDQg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:in-reply-to:references:mime-version:date :message-id:subject:to:cc; bh=9kiOS0kS4yKff+UlAeTSaGWN0fGrod1OOxRfSBSdDw8=; b=AQPMu1E4p7yHAzrE/xxOqXVWpodd+pkAsEPbdp1MpsG3UK32+ntQe0v++DdneykanP YEFCHGfKT1nkj7jrG3go59Q8rTWTU0MJraPmUJDWzwyQB36eqY4KZ4y4eJZLtp8+vUPR 2J4YdBL0tKUwFDLdPDpad1vfm62f5V4BHErfbEkpK1pmQtyDRs2ekjzyIE8DRK1B0wGl j+EWdhPEAYmoOgr3mSZUir0ZVnCNuKD3uo0vZqIoDc6CBiYfPwaZV9xm0FCWLe/23r9f H4U1M6/xjbDnjNHu2izZC2XUltKTUZQ3BUBhZ7ri7txZIHx7z9Wl3Wyyp/0Lt4ULZDFr k3FA==
X-Gm-Message-State: AOAM5305Ig285YucAA0Uq5WqZ5dU1c5oB8BjQtDyedtNvual7978cXej nxSRBnSuj9BVQw46DZlS+/jGOSrWRn0r4gDrpEWtywDwVjWnoA==
X-Google-Smtp-Source: ABdhPJyFxtSsHmKgq2v/EWxEZauKA3EG+QZAXSSHOAeQ8re7GsZmKjxw0JEWDomWMqRjP3UW3wgz/q6qBq+YVQfjhug=
X-Received: by 2002:a25:7cc2:: with SMTP id x185mr34339054ybc.263.1605109728774; Wed, 11 Nov 2020 07:48:48 -0800 (PST)
Received: from 1058052472880 named unknown by gmailapi.google.com with HTTPREST; Wed, 11 Nov 2020 07:48:48 -0800
From: Mikkel Fahnøe Jørgensen <mikkelfj@gmail.com>
In-Reply-To: <CAN1APdetAp8db5g1dgEdsXL9qW7s23myBYd4_a6m0zAyue=6XA@mail.gmail.com>
References: <CAKKJt-dOz4JE3_-AVn77H6oY-gjeOL+NNcSWqwpjwM7_LD_0NQ@mail.gmail.com> <CACpbDceKcHG4TwjsvHZsy4=yrb3BUxUBNHDCdYJaq1pBP9kV0w@mail.gmail.com> <CAC8QAcdqL0HaaFJwPF5Dp=wcHSdGuRgZEuM9ehA0BJVjm+3j8w@mail.gmail.com> <CALGR9oYdgHXvOOu7sh1qw+ZewjTapv1QR51fzjxVzke9E3W-+g@mail.gmail.com> <CAKKJt-egOSaakzfiR6Zb8owLRWbTJmxHHMRwBsTUF3p4jh1R5g@mail.gmail.com> <CALGR9oaS3mq5OsitAsCEv8gfAhjW59yKJWJx73vGEM_+tLyvrg@mail.gmail.com> <etPan.5fa58bad.3aecac40.166ff@gmail.com> <CAKKJt-fY8zOYLo62CdxkmDwa=9esiUJRrWyMy10qkhvcqGJ4fQ@mail.gmail.com> <CAN1APdfk6oFTcGzrpDJ6Nm4iOFOuMM-qq_Dk9JVdWwqWj5eWTA@mail.gmail.com> <CAKKJt-cSMp1+ZcF8Le_GqKa7Jm2UVw5G7Qj-7zY21y_gEhLbVA@mail.gmail.com> <8bf17aeb-2545-4b8a-24bd-a495a38bda9d@huitema.net> <CAN1APdfzYNr_=z8im8FH-1tsyzZ9XedXwkHKeU5=oNnP695Adw@mail.gmail.com> <CAC8QAccsf3rg6eDFHA7Mdzuv53fzZSWFKgrQ31Y40kz4kWPViQ@mail.gmail.com> <CALGR9oaruwFvtWLMSw71NXbo03jYpajfmXRcZB_RVm-M-i6a4g@mail.gmail.com> <c39ea2c0-dfaa-6790-b307-c654b918158b@uclouvain.be> <CALGR9oZ+OXGeJrLzHjaak01vX5W2Ty9Z=8Nut5ifMkYz1Xw4SA@mail.gmail.com> <CAC8QAcfZc0rhNzH8+0EfAsE2vj7ZTcc6eCeaGF00n5bk-aKvCA@mail.gmail.com> <7931447A-E557-4B7E-8256-BD6004F29CBF@fb.com> <CAN1APddbM0M7oEw_0f_8dZyWP_ns-J7SXxkk3ZNSL9PjEn+NUw@mail.gmail.com> <885B4E73-5639-40E9-BB92-BCB686FD55B1@fb.com> <CAN1APdetAp8db5g1dgEdsXL9qW7s23myBYd4_a6m0zAyue=6XA@mail.gmail.com>
MIME-Version: 1.0
Date: Wed, 11 Nov 2020 07:48:48 -0800
Message-ID: <CAN1APdd7HnoTpCuYKUkVqahgfQT11hBtVSwAkCWyLHVtCkT-vQ@mail.gmail.com>
Subject: Re: What to do about multipath in QUIC
To: Lucas Pardue <lucaspardue.24.7@gmail.com>, "sarikaya@ieee.org" <sarikaya@ieee.org>, Roberto Peon <fenix@fb.com>
Cc: Christian Huitema <huitema@huitema.net>, Olivier Bonaventure <olivier.bonaventure@uclouvain.be>, QUIC WG <quic@ietf.org>
Content-Type: multipart/alternative; boundary="0000000000003590fa05b3d6bb7c"
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/P5Et67qdyPOZYsPkcEJVQ9j3sAo>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 11 Nov 2020 15:48:52 -0000
Sorry, regarding LBs, forgot say: Some LBs cut connections even if they are not idle. Google L7 does that on websockets on regular HTTPS 1.1 Thus you cannot just ping you way to liveness. Kind Regards, Mikkel Fahnøe Jørgensen On 11 November 2020 at 16.46.54, Mikkel Fahnøe Jørgensen (mikkelfj@gmail.com) wrote: You can’t do that at the app layer. You said yourself, wait for ACK on one path before sending on the other. I can see a protocol that uses both QUIC and TPC to overcome UDP blocks, but then you could also argue that QUIC should support TCP as a transport fallback. I think that would go down the wrong path so to speak. It is better to let QUIC motivate passing UDP in more cases. So I’d rather have a super QUIC doing multiple paths. There is of course WebRTC that uses all sorts of UDP/TCP patterns, but it is fairly complex to get right in all scenarios. Perhaps it is best to let WebRTC handle TCP and move forward with new tech for QUIC. Then you can hack WebRTC to also support QUIC if you can afford it. On a related note: I would be useful to be able to connect to an endpoint through multiple different public access points, i.e. load balancers. The LBs have a tendency to cut idle connections after a while, so being able to connect through multiple would avoid downtime on that account. This is sort of complicated because the connection establishment would see different IPs to the same endpiont, but I guess that is what multipath is? Kind Regards, Mikkel Fahnøe Jørgensen On 10 November 2020 at 18.33.44, Roberto Peon (fenix@fb.com) wrote: I believe it does solve the serialization problem, but let’s talk it out! 😊 My example of TCP+QUIC is the kind of thing you’d do when you were uncertain about one failing (perhaps UDP is being blocked). What I’m talking about is a likely combination of two things: 1) Ability to get some new path or connection to a session (likely a single “server”, but that is the decision of the “server”). - can also be within the connection, so long as the path is addressable by the application - can be external to the app, where different connections are used. 2) Ability of application to schedule data to each path or session (i.e. to mux/demux onto the paths). - This could be via a standardized config, or - It could be up to the application to figure it out - An API for determining how packets are scheduled seems necessary regardless of any other multipath implementation path. An example standardized config could be something like: (race s=1 (path A) (delay 0.1 (path B))) This would say that data should flow on path A by default (it is first), and if you’ve not gotten an ack in 100ms (0.1 s), try path B. If one wanted to send data in duplicate on all paths: (race s=2 (path A) (path B)) If one wanted to send the first X bytes on path A, and the rest on path B: (partition X (path A) (path B)) This kind of config is something we’ve been using for a couple of years for video things, to good effect. It is most certainly not perfect, but it does allow quite a bit of flexibility with a fairly minimal non-Turing complete config. I’m not claiming we should use it, just showing it absolutely can be done in a way that supports about any application need without needing to write code everywhere. -=R *From: *Mikkel Fahnøe Jørgensen <mikkelfj@gmail.com> *Date: *Tuesday, November 10, 2020 at 3:37 AM *To: *Lucas Pardue <lucaspardue.24.7@gmail.com>, Roberto Peon <fenix@fb.com>, "sarikaya@ieee.org" <sarikaya@ieee.org> *Cc: *Olivier Bonaventure <olivier.bonaventure@uclouvain.be>, Christian Huitema <huitema@huitema.net>, QUIC WG <quic@ietf.org> *Subject: *Re: What to do about multipath in QUIC But this doesn’t solve the serialization of a single stream over multiple paths. Also, it doesn’t really make sense to mix a video stream on QUIC on one path and TCP on another. That would cause all kinds of problems, not to mention privacy. I agree that there is a risk of a complex unnecesary feature being poorly implemented in QUIC. But true multipath cannot really be solved outiside of QUIC. I’m fine with making it optional or giving it dedicated version. Someone suggested building multipath on top of multiple QUIC connections. I think that is viable if it is a newer QUIC version that delagates work to older QUIC versions. The key point is that externally this happens transparently, and there are optionas for the QUIC stacks to coordinate locally or remotely via a signalling path. Not sure which solution is ultimately the best, but you build upon what you already have. Kind Regards, Mikkel Fahnøe Jørgensen On 9 November 2020 at 18.32.06, Roberto Peon (fenix@fb.com) wrote: I’m still concerned that we’re looking at solving this inside the connection, instead of providing a way for this to be solved irrespective of the connection. There is a fundamental routing problem we have here that we could address (addressing a session), but we’re not addressing with what I’m seeing discussed (addressing a session within the same connection object). If we consider this problem as making the session addressable, then applications can do it the way that makes sense for them, without having to put everything in every stack everywhere, plus new APIs to actually make them work. I’m afraid if we add multipath, it’ll be like what happened with server push. The lack of appropriate APIs made using it with the browser fraught with tradeoffs with no reasonable way for an application to fix. Solve the addressing-of-a-session problem, however, and we make it easier to solve the likely API problem that will accompany multipath. Example: I could have a virtual connection which is composed of a TCP connection on path A, and a QUIC connection on path B. .. or maybe I want to try out a new version/extension on QUIC, so I have a virtual connection with QUIC and QUIC+extension. I could declare that I’d like for data to flow down QUIC+extension path unless that is too slow, then duplicate the data onto the QUIC path. In my mind, the application should establishes the virtual connection, and provide at least one path, and can optionally add (and remove) subsequent paths. This is something we do already in “storage-land”, where diversity and separable failure domains are important, and where the use-cases are extremely diverse in latency, data-amounts, and cost. -=R *From: *QUIC <quic-bounces@ietf.org> on behalf of Behcet Sarikaya < sarikaya2012@gmail.com> *Reply-To: *"sarikaya@ieee.org" <sarikaya@ieee.org> *Date: *Monday, November 9, 2020 at 8:58 AM *To: *Lucas Pardue <lucaspardue.24.7@gmail.com> *Cc: *Christian Huitema <huitema@huitema.net>, Behcet Sarikaya < sarikaya@ieee.org>, QUIC WG <quic@ietf.org>, Olivier Bonaventure < Olivier.Bonaventure@uclouvain.be>, Mikkel Fahnøe Jørgensen < mikkelfj@gmail.com> *Subject: *Re: What to do about multipath in QUIC Hi Lucas, Olivier, On Mon, Nov 9, 2020 at 10:51 AM Lucas Pardue <lucaspardue.24.7@gmail.com> wrote: Hey Olivier, On Mon, Nov 9, 2020 at 4:31 PM Olivier Bonaventure < Olivier.Bonaventure@uclouvain.be> wrote: Lucas, > > On Mon, Nov 9, 2020 at 3:55 PM Behcet Sarikaya <sarikaya2012@gmail.com > <mailto:sarikaya2012@gmail.com>> wrote: > > Hi Folks, > I agree with Mikkel's points. > To Lucas: I meant my short mail sometime ago I think it was before > the interim (?) where I explained that connection migration is > mobility support which should (from layering point of view) be in IP > layer. In fact if IP layer has this support then then no need for > connection migration in QUIC, so those procedures in the code do not > get executed. > > Multipath is multiple interface support. It seems more and more like > multipath probably better belongs in transport layer. Traffic in > each interface may go over different networks (in my case on over T > Mobile and the other AT&T). I believe a different PN is well > justified in multipath as we have it in the base draft because of > these traffic conditions (no offense to Christian). > > > I still don't see why the current features of connection migration are > not in some way a form of multipath. You are right, connection migration is the weakest form of multipath. Thanks. We heard use cases that would like stronger forms. I think it will help continue to move the discussion forward if we can establish some common ground on terms and capabilities. This paragraph of RFC6824 then continues as follows : However, to the network layer, each MPTCP subflow looks like a regular TCP flow whose segments carry a new TCP option type. Multipath TCP manages the creation, removal, and utilization of these subflows to send data. The number of subflows that are managed within a Multipath TCP connection is not fixed and it can fluctuate during the lifetime of the Multipath TCP connection. This is not really connection migration and MPTCP provides much more multipath capabilities than connection migration. Yeah I follow. As someone coming from QUIC, the first sentence is kind of easily negated (which is a benefit IIUC). I think the remainder of the paragraph is partially satisfied by QUIC v1 if we consider PATH_CHALLENGE/PATH_RESPONSE and NEW_CONNECTION_ID/RETIRE_CONNECTION_ID. But it starts to fall apart when you want to do more complicated things. I think understanding the gaps in the transport signalling would be useful to document in isolation to any specific solution. draft-deconinck-quic-multipath has done some of that work already but it gets a little too tied up with the solution IMO. I don't think Olivier would wish to undermine the most important feature of multipath: multiple paths going over concurrently possible over different networks. Then he can not justify many features in draft-deconinck-quic-multipath. Behcet Cheers Lucas
- What to do about multipath in QUIC Spencer Dawkins at IETF
- Re: What to do about multipath in QUIC Martin Duke
- Re: What to do about multipath in QUIC Spencer Dawkins at IETF
- Re: What to do about multipath in QUIC Jana Iyengar
- Re: What to do about multipath in QUIC Spencer Dawkins at IETF
- Re: What to do about multipath in QUIC Behcet Sarikaya
- Re: What to do about multipath in QUIC Lucas Pardue
- Re: What to do about multipath in QUIC Spencer Dawkins at IETF
- Re: What to do about multipath in QUIC Lucas Pardue
- Re: What to do about multipath in QUIC Mikkel Fahnøe Jørgensen
- Re: What to do about multipath in QUIC Spencer Dawkins at IETF
- Re: What to do about multipath in QUIC Spencer Dawkins at IETF
- Re: What to do about multipath in QUIC Mikkel Fahnøe Jørgensen
- Re: What to do about multipath in QUIC Spencer Dawkins at IETF
- Re: What to do about multipath in QUIC Christian Huitema
- Re: What to do about multipath in QUIC Mikkel Fahnøe Jørgensen
- Re: What to do about multipath in QUIC Behcet Sarikaya
- Re: What to do about multipath in QUIC Lucas Pardue
- Re: What to do about multipath in QUIC Behcet Sarikaya
- Re: What to do about multipath in QUIC Olivier Bonaventure
- Re: What to do about multipath in QUIC Behcet Sarikaya
- Re: What to do about multipath in QUIC Lucas Pardue
- Re: What to do about multipath in QUIC Behcet Sarikaya
- Re: What to do about multipath in QUIC Roberto Peon
- Re: What to do about multipath in QUIC Mikkel Fahnøe Jørgensen
- Re: What to do about multipath in QUIC Kazuho Oku
- Re: What to do about multipath in QUIC Mikkel Fahnøe Jørgensen
- Re: What to do about multipath in QUIC Lars Eggert
- Re: What to do about multipath in QUIC Mikkel Fahnøe Jørgensen
- Re: What to do about multipath in QUIC Spencer Dawkins at IETF
- Re: What to do about multipath in QUIC Lars Eggert
- Re: What to do about multipath in QUIC Mikkel Fahnøe Jørgensen
- Re: What to do about multipath in QUIC Mikkel Fahnøe Jørgensen
- Re: What to do about multipath in QUIC Mikkel Fahnøe Jørgensen
- Re: What to do about multipath in QUIC Roberto Peon
- Re: What to do about multipath in QUIC Mikkel Fahnøe Jørgensen
- Re: What to do about multipath in QUIC Martin Thomson
- Re: What to do about multipath in QUIC Yanmei Liu
- Re: What to do about multipath in QUIC Mikkel Fahnøe Jørgensen
- Re: What to do about multipath in QUIC Spencer Dawkins at IETF
- Re: What to do about multipath in QUIC Mikkel Fahnøe Jørgensen
- Terminology for multipath QUIC (was Re: What to d… Lucas Pardue
- Re: What to do about multipath in QUIC Spencer Dawkins at IETF
- Re: Terminology for multipath QUIC (was Re: What … Spencer Dawkins at IETF
- Re: What to do about multipath in QUIC Christian Huitema
- Re: What to do about multipath in QUIC Spencer Dawkins at IETF
- Re: What to do about multipath in QUIC Mirja Kuehlewind
- Re: What to do about multipath in QUIC Mirja Kuehlewind
- Re: What to do about multipath in QUIC Spencer Dawkins at IETF
- Re: What to do about multipath in QUIC Yanmei Liu
- Re: What to do about multipath in QUIC Behcet Sarikaya
- Re: What to do about multipath in QUIC Yanmei Liu
- Re: What to do about multipath in QUIC Kazuho Oku
- Re: Re: What to do about multipath in QUIC Ma, Yunfei
- Re: What to do about multipath in QUIC Yunfei Ma
- Re: What to do about multipath in QUIC Kazuho Oku
- Re: What to do about multipath in QUIC Florentin Rochet
- Re: What to do about multipath in QUIC Christian Huitema
- Re: What to do about multipath in QUIC Mikkel Fahnøe Jørgensen
- Re: What to do about multipath in QUIC Christian Huitema
- Re: What to do about multipath in QUIC Mikkel Fahnøe Jørgensen
- Re: What to do about multipath in QUIC Kazuho Oku
- Re: What to do about multipath in QUIC Christian Huitema
- Re: What to do about multipath in QUIC Kazuho Oku
- Re: What to do about multipath in QUIC Christian Huitema
- Re: What to do about multipath in QUIC Kazuho Oku
- Re: What to do about multipath in QUIC Christian Huitema
- Re: What to do about multipath in QUIC Kazuho Oku
- Re: What to do about multipath in QUIC Christian Huitema
- Re: What to do about multipath in QUIC Spencer Dawkins at IETF
- Packet number spaces in multipath (was Re: What t… Jana Iyengar
- RE: Packet number spaces in multipath (was Re: Wh… Markus.Amend
- Re: Packet number spaces in multipath (was Re: Wh… Christian Huitema
- RE: Packet number spaces in multipath (was Re: Wh… Markus.Amend
- RE: Packet number spaces in multipath (was Re: Wh… Mikkel Fahnøe Jørgensen
- RE: Packet number spaces in multipath (was Re: Wh… Mikkel Fahnøe Jørgensen
- Re: Packet number spaces in multipath (was Re: Wh… Roberto Peon
- RE: Packet number spaces in multipath (was Re: Wh… Markus.Amend
- Re: Packet number spaces in multipath (was Re: Wh… Mirja Kuehlewind
- Re: Packet number spaces in multipath (was Re: Wh… Jana Iyengar
- Re: Packet number spaces in multipath (was Re: Wh… Quentin De Coninck
- Re: Packet number spaces in multipath (was Re: Wh… Mikkel Fahnøe Jørgensen
- Re: Packet number spaces in multipath (was Re: Wh… Kazuho Oku
- Re: Packet number spaces in multipath (was Re: Wh… Quentin De Coninck
- Re: Packet number spaces in multipath (was Re: Wh… Martin Thomson
- Re: Packet number spaces in multipath (was Re: Wh… Kazuho Oku
- RE: Packet number spaces in multipath (was Re: Wh… Mike Bishop