Re: Options for QUIC Multipath
Christian Huitema <huitema@huitema.net> Sat, 10 April 2021 09:11 UTC
Return-Path: <huitema@huitema.net>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8330E3A2A98 for <quic@ietfa.amsl.com>; Sat, 10 Apr 2021 02:11:38 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.589
X-Spam-Level:
X-Spam-Status: No, score=-2.589 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, NICE_REPLY_A=-0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JpQEAP-t2Mbf for <quic@ietfa.amsl.com>; Sat, 10 Apr 2021 02:11:33 -0700 (PDT)
Received: from mx36-out10.antispamcloud.com (mx36-out10.antispamcloud.com [209.126.121.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 072DD3A2A96 for <quic@ietf.org>; Sat, 10 Apr 2021 02:11:32 -0700 (PDT)
Received: from xse196.mail2web.com ([66.113.196.196] helo=xse.mail2web.com) by mx134.antispamcloud.com with esmtp (Exim 4.92) (envelope-from <huitema@huitema.net>) id 1lV9eI-000H3Q-4x for quic@ietf.org; Sat, 10 Apr 2021 11:11:29 +0200
Received: from xsmtp22.mail2web.com (unknown [10.100.68.61]) by xse.mail2web.com (Postfix) with ESMTPS id 4FHThf4VlLz129J for <quic@ietf.org>; Sat, 10 Apr 2021 02:09:22 -0700 (PDT)
Received: from [10.5.2.14] (helo=xmail04.myhosting.com) by xsmtp22.mail2web.com with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.92) (envelope-from <huitema@huitema.net>) id 1lV9cI-0000Ha-Fa for quic@ietf.org; Sat, 10 Apr 2021 02:09:22 -0700
Received: (qmail 6399 invoked from network); 10 Apr 2021 09:09:20 -0000
Received: from unknown (HELO [192.168.1.21]) (Authenticated-user:_huitema@huitema.net@[92.184.116.14]) (envelope-sender <huitema@huitema.net>) by xmail04.myhosting.com (qmail-ldap-1.03) with ESMTPA for <quic@ietf.org>; 10 Apr 2021 09:09:19 -0000
To: Mirja Kuehlewind <mirja.kuehlewind=40ericsson.com@dmarc.ietf.org>, IETF QUIC WG <quic@ietf.org>
References: <cbd1acfa-bfdd-0ce7-f381-ad87cacd85aa@huitema.net> <9BBE9A2F-8203-4467-BCAB-2C97B89F4371@ericsson.com>
From: Christian Huitema <huitema@huitema.net>
Subject: Re: Options for QUIC Multipath
Message-ID: <641bc79f-7839-77df-2f5f-01048d220840@huitema.net>
Date: Sat, 10 Apr 2021 02:09:18 -0700
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.9.0
MIME-Version: 1.0
In-Reply-To: <9BBE9A2F-8203-4467-BCAB-2C97B89F4371@ericsson.com>
Content-Type: text/plain; charset="utf-8"; format="flowed"
Content-Transfer-Encoding: quoted-printable
Content-Language: en-US
X-Originating-IP: 66.113.196.196
X-Spampanel-Domain: xsmtpout.mail2web.com
X-Spampanel-Username: 66.113.196.0/24
Authentication-Results: antispamcloud.com; auth=pass smtp.auth=66.113.196.0/24@xsmtpout.mail2web.com
X-Spampanel-Outgoing-Class: ham
X-Spampanel-Outgoing-Evidence: Combined (0.09)
X-Recommended-Action: accept
X-Filter-ID: Pt3MvcO5N4iKaDQ5O6lkdGlMVN6RH8bjRMzItlySaT9AUfYGofQVFgAXvvGE4a0aPUtbdvnXkggZ 3YnVId/Y5jcf0yeVQAvfjHznO7+bT5x6h2yQpzTslcOqazQkKtAFKj/EwzSHE5FGYwwjsNRPCKJd W+7jSzNTBG9/vlK5S3nmD6wdmZPcItWbGe10hXJtXL4FsauCVkDjmcYJdU3yWp7KuHNaaKdg7iBE ZefdsNUFWKwa/wzJUjmazeC7Imca+xnn3Ohf8XY5WZ+f/Kk3UxQ6V51u76v35b1wNe/MvdIN+Yj9 JT+HIE3AciYbXmyy2+J9PgaoF8SQHto3le4zsAApCVB1N/BtJyJqv7YkIyyKggeTQ85o+W6+jEZD z+LhiyQEs+dlGXUJLWZ+Gc08Nmllke3azHdKmySKNUVQl4ntlVxnbS8qIO7oudHyb2T1VQ58xe/l rqiRGalI3YPsxOTrFXToVyBmRCgQVX6zVyFUu8qzeMQP6uTHL0d9UjfY+eX5ZvcELCIKs663F/co VFYFvf25LVONYbYifH5OzZDcG6hsRQZiAIgw+z837AqgX7ewI8e1h7RITgN14BHmGVt/ReJ9Mfhz zmbKTH7wI9GEU1utNskUAORCV2WFZX0juPh8WNrp6UcEFtxxstnQGF7lLXQUcNAszDsnoUOr0Bhc PVV/C1Wq/zlXkdnKZwxkOI+gTB/pfSlbi1HgG7umZzYYs4qkxKLSV4C340uY5KqGbN7BITAZon7Z Iz1ONK9yUo4/+EUytKrR9Md9I2Rs14wElksq+mZMdxpTctQKrtFPCC/cRgvQKtcrMMueERx37Rkr iRC0KoCMs3KeauZR/iXDyPAgVgY7EKoR5dPqkGeIaIzNoZzswxuMaWjBAlpw1NZWZ+OtguoDDp0+ gck0GfUf9oDBqtClgM5jH/om1Q5UomG0v+rwIiID/kwKc8V5Tj9+FRkaOS/DNjANmb8tO61SbYdY AwdpaVzHW7wHO7YhEWyJzIkwSFAW0Pw8uiKeubcolFl/rX+2ReQklqJDASQX2Id+W5hjJNcdGs0+ iHjXODmj5PX/tZQU3bYnWKpb
X-Report-Abuse-To: spam@quarantine11.antispamcloud.com
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/UCqVVGvMMgtYREuGviEsZVg-zUw>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 10 Apr 2021 09:11:39 -0000
On 4/9/2021 6:36 AM, Mirja Kuehlewind wrote: > Hi Christian, hi all, > > reviving this thread. First of all thanks for implementing both options and writing the blog post! > > After re-reading you post and reviewing the drafts, however, I have to say that I’m more convinced now that multiple packet number spaces might be the better approach forward. I actually always thought of the packet number as a per-path property where the wire image for each path should be as much independent as possible. So that seem architecturally more clean to me. There are pros and cons with the "path property" approach. The main issue is that the definition of a path is not as obvious as it appears at first sight. We may think of a path as defined by the pair of sender and receiver addresses and ports, but this gets unclear if NATs are present. We end up with either a narrower definition using connection identifiers as in draft-liu, or avoiding the problem altogether by making packet numbers global. There are issues with both approaches, notably: * Support for 0-length connection identifiers in draft-liu * Need to maintain a path object with wider scope than the connection identifier in draft-liu, e.g. remembering congestion control, PMTU and RTT through a connection-ID renewal * Size of acknowledgement frames if using a global number approach The good news is that we are making progress on these issues, using a bit more sophistication. Ideas include: * Identifying paths through using the corresponding source ID if the DCID is zero length. This enables control per path such as "don't use this path anymore" even if the peer uses zero-length DCID. * If a peer uses zero length CID, use global path numbers when sending to that peer -- essentially a fusion of draft-liu and global numbers approach * Manage the size of ACK frames by limiting the number of times a given ACK range is repeated in ACKs. This is a very simple implementation recommendation that mitigates the growth of ACK frames when global numbers are used, including in the zero-length CID case. > > However, I’m really not convinced by your argument about implementation complexity below. First of all when we talk about implementation complexity, we should not only consider lines of code or number of tests or something like that but I think it is more important to assess the potential for implementation error. That is harder to assess but I think having a clean design and reduce the number of interdependencies is a factor. > > Further, implementation complexity should never be considered as a the sole metric. You actually convinced me in your blog post that what you call efficiency might be even more important because there are two aspects here: number of bits on the wire (for ACK frames that might have a lot of wholes) and amount of bits in local memory. My personal concern is the impact on code quality. If we end up with code branches that are rarely used, these branches will not be as well tested as the "main path", and bugs in these branches may surface later. The inverse correlation between complexity and security is well known. > With this conclusion I see draft-liu-multipath-quic as a really good starting point for future work (however, that so far my personal assessment). In both cases I support the approach to design a multipath extension that minimizes the changes needed from the base protocol. So reusing the connection ID and connection ID update mechanism is I think definitely the right approach to take. I am certainly willing to use draft-liu as a starting point. I would not be a co-author of that draft if I did not believe that. > I also think that any mechanism for address/path negotiation do not need to be part of the initial extension. In the most common scenario the client might just open a second path without further negotiation or coordination with the server when the interface/IP address of that new path come available. However, even if any negotiation is needed, this can be done on the application layer or added by another extension later on. We agree. > For draft-liu-multipath-quic I would even recommend to even move the part about scheduling and QoS support into the separate draft. I think QoS signal can definitely be a separate extension because that might even be useful without multiple paths (e.g. as input for congestion control). And for scheduling, I recommend to just specify some per-stream scheduling as the default behavior for now, but leave more complex schemes for future work (or research; scheduling doesn’t need standardization as it can be changed sender-side only). My co-authors have been doing an excellent work investigating scheduling issues in a multipath environment. The question that we want to answer is essentially, when is a multipath setup worse than a single path configuration, and how can we mitigate that? The main answer is that multipath does degrade performance if packets are scheduled on a lossy path and later need to be repeated on another path, creating additional delays. I would rather wait the next draft release for explaining mitigations, because it is a bit long for email. But describing such mitigations absolutely belongs in the multipath draft. > > So as soon as we could converge on the packet number question, I think we have a good starting to move on! > > Again thanks for your work and for the drafts! > > Mirja You are welcome. -- Christian Huitema > > > From: QUIC <quic-bounces@ietf.org> on behalf of Christian Huitema <huitema@huitema.net> > Date: Sunday, 14. February 2021 at 23:23 > To: IETF QUIC WG <quic@ietf.org> > Subject: Options for QUIC Multipath > > > I authored two drafts proposing two different solutions for Multipath QUIC: QUIC Multipath Negotiation Option (https://datatracker.ietf.org/doc/draft-huitema-quic-mpath-option/); and, in collaboration with colleagues at Ali Baba, Multipath Extension for QUIC (https://datatracker.ietf.org/doc/draft-liu-multipath-quic/). Apart from some details that could easily be aligned, the main difference is that the “negotiation option” maintains the property of QUIC Transport<https://datatracker.ietf.org/doc/draft-ietf-quic-transport/> to have a single packet number space for all application packets while the “multipath extension for QUIC” specifies that there will be a specific packet number space for each path. I have now implemented both options in Picoquic. This blog describes what I learned: https://huitema.wordpress.com/2021/02/14/how-many-packet-number-spaces-for-quic-multipath/. > > To summarize, I believe now that both options work. The simple option requires some additional work for managing acknowledgement, but the multiple number space option adds a lot more complexity (41 new code branches compared to only 6), and will require a lot more testing because it also change the processing of the "single path" scenarios. The multiple number space option also prevents the use of zero-length connection IDs, and thus causes additional overhead in some important deployment scenarios. So, yes, both options work, but the simpler option provides simpler code and also less overhead. > > In any case, I hope that this exercise will inform our efforts to standardize multipath support in QUIC. > > -- Christian Huitema > >
- Options for QUIC Multipath Christian Huitema
- Re: Options for QUIC Multipath Behcet Sarikaya
- Re: Options for QUIC Multipath Martin Duke
- Re: Options for QUIC Multipath Mirja Kuehlewind
- Re: Options for QUIC Multipath Christian Huitema
- Re: Options for QUIC Multipath Mirja Kuehlewind