Re: [v6ops] MPTCP [was: New Version Notification for draft-fbnvv-v6ops-site-multihoming-01.txt]

Hi Brian, thanks for the references. MPTCP Wiki is helpful.

> https://www.mptcp.dev/
> https://github.com/multipath-tcp/mptcp_net-next/wiki

We could put aside the satiation when an application is binding source IP address - it is possible without MPTCP. Let's see what MPTCP could do by itself.
It is possible to ask 'fullmesh' for "multiple subflows for each pair of IP-addresses": https://multipath-tcp.org/pmwiki.php/Users/ConfigureMPTCP
But then it is needed to manually configure relationships between source IP addresses and forwarding interfeces https://multipath-tcp.org/pmwiki.php/Users/ConfigureRouting
Of course, automation scripts are proposed on the same page (for some situations).
It is difficult to qualify as the solution - not many would be capable to do it on the client side, server side would probably do bind() anyway.

Hence, it looks like MPTCP is not helping MHMP. It was not the goal despite that such a goal was possible to add.
All IETF drafts should be "as small as possible" and touch "the least possible number of topics" to have a chance for progress. MPTCP is already pretty big - the absence of MHMP on the agenda is predictable - it is a very big topic by itself.

MHMP is still dependent on applications to encode RFC 6724 logic at least partially, because asking humans is possible only on the server side. The question of how many applications did it on the client side is very interesting. Any example?

Eduard
-----Original Message-----
From: Brian E Carpenter [mailto:brian.e.carpenter@gmail.com] 
Sent: Thursday, July 13, 2023 4:40 AM
To: Vasilenko Eduard <vasilenko.eduard@huawei.com>; Mark Smith <markzzzsmith@gmail.com>
Cc: 6man@ietf.org; v6ops@ietf.org
Subject: MPTCP [was: New Version Notification for draft-fbnvv-v6ops-site-multihoming-01.txt]

Eduard,

I think it might be wise to look at actual implementations of MPTCP before jumping to conclusions about how it obtains the list of source and destination addresses.

https://www.mptcp.dev/
https://github.com/multipath-tcp/mptcp_net-next/wiki
https://obonaventure.github.io/mmtp-book/
https://pypi.org/project/mptcplib/0.1.2/

Or maybe just get someone who knows about it, and QUIC multipath, to join this conversation?

Regards
    Brian Carpenter

On 11-Jul-23 18:57, Vasilenko Eduard wrote:
> Hi Brian,
> It is a theoretical "good wish". Practically, MPTCP RFC blocks this 
> possibility by the statement below, MPTCP in general has the goal to be transparent for the application.
> Hence, MPTCP is not capable in principle to solve any MHMP issues.
> 
> I strongly suspect that QUIC real applications are not mangling with source IPv6 addresses - they use getaddrinfo() with the random next hop and source address selection.
> Albeit, QUIC implementation of multipathing assumes that the application should be developed specially (MP QUIC by design is not transparent for the application layer). Hence, QUIC is capable potentially to implement the logic for choosing the source address first (by bind()).
> 
> Is anybody knows what the particular MP QUIC implementation is using for connection: getaddrinfo() or bind()?
> If any use bind() then it would resolve the most important MHMP issues 
> (not all - see section 5.2.1)
> 
> Eduard
> -----Original Message-----
> From: Brian E Carpenter [mailto:brian.e.carpenter@gmail.com]
> Sent: Tuesday, July 11, 2023 7:12 AM
> To: Vasilenko Eduard <vasilenko.eduard@huawei.com>; Mark Smith 
> <markzzzsmith@gmail.com>
> Cc: 6man@ietf.org; v6ops@ietf.org
> Subject: Re: [IPv6] New Version Notification for 
> draft-fbnvv-v6ops-site-multihoming-01.txt
> 
> 
> 
> Regards
>      Brian Carpenter
> 
> On 10-Jul-23 19:27, Vasilenko Eduard wrote:
>> Hi Mark,
>> Thanks a lot for this message. I would discuss it with co-authors, but IMHO: we need to mention MPTCP and QUIC Multipath. Many may have the same illusion as you.
>>
>> I. Let's consider the situation when we have only one interface on the host. Multiple Carriers are connected upstream (on the CPE?).
>>
>> About what is chosen first - the next hop or the source address:
>> MPTCP (rfc6182) has strict in 2 places: " MPTCP MUST NOT be used for applications that request to bind to a specific address or interface, since such applications are making a deliberate choice of path in use ".
>> QUIC Multipath (draft-ietf-quic-multipath) has nothing about binding (or not binding) to a specific source address or interface.
>>
>> Hence, it looks like they both would probably use getaddrinfo() that would randomly choose the next hop and then randomly again source address (as explained in section 5.2 of draft-fbnvv-v6ops-site-multihoming) which does not help in multihoming, not at all.
> 
> Multipath means: try all the available paths. If you have N source 
> addresses and M destination addresses, that means N*M paths are 
> available. The order of probing is of course a good question. I have 
> no knowledge of QUIC but there's a little about MPTCP (and SHIM6) in 
> this expired draft and some of its references: 
> https://datatracker.ietf.org/doc/html/draft-naderi-ipv6-probing-01
> 
>       Brian
> 
> 
> 
>>
>> Do you know of any MPTCP or QUIC Multipath implementation that uses bind() initially (not getaddrinfo()) to reverse the logic (choose source address before the next hop)?
>> Then it would be a big deal that MUST be mentioned.
>>
>> Anyway, section 5.2 explains the situation properly for both cases (getaddrinfo() or bind()).
>> Just we need to mention that depending on what is chosen first for Multipathing solutions (next hop or source address) would drive the case to section 5.2.1 or 5.2.2.
>> Bind() (section 5.2.1) would have much more help to MHMP than getaddrinfo() (section 5.2.2).
>>
>> II. Let's consider the situation when we have many interfaces on the host. And at least one carrier is connected directly to the host (3GPP Modem?).
>>
>> Indeed, in this case, it does not matter what would be chosen first: the next hop or source address, getaddrinfo() would lead to the same result as Bind().
>> MHMP problem is resolved automatically. Does not matter whether MPTCP is used or not.
>> IMHO: we have lost the message in the draft that the host *directly connected* (by a separate interface) to at least one carrier has no MHMP problem in principle.
>>
>> Eduard
>> -----Original Message-----
>> From: Mark Smith [mailto:markzzzsmith@gmail.com]
>> Sent: Monday, July 10, 2023 1:24 AM
>> To: Vasilenko Eduard <vasilenko.eduard@huawei.com>
>> Cc: 6man@ietf.org
>> Subject: Re: [IPv6] New Version Notification for 
>> draft-fbnvv-v6ops-site-multihoming-01.txt
>>
>> Hi,
>>
>> There's one multihoming solution that hasn't been mentioned in the draft that has been very widely deployed for both IPv4 and IPv6.
>>
>> Multipath TCP, deployed on Apple's IOS 7, and used by Siri.
>>
>> https://perso.uclouvain.be/olivier.bonaventure/blog/html/2013/09/18/m
>> p
>> tcp.html
>>
>> https://blog.apnic.net/2022/08/23/analyzing-mptcp-adoption-in-the-int
>> e
>> rnet/
>>
>> QUIC is gaining multipath capabilities as well.
>>
>> https://multipath-quic.org/
>>
>> The only issue left for a multi-homed network is to steer traffic out of the correct ISP connection based on source address so that the packets don't get dropped by BCP 38 filters. Source routing, segment routing or simply IP tunnelling from the first hop router directly to the correct ISP egress edge router via addition of an IP in IP or GRE header solves that.
>>
>> Regards,
>> Mark.
>>
>>
>>
>> On Fri, 7 Jul 2023 at 19:34, Vasilenko Eduard <vasilenko.eduard=40huawei.com@dmarc.ietf.org> wrote:
>>>
>>> Hi IPv6 WG,
>>>
>>> We have heavily edited the draft between versions 00 and 01 (216 commits on github).
>>>
>>> Primarily, we have expressed the general WG opinion in different sections on "how bad NAT and NPT are", propagating the hate to the table of content, including mentioning that NPT is experimental.
>>> Yet, we have not deleted solutions that exist in the real world - like we or not.
>>>
>>> The section on another "not good" solution is added - unfortunately, "application proxy" is very popular in Enterprise that may address the draft subject.
>>> Again, we were trying to be very negative on it, yet balanced - it is part of the real Enterprise practice and needs to be overviewed.
>>>
>>> More places to mention PVD when applicable.
>>>
>>> Paolo Nero has discovered that Microsoft is capable to use "preference" from RA even if RFC 4191 is not claimed.
>>>
>>> And many other small changes addressing comments from Tim Winters, 
>>> Eric Vincke, Toerless Eckert, Lorenzo Colitti, Sheng Jiang, Erik Nygren, Fernando Gont, Jared Mauch, and Juliusz Chroboczek.
>>> Thanks for the feedback!
>>>
>>> Eduard
>>> -----Original Message-----
>>> From: internet-drafts@ietf.org [mailto:internet-drafts@ietf.org]
>>> Sent: Friday, July 7, 2023 11:58 AM
>>> To: Vasilenko Eduard <vasilenko.eduard@huawei.com>; Vasilenko Eduard 
>>> <vasilenko.eduard@huawei.com>; Klaus Frank <klaus.frank@posteo.de>; 
>>> Nick Buraglio <buraglio@forwardingplane.net>; Paolo Nero 
>>> <oselists@gmail.com>; Paolo Volpato <paolo.volpato@huawei.com>
>>> Subject: New Version Notification for 
>>> draft-fbnvv-v6ops-site-multihoming-01.txt
>>>
>>>
>>> A new version of I-D, draft-fbnvv-v6ops-site-multihoming-01.txt
>>> has been successfully submitted by Eduard Vasilenko and posted to the IETF repository.
>>>
>>> Name:           draft-fbnvv-v6ops-site-multihoming
>>> Revision:       01
>>> Title:          IPv6 Site connection to many Carriers
>>> Document date:  2023-07-07
>>> Group:          Individual Submission
>>> Pages:          38
>>> URL:            https://www.ietf.org/archive/id/draft-fbnvv-v6ops-site-multihoming-01.txt
>>> Status:         https://datatracker.ietf.org/doc/draft-fbnvv-v6ops-site-multihoming/
>>> Htmlized:       https://datatracker.ietf.org/doc/html/draft-fbnvv-v6ops-site-multihoming
>>> Diff:           https://author-tools.ietf.org/iddiff?url2=draft-fbnvv-v6ops-site-multihoming-01
>>>
>>> Abstract:
>>>      Carrier resilience is a typical business requirement. IPv4
>>>      deployments have traditionally solved this challenge through private
>>>      internal site addressing in combination with separate NAT engines
>>>      attached to multiple redundant carriers. IPv6 brings support for
>>>      true end-to-end connectivity on the Internet, and hence NAT is the
>>>      least desirable option in such deployments. Native IPv6 solutions
>>>      for carrier resiliency, however, have drawbacks. This document
>>>      discusses all currently-available options to organize carrier
>>>      resiliency for a site, their strengths and weaknesses, and provides
>>>      a history of past IETF efforts approaching the issue. The views
>>>      presented here are the summary of discussions on the v6ops mailing
>>>      list and are not just the personal opinion of the authors.
>>>
>>>
>>>
>>>
>>> The IETF Secretariat
>>>
>>>
>>>
>>> --------------------------------------------------------------------
>>> IETF IPv6 working group mailing list ipv6@ietf.org Administrative 
>>> Requests: https://www.ietf.org/mailman/listinfo/ipv6
>>> --------------------------------------------------------------------
>>
>> --------------------------------------------------------------------
>> IETF IPv6 working group mailing list
>> ipv6@ietf.org
>> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
>> --------------------------------------------------------------------
> 