Re: [v6ops] [IPv6] MPTCP [was: New Version Notification for draft-fbnvv-v6ops-site-multihoming-01.txt]

[Vasilenko Eduard <[e-mail]>]

Hi Brian,
You are right about the strategic/big picture. MHMP solution is very needed.
I told already many times and even proposed the solution because the root cause is the choice of a next hop before a source address inside RFC 6724 (or getaddrinfo() that follow it). As soon as it would be reversed then RFC 8028 would be enough. The errata of RFC 8028 should become not an errata anymore. Simple to say, difficult to do.

But I was in the much smaller scope at the time of questions below: just "draft-fbnvv-v6ops-site-multihoming-01" for what is possible right now.
In this scope, my questions are relevant. People claim that MPTCP or MPQUIC are solutions *right now*. We need to clarify what to add to draft-fbnvv-v6ops-site-multihoming-02.

Eduard
-----Original Message-----
From: Brian E Carpenter [mailto:[e-mail]] 
Sent: Saturday, July 15, 2023 12:04 AM
To: Vasilenko Eduard <[e-mail]>; [e-mail]
Cc: [e-mail]; [e-mail]
Subject: Re: [IPv6] MPTCP [was: New Version Notification for draft-fbnvv-v6ops-site-multihoming-01.txt]

Eduard,
On 14-Jul-23 23:48, Vasilenko Eduard wrote:
> Hi Olivier,
> Thanks for my education!
> 
> The list of protocol implementations is good to have.
> But are there any production deployments?
> Do they tweak routing tables manually or use some scripts?
> Is it only for the host with directly connected Carriers or maybe for the network with many hosts behind the CPE/Router?

I don't think those are the right questions. As per Ole's and my previous messages, moving to multipath transport would be a strategic solution to MPMH and would need changes at many levels. So not really relevant to v6ops today! I think it would be good to separate the analysis into two parts:

1) What can be done today or in the near future for (potentially) thousands of sites?
2) What should be done strategically for (potentially) millions of sites?

For me, multipath transport is still in category 2.

    Brian

> 
> Eduard
> -----Original Message-----
> From: Olivier Bonaventure [mailto:[e-mail]]
> Sent: Friday, July 14, 2023 1:53 PM
> To: Vasilenko Eduard <[e-mail]>
> Cc: [e-mail]; [e-mail]
> Subject: Re: MPTCP [was: New Version Notification for 
> draft-fbnvv-v6ops-site-multihoming-01.txt]
> 
> Hello,
> 
>> Thanks for your participation. Please look below.
>>
>> 1. For the case when MHMP is transparent for the application, Do you 
>> agree to the statement below that it is not enough to make a proper configuration of MPTCP, it is needed additionally to tweak routing on the host.
> 
> Routing needs to ensure that a packet with source address X reaches a border router that is connected to the provider that allocated prefix X.
> 
>> Could we assume that "a smartphone vendor that monitors the quality of the wireless interfaces" is doing both?
> On a smartphone, this is a local decision, the stack uses the 
> interface corresponding to the source address
> 
>> How many such MPTCP or MPQUIC implementations you know?
> 
> Implementations can do this, this is not the main part of the discussion. The discussion should identify whether a protocol is capable of supporting a given scenario. If yes, implementations will adapt when there is a use case.
> 
>> 2. For the case when MHMP is implemented in the application, You did 
>> mention "an application can bind to a specific address to force the establishment using this address".
>> How many such applications do you know (without names, just number)? MPTCP or MPQUIC - does not matter.
> 
> The MPTCP deployments that I'm aware do this. This is true on 
> smartphones and hybrid access networks 
> https://arxiv.org/abs/1907.04570
> 
> For MPQUIC, here is the list of implementations 
> https://github.com/quicwg/multipath/wiki/QUIC-Implementations-with-mul
> tipath-support An interop test is planned at the next IETF
> 
>> 3. For the text to add to the draft, It is not urgent. -01 draft is already published (IETF 117 deadline was the last week). -02 is targeted for the next IETF (118).
>> Multipathing is the 1st big modification that is in the pipeline for -02.
> 
> 
> Happy to contribute in september
> 
> 
> Olivier
>> Eduard
>> -----Original Message-----
>> From: Vasilenko Eduard
>> Sent: Thursday, July 13, 2023 10:18 AM
>> To: 'Brian E Carpenter' <[e-mail]>; Mark Smith 
>> <[e-mail]>
>> Cc: [e-mail]; [e-mail]
>> Subject: RE: MPTCP [was: New Version Notification for 
>> draft-fbnvv-v6ops-site-multihoming-01.txt]
>>
>> Hi Brian, thanks for the references. MPTCP Wiki is helpful.
>>
>>> https://www.mptcp.dev/
>>> https://github.com/multipath-tcp/mptcp_net-next/wiki
>>
>> We could put aside the satiation when an application is binding source IP address - it is possible without MPTCP. Let's see what MPTCP could do by itself.
>> It is possible to ask 'fullmesh' for "multiple subflows for each pair 
>> of IP-addresses":
>> https://multipath-tcp.org/pmwiki.php/Users/ConfigureMPTCP
>> But then it is needed to manually configure relationships between 
>> source IP addresses and forwarding interfeces 
>> https://multipath-tcp.org/pmwiki.php/Users/ConfigureRouting
>> Of course, automation scripts are proposed on the same page (for some situations).
>> It is difficult to qualify as the solution - not many would be capable to do it on the client side, server side would probably do bind() anyway.
>>
>> Hence, it looks like MPTCP is not helping MHMP. It was not the goal despite that such a goal was possible to add.
>> All IETF drafts should be "as small as possible" and touch "the least possible number of topics" to have a chance for progress. MPTCP is already pretty big - the absence of MHMP on the agenda is predictable - it is a very big topic by itself.
>>
>> MHMP is still dependent on applications to encode RFC 6724 logic at least partially, because asking humans is possible only on the server side. The question of how many applications did it on the client side is very interesting. Any example?
>>
>> Eduard
>> -----Original Message-----
>> From: Brian E Carpenter [mailto:[e-mail]]
>> Sent: Thursday, July 13, 2023 4:40 AM
>> To: Vasilenko Eduard <[e-mail]>; Mark Smith 
>> <[e-mail]>
>> Cc: [e-mail]; [e-mail]
>> Subject: MPTCP [was: New Version Notification for 
>> draft-fbnvv-v6ops-site-multihoming-01.txt]
>>
>> Eduard,
>>
>> I think it might be wise to look at actual implementations of MPTCP before jumping to conclusions about how it obtains the list of source and destination addresses.
>>
>> https://www.mptcp.dev/
>> https://github.com/multipath-tcp/mptcp_net-next/wiki
>> https://obonaventure.github.io/mmtp-book/
>> https://pypi.org/project/mptcplib/0.1.2/
>>
>> Or maybe just get someone who knows about it, and QUIC multipath, to join this conversation?
>>
>> Regards
>>       Brian Carpenter
>>
>> On 11-Jul-23 18:57, Vasilenko Eduard wrote:
>>> Hi Brian,
>>> It is a theoretical "good wish". Practically, MPTCP RFC blocks this 
>>> possibility by the statement below, MPTCP in general has the goal to be transparent for the application.
>>> Hence, MPTCP is not capable in principle to solve any MHMP issues.
>>>
>>> I strongly suspect that QUIC real applications are not mangling with source IPv6 addresses - they use getaddrinfo() with the random next hop and source address selection.
>>> Albeit, QUIC implementation of multipathing assumes that the application should be developed specially (MP QUIC by design is not transparent for the application layer). Hence, QUIC is capable potentially to implement the logic for choosing the source address first (by bind()).
>>>
>>> Is anybody knows what the particular MP QUIC implementation is using for connection: getaddrinfo() or bind()?
>>> If any use bind() then it would resolve the most important MHMP 
>>> issues (not all - see section 5.2.1)
>>>
>>> Eduard
>>> -----Original Message-----
>>> From: Brian E Carpenter [mailto:[e-mail]]
>>> Sent: Tuesday, July 11, 2023 7:12 AM
>>> To: Vasilenko Eduard <[e-mail]>; Mark Smith 
>>> <[e-mail]>
>>> Cc: [e-mail]; [e-mail]
>>> Subject: Re: [IPv6] New Version Notification for 
>>> draft-fbnvv-v6ops-site-multihoming-01.txt
>>>
>>>
>>>
>>> Regards
>>>        Brian Carpenter
>>>
>>> On 10-Jul-23 19:27, Vasilenko Eduard wrote:
>>>> Hi Mark,
>>>> Thanks a lot for this message. I would discuss it with co-authors, but IMHO: we need to mention MPTCP and QUIC Multipath. Many may have the same illusion as you.
>>>>
>>>> I. Let's consider the situation when we have only one interface on the host. Multiple Carriers are connected upstream (on the CPE?).
>>>>
>>>> About what is chosen first - the next hop or the source address:
>>>> MPTCP (rfc6182) has strict in 2 places: " MPTCP MUST NOT be used for applications that request to bind to a specific address or interface, since such applications are making a deliberate choice of path in use ".
>>>> QUIC Multipath (draft-ietf-quic-multipath) has nothing about binding (or not binding) to a specific source address or interface.
>>>>
>>>> Hence, it looks like they both would probably use getaddrinfo() that would randomly choose the next hop and then randomly again source address (as explained in section 5.2 of draft-fbnvv-v6ops-site-multihoming) which does not help in multihoming, not at all.
>>>
>>> Multipath means: try all the available paths. If you have N source 
>>> addresses and M destination addresses, that means N*M paths are 
>>> available. The order of probing is of course a good question. I have 
>>> no knowledge of QUIC but there's a little about MPTCP (and SHIM6) in 
>>> this expired draft and some of its references:
>>> https://datatracker.ietf.org/doc/html/draft-naderi-ipv6-probing-01
>>>
>>>         Brian
>>>
>>>
>>>
>>>>
>>>> Do you know of any MPTCP or QUIC Multipath implementation that uses bind() initially (not getaddrinfo()) to reverse the logic (choose source address before the next hop)?
>>>> Then it would be a big deal that MUST be mentioned.
>>>>
>>>> Anyway, section 5.2 explains the situation properly for both cases (getaddrinfo() or bind()).
>>>> Just we need to mention that depending on what is chosen first for Multipathing solutions (next hop or source address) would drive the case to section 5.2.1 or 5.2.2.
>>>> Bind() (section 5.2.1) would have much more help to MHMP than getaddrinfo() (section 5.2.2).
>>>>
>>>> II. Let's consider the situation when we have many interfaces on the host. And at least one carrier is connected directly to the host (3GPP Modem?).
>>>>
>>>> Indeed, in this case, it does not matter what would be chosen first: the next hop or source address, getaddrinfo() would lead to the same result as Bind().
>>>> MHMP problem is resolved automatically. Does not matter whether MPTCP is used or not.
>>>> IMHO: we have lost the message in the draft that the host *directly connected* (by a separate interface) to at least one carrier has no MHMP problem in principle.
>>>>
>>>> Eduard
>>>> -----Original Message-----
>>>> From: Mark Smith [mailto:[e-mail]]
>>>> Sent: Monday, July 10, 2023 1:24 AM
>>>> To: Vasilenko Eduard <[e-mail]>
>>>> Cc: [e-mail]
>>>> Subject: Re: [IPv6] New Version Notification for 
>>>> draft-fbnvv-v6ops-site-multihoming-01.txt
>>>>
>>>> Hi,
>>>>
>>>> There's one multihoming solution that hasn't been mentioned in the draft that has been very widely deployed for both IPv4 and IPv6.
>>>>
>>>> Multipath TCP, deployed on Apple's IOS 7, and used by Siri.
>>>>
>>>> https://perso.uclouvain.be/olivier.bonaventure/blog/html/2013/09/18
>>>> /
>>>> m
>>>> p
>>>> tcp.html
>>>>
>>>> https://blog.apnic.net/2022/08/23/analyzing-mptcp-adoption-in-the-i
>>>> n
>>>> t
>>>> e
>>>> rnet/
>>>>
>>>> QUIC is gaining multipath capabilities as well.
>>>>
>>>> https://multipath-quic.org/
>>>>
>>>> The only issue left for a multi-homed network is to steer traffic out of the correct ISP connection based on source address so that the packets don't get dropped by BCP 38 filters. Source routing, segment routing or simply IP tunnelling from the first hop router directly to the correct ISP egress edge router via addition of an IP in IP or GRE header solves that.
>>>>
>>>> Regards,
>>>> Mark.
>>>>
>>>>
>>>>
>>>> On Fri, 7 Jul 2023 at 19:34, Vasilenko Eduard <vasilenko.eduard=[e-mail]> wrote:
>>>>>
>>>>> Hi IPv6 WG,
>>>>>
>>>>> We have heavily edited the draft between versions 00 and 01 (216 commits on github).
>>>>>
>>>>> Primarily, we have expressed the general WG opinion in different sections on "how bad NAT and NPT are", propagating the hate to the table of content, including mentioning that NPT is experimental.
>>>>> Yet, we have not deleted solutions that exist in the real world - like we or not.
>>>>>
>>>>> The section on another "not good" solution is added - unfortunately, "application proxy" is very popular in Enterprise that may address the draft subject.
>>>>> Again, we were trying to be very negative on it, yet balanced - it is part of the real Enterprise practice and needs to be overviewed.
>>>>>
>>>>> More places to mention PVD when applicable.
>>>>>
>>>>> Paolo Nero has discovered that Microsoft is capable to use "preference" from RA even if RFC 4191 is not claimed.
>>>>>
>>>>> And many other small changes addressing comments from Tim Winters, 
>>>>> Eric Vincke, Toerless Eckert, Lorenzo Colitti, Sheng Jiang, Erik Nygren, Fernando Gont, Jared Mauch, and Juliusz Chroboczek.
>>>>> Thanks for the feedback!
>>>>>
>>>>> Eduard
>>>>> -----Original Message-----
>>>>> From: [e-mail] [mailto:[e-mail]]
>>>>> Sent: Friday, July 7, 2023 11:58 AM
>>>>> To: Vasilenko Eduard <[e-mail]>; Vasilenko 
>>>>> Eduard <[e-mail]>; Klaus Frank 
>>>>> <[e-mail]>; Nick Buraglio 
>>>>> <[e-mail]>; Paolo Nero <[e-mail]>; 
>>>>> Paolo Volpato <[e-mail]>
>>>>> Subject: New Version Notification for 
>>>>> draft-fbnvv-v6ops-site-multihoming-01.txt
>>>>>
>>>>>
>>>>> A new version of I-D, draft-fbnvv-v6ops-site-multihoming-01.txt
>>>>> has been successfully submitted by Eduard Vasilenko and posted to the IETF repository.
>>>>>
>>>>> Name:           draft-fbnvv-v6ops-site-multihoming
>>>>> Revision:       01
>>>>> Title:          IPv6 Site connection to many Carriers
>>>>> Document date:  2023-07-07
>>>>> Group:          Individual Submission
>>>>> Pages:          38
>>>>> URL:            https://www.ietf.org/archive/id/draft-fbnvv-v6ops-site-multihoming-01.txt
>>>>> Status:         https://datatracker.ietf.org/doc/draft-fbnvv-v6ops-site-multihoming/
>>>>> Htmlized:       https://datatracker.ietf.org/doc/html/draft-fbnvv-v6ops-site-multihoming
>>>>> Diff:           https://author-tools.ietf.org/iddiff?url2=draft-fbnvv-v6ops-site-multihoming-01
>>>>>
>>>>> Abstract:
>>>>>        Carrier resilience is a typical business requirement. IPv4
>>>>>        deployments have traditionally solved this challenge through private
>>>>>        internal site addressing in combination with separate NAT engines
>>>>>        attached to multiple redundant carriers. IPv6 brings support for
>>>>>        true end-to-end connectivity on the Internet, and hence NAT is the
>>>>>        least desirable option in such deployments. Native IPv6 solutions
>>>>>        for carrier resiliency, however, have drawbacks. This document
>>>>>        discusses all currently-available options to organize carrier
>>>>>        resiliency for a site, their strengths and weaknesses, and provides
>>>>>        a history of past IETF efforts approaching the issue. The views
>>>>>        presented here are the summary of discussions on the v6ops mailing
>>>>>        list and are not just the personal opinion of the authors.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> The IETF Secretariat
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------
>>>>> -
>>>>> - IETF IPv6 working group mailing list [e-mail] 
>>>>> Administrative
>>>>> Requests: https://www.ietf.org/mailman/listinfo/ipv6
>>>>> ------------------------------------------------------------------
>>>>> -
>>>>> -
>>>>
>>>> -------------------------------------------------------------------
>>>> - IETF IPv6 working group mailing list [e-mail] Administrative
>>>> Requests: https://www.ietf.org/mailman/listinfo/ipv6
>>>> -------------------------------------------------------------------
>>>> -
>>>
> 
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [e-mail]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------