Re: [v6ops] [IPv6] MPTCP [was: New Version Notification for draft-fbnvv-v6ops-site-multihoming-01.txt]

[Brian Carpenter <[e-mail]>]

RFC 8028 section 2.2 observes that SADR is needed. But it isn't an
operational BCP so doesn't go into details.

(via tiny screen & keyboard)
Regards,
        Brian Carpenter

On Mon, 17 Jul 2023, 20:26 Ole Troan, <otroan=[e-mail]>
wrote:

> Vasilenko,
>
> > By "egress router" you probably mean "link router".
>
> No.
>
> > You probably imply various corner cases that are mostly related to
> "flush renumbering".
>
> No.
>
> I am thinking of any network topology more complicated than the hosts
> directly connected to the egress routers. 8028 only “solves” a corner case.
> Not the general problem.
> Which would require SADR, policy routing or an overlay.
>
> O.
>
> > I suspect that host-router state synchronization may be split into a
> different draft.
> > But I made a mistake attempting to resolve everything in one document:
> https://datatracker.ietf.org/doc/html/draft-vv-6man-nd-prefix-robustness-03
> > That did fail to get an interest.
> > Eduard
> > -----Original Message-----
> > From: ipv6 [mailto:[e-mail]] On Behalf Of Ole Troan
> > Sent: Monday, July 17, 2023 10:55 AM
> > To: Vasilenko Eduard <vasilenko.eduard=[e-mail]>
> > Cc: [e-mail]; [e-mail]; [e-mail]
> > Subject: Re: [IPv6] MPTCP [was: New Version Notification for
> draft-fbnvv-v6ops-site-multihoming-01.txt]
> >
> >> You are right about the strategic/big picture. MHMP solution is very
> needed.
> >> I told already many times and even proposed the solution because the
> root cause is the choice of a next hop before a source address inside RFC
> 6724 (or getaddrinfo() that follow it). As soon as it would be reversed
> then RFC 8028 would be enough. The errata of RFC 8028 should become not an
> errata anymore. Simple to say, difficult to do.
> >
> > RFC8028 is not enough. It fails when a host is not directly connected to
> the egress routers.
> > You can even do MHMP without 8028 if routers or coordinated.
> >
> > Let’s make sure any solution in this space solves the corner cases.
> >
> > O.
> >
> >
> >>
> >> But I was in the much smaller scope at the time of questions below:
> just "draft-fbnvv-v6ops-site-multihoming-01" for what is possible right now.
> >> In this scope, my questions are relevant. People claim that MPTCP or
> MPQUIC are solutions *right now*. We need to clarify what to add to
> draft-fbnvv-v6ops-site-multihoming-02.
> >>
> >> Eduard
> >> -----Original Message-----
> >> From: Brian E Carpenter [mailto:[e-mail]]
> >> Sent: Saturday, July 15, 2023 12:04 AM
> >> To: Vasilenko Eduard <[e-mail]>;
> >> [e-mail]
> >> Cc: [e-mail]; [e-mail]
> >> Subject: Re: [IPv6] MPTCP [was: New Version Notification for
> >> draft-fbnvv-v6ops-site-multihoming-01.txt]
> >>
> >> Eduard,
> >> On 14-Jul-23 23:48, Vasilenko Eduard wrote:
> >>> Hi Olivier,
> >>> Thanks for my education!
> >>>
> >>> The list of protocol implementations is good to have.
> >>> But are there any production deployments?
> >>> Do they tweak routing tables manually or use some scripts?
> >>> Is it only for the host with directly connected Carriers or maybe for
> the network with many hosts behind the CPE/Router?
> >>
> >> I don't think those are the right questions. As per Ole's and my
> previous messages, moving to multipath transport would be a strategic
> solution to MPMH and would need changes at many levels. So not really
> relevant to v6ops today! I think it would be good to separate the analysis
> into two parts:
> >>
> >> 1) What can be done today or in the near future for (potentially)
> thousands of sites?
> >> 2) What should be done strategically for (potentially) millions of
> sites?
> >>
> >> For me, multipath transport is still in category 2.
> >>
> >>   Brian
> >>
> >>>
> >>> Eduard
> >>> -----Original Message-----
> >>> From: Olivier Bonaventure [mailto:[e-mail]]
> >>> Sent: Friday, July 14, 2023 1:53 PM
> >>> To: Vasilenko Eduard <[e-mail]>
> >>> Cc: [e-mail]; [e-mail]
> >>> Subject: Re: MPTCP [was: New Version Notification for
> >>> draft-fbnvv-v6ops-site-multihoming-01.txt]
> >>>
> >>> Hello,
> >>>
> >>>> Thanks for your participation. Please look below.
> >>>>
> >>>> 1. For the case when MHMP is transparent for the application, Do you
> >>>> agree to the statement below that it is not enough to make a proper
> configuration of MPTCP, it is needed additionally to tweak routing on the
> host.
> >>>
> >>> Routing needs to ensure that a packet with source address X reaches a
> border router that is connected to the provider that allocated prefix X.
> >>>
> >>>> Could we assume that "a smartphone vendor that monitors the quality
> of the wireless interfaces" is doing both?
> >>> On a smartphone, this is a local decision, the stack uses the
> >>> interface corresponding to the source address
> >>>
> >>>> How many such MPTCP or MPQUIC implementations you know?
> >>>
> >>> Implementations can do this, this is not the main part of the
> discussion. The discussion should identify whether a protocol is capable of
> supporting a given scenario. If yes, implementations will adapt when there
> is a use case.
> >>>
> >>>> 2. For the case when MHMP is implemented in the application, You did
> >>>> mention "an application can bind to a specific address to force the
> establishment using this address".
> >>>> How many such applications do you know (without names, just number)?
> MPTCP or MPQUIC - does not matter.
> >>>
> >>> The MPTCP deployments that I'm aware do this. This is true on
> >>> smartphones and hybrid access networks
> >>> https://arxiv.org/abs/1907.04570
> >>>
> >>> For MPQUIC, here is the list of implementations
> >>> https://github.com/quicwg/multipath/wiki/QUIC-Implementations-with-mu
> >>> l tipath-support An interop test is planned at the next IETF
> >>>
> >>>> 3. For the text to add to the draft, It is not urgent. -01 draft is
> already published (IETF 117 deadline was the last week). -02 is targeted
> for the next IETF (118).
> >>>> Multipathing is the 1st big modification that is in the pipeline for
> -02.
> >>>
> >>>
> >>> Happy to contribute in september
> >>>
> >>>
> >>> Olivier
> >>>> Eduard
> >>>> -----Original Message-----
> >>>> From: Vasilenko Eduard
> >>>> Sent: Thursday, July 13, 2023 10:18 AM
> >>>> To: 'Brian E Carpenter' <[e-mail]>; Mark Smith
> >>>> <[e-mail]>
> >>>> Cc: [e-mail]; [e-mail]
> >>>> Subject: RE: MPTCP [was: New Version Notification for
> >>>> draft-fbnvv-v6ops-site-multihoming-01.txt]
> >>>>
> >>>> Hi Brian, thanks for the references. MPTCP Wiki is helpful.
> >>>>
> >>>>> https://www.mptcp.dev/
> >>>>> https://github.com/multipath-tcp/mptcp_net-next/wiki
> >>>>
> >>>> We could put aside the satiation when an application is binding
> source IP address - it is possible without MPTCP. Let's see what MPTCP
> could do by itself.
> >>>> It is possible to ask 'fullmesh' for "multiple subflows for each
> >>>> pair of IP-addresses":
> >>>> https://multipath-tcp.org/pmwiki.php/Users/ConfigureMPTCP
> >>>> But then it is needed to manually configure relationships between
> >>>> source IP addresses and forwarding interfeces
> >>>> https://multipath-tcp.org/pmwiki.php/Users/ConfigureRouting
> >>>> Of course, automation scripts are proposed on the same page (for some
> situations).
> >>>> It is difficult to qualify as the solution - not many would be
> capable to do it on the client side, server side would probably do bind()
> anyway.
> >>>>
> >>>> Hence, it looks like MPTCP is not helping MHMP. It was not the goal
> despite that such a goal was possible to add.
> >>>> All IETF drafts should be "as small as possible" and touch "the least
> possible number of topics" to have a chance for progress. MPTCP is already
> pretty big - the absence of MHMP on the agenda is predictable - it is a
> very big topic by itself.
> >>>>
> >>>> MHMP is still dependent on applications to encode RFC 6724 logic at
> least partially, because asking humans is possible only on the server side.
> The question of how many applications did it on the client side is very
> interesting. Any example?
> >>>>
> >>>> Eduard
> >>>> -----Original Message-----
> >>>> From: Brian E Carpenter [mailto:[e-mail]]
> >>>> Sent: Thursday, July 13, 2023 4:40 AM
> >>>> To: Vasilenko Eduard <[e-mail]>; Mark Smith
> >>>> <[e-mail]>
> >>>> Cc: [e-mail]; [e-mail]
> >>>> Subject: MPTCP [was: New Version Notification for
> >>>> draft-fbnvv-v6ops-site-multihoming-01.txt]
> >>>>
> >>>> Eduard,
> >>>>
> >>>> I think it might be wise to look at actual implementations of MPTCP
> before jumping to conclusions about how it obtains the list of source and
> destination addresses.
> >>>>
> >>>> https://www.mptcp.dev/
> >>>> https://github.com/multipath-tcp/mptcp_net-next/wiki
> >>>> https://obonaventure.github.io/mmtp-book/
> >>>> https://pypi.org/project/mptcplib/0.1.2/
> >>>>
> >>>> Or maybe just get someone who knows about it, and QUIC multipath, to
> join this conversation?
> >>>>
> >>>> Regards
> >>>>     Brian Carpenter
> >>>>
> >>>> On 11-Jul-23 18:57, Vasilenko Eduard wrote:
> >>>>> Hi Brian,
> >>>>> It is a theoretical "good wish". Practically, MPTCP RFC blocks this
> >>>>> possibility by the statement below, MPTCP in general has the goal to
> be transparent for the application.
> >>>>> Hence, MPTCP is not capable in principle to solve any MHMP issues.
> >>>>>
> >>>>> I strongly suspect that QUIC real applications are not mangling with
> source IPv6 addresses - they use getaddrinfo() with the random next hop and
> source address selection.
> >>>>> Albeit, QUIC implementation of multipathing assumes that the
> application should be developed specially (MP QUIC by design is not
> transparent for the application layer). Hence, QUIC is capable potentially
> to implement the logic for choosing the source address first (by bind()).
> >>>>>
> >>>>> Is anybody knows what the particular MP QUIC implementation is using
> for connection: getaddrinfo() or bind()?
> >>>>> If any use bind() then it would resolve the most important MHMP
> >>>>> issues (not all - see section 5.2.1)
> >>>>>
> >>>>> Eduard
> >>>>> -----Original Message-----
> >>>>> From: Brian E Carpenter [mailto:[e-mail]]
> >>>>> Sent: Tuesday, July 11, 2023 7:12 AM
> >>>>> To: Vasilenko Eduard <[e-mail]>; Mark Smith
> >>>>> <[e-mail]>
> >>>>> Cc: [e-mail]; [e-mail]
> >>>>> Subject: Re: [IPv6] New Version Notification for
> >>>>> draft-fbnvv-v6ops-site-multihoming-01.txt
> >>>>>
> >>>>>
> >>>>>
> >>>>> Regards
> >>>>>      Brian Carpenter
> >>>>>
> >>>>> On 10-Jul-23 19:27, Vasilenko Eduard wrote:
> >>>>>> Hi Mark,
> >>>>>> Thanks a lot for this message. I would discuss it with co-authors,
> but IMHO: we need to mention MPTCP and QUIC Multipath. Many may have the
> same illusion as you.
> >>>>>>
> >>>>>> I. Let's consider the situation when we have only one interface on
> the host. Multiple Carriers are connected upstream (on the CPE?).
> >>>>>>
> >>>>>> About what is chosen first - the next hop or the source address:
> >>>>>> MPTCP (rfc6182) has strict in 2 places: " MPTCP MUST NOT be used
> for applications that request to bind to a specific address or interface,
> since such applications are making a deliberate choice of path in use ".
> >>>>>> QUIC Multipath (draft-ietf-quic-multipath) has nothing about
> binding (or not binding) to a specific source address or interface.
> >>>>>>
> >>>>>> Hence, it looks like they both would probably use getaddrinfo()
> that would randomly choose the next hop and then randomly again source
> address (as explained in section 5.2 of draft-fbnvv-v6ops-site-multihoming)
> which does not help in multihoming, not at all.
> >>>>>
> >>>>> Multipath means: try all the available paths. If you have N source
> >>>>> addresses and M destination addresses, that means N*M paths are
> >>>>> available. The order of probing is of course a good question. I
> >>>>> have no knowledge of QUIC but there's a little about MPTCP (and
> >>>>> SHIM6) in this expired draft and some of its references:
> >>>>> https://datatracker.ietf.org/doc/html/draft-naderi-ipv6-probing-01
> >>>>>
> >>>>>       Brian
> >>>>>
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> Do you know of any MPTCP or QUIC Multipath implementation that uses
> bind() initially (not getaddrinfo()) to reverse the logic (choose source
> address before the next hop)?
> >>>>>> Then it would be a big deal that MUST be mentioned.
> >>>>>>
> >>>>>> Anyway, section 5.2 explains the situation properly for both cases
> (getaddrinfo() or bind()).
> >>>>>> Just we need to mention that depending on what is chosen first for
> Multipathing solutions (next hop or source address) would drive the case to
> section 5.2.1 or 5.2.2.
> >>>>>> Bind() (section 5.2.1) would have much more help to MHMP than
> getaddrinfo() (section 5.2.2).
> >>>>>>
> >>>>>> II. Let's consider the situation when we have many interfaces on
> the host. And at least one carrier is connected directly to the host (3GPP
> Modem?).
> >>>>>>
> >>>>>> Indeed, in this case, it does not matter what would be chosen
> first: the next hop or source address, getaddrinfo() would lead to the same
> result as Bind().
> >>>>>> MHMP problem is resolved automatically. Does not matter whether
> MPTCP is used or not.
> >>>>>> IMHO: we have lost the message in the draft that the host *directly
> connected* (by a separate interface) to at least one carrier has no MHMP
> problem in principle.
> >>>>>>
> >>>>>> Eduard
> >>>>>> -----Original Message-----
> >>>>>> From: Mark Smith [mailto:[e-mail]]
> >>>>>> Sent: Monday, July 10, 2023 1:24 AM
> >>>>>> To: Vasilenko Eduard <[e-mail]>
> >>>>>> Cc: [e-mail]
> >>>>>> Subject: Re: [IPv6] New Version Notification for
> >>>>>> draft-fbnvv-v6ops-site-multihoming-01.txt
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> There's one multihoming solution that hasn't been mentioned in the
> draft that has been very widely deployed for both IPv4 and IPv6.
> >>>>>>
> >>>>>> Multipath TCP, deployed on Apple's IOS 7, and used by Siri.
> >>>>>>
> >>>>>> https://perso.uclouvain.be/olivier.bonaventure/blog/html/2013/09/1
> >>>>>> 8
> >>>>>> /
> >>>>>> m
> >>>>>> p
> >>>>>> tcp.html
> >>>>>>
> >>>>>> https://blog.apnic.net/2022/08/23/analyzing-mptcp-adoption-in-the-
> >>>>>> i
> >>>>>> n
> >>>>>> t
> >>>>>> e
> >>>>>> rnet/
> >>>>>>
> >>>>>> QUIC is gaining multipath capabilities as well.
> >>>>>>
> >>>>>> https://multipath-quic.org/
> >>>>>>
> >>>>>> The only issue left for a multi-homed network is to steer traffic
> out of the correct ISP connection based on source address so that the
> packets don't get dropped by BCP 38 filters. Source routing, segment
> routing or simply IP tunnelling from the first hop router directly to the
> correct ISP egress edge router via addition of an IP in IP or GRE header
> solves that.
> >>>>>>
> >>>>>> Regards,
> >>>>>> Mark.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> On Fri, 7 Jul 2023 at 19:34, Vasilenko Eduard <vasilenko.eduard=
> [e-mail]> wrote:
> >>>>>>>
> >>>>>>> Hi IPv6 WG,
> >>>>>>>
> >>>>>>> We have heavily edited the draft between versions 00 and 01 (216
> commits on github).
> >>>>>>>
> >>>>>>> Primarily, we have expressed the general WG opinion in different
> sections on "how bad NAT and NPT are", propagating the hate to the table of
> content, including mentioning that NPT is experimental.
> >>>>>>> Yet, we have not deleted solutions that exist in the real world -
> like we or not.
> >>>>>>>
> >>>>>>> The section on another "not good" solution is added -
> unfortunately, "application proxy" is very popular in Enterprise that may
> address the draft subject.
> >>>>>>> Again, we were trying to be very negative on it, yet balanced - it
> is part of the real Enterprise practice and needs to be overviewed.
> >>>>>>>
> >>>>>>> More places to mention PVD when applicable.
> >>>>>>>
> >>>>>>> Paolo Nero has discovered that Microsoft is capable to use
> "preference" from RA even if RFC 4191 is not claimed.
> >>>>>>>
> >>>>>>> And many other small changes addressing comments from Tim
> >>>>>>> Winters, Eric Vincke, Toerless Eckert, Lorenzo Colitti, Sheng
> Jiang, Erik Nygren, Fernando Gont, Jared Mauch, and Juliusz Chroboczek.
> >>>>>>> Thanks for the feedback!
> >>>>>>>
> >>>>>>> Eduard
> >>>>>>> -----Original Message-----
> >>>>>>> From: [e-mail] [mailto:[e-mail]]
> >>>>>>> Sent: Friday, July 7, 2023 11:58 AM
> >>>>>>> To: Vasilenko Eduard <[e-mail]>; Vasilenko
> >>>>>>> Eduard <[e-mail]>; Klaus Frank
> >>>>>>> <[e-mail]>; Nick Buraglio
> >>>>>>> <[e-mail]>; Paolo Nero <[e-mail]>;
> >>>>>>> Paolo Volpato <[e-mail]>
> >>>>>>> Subject: New Version Notification for
> >>>>>>> draft-fbnvv-v6ops-site-multihoming-01.txt
> >>>>>>>
> >>>>>>>
> >>>>>>> A new version of I-D, draft-fbnvv-v6ops-site-multihoming-01.txt
> >>>>>>> has been successfully submitted by Eduard Vasilenko and posted to
> the IETF repository.
> >>>>>>>
> >>>>>>> Name:           draft-fbnvv-v6ops-site-multihoming
> >>>>>>> Revision:       01
> >>>>>>> Title:          IPv6 Site connection to many Carriers
> >>>>>>> Document date:  2023-07-07
> >>>>>>> Group:          Individual Submission
> >>>>>>> Pages:          38
> >>>>>>> URL:
> https://www.ietf.org/archive/id/draft-fbnvv-v6ops-site-multihoming-01.txt
> >>>>>>> Status:
> https://datatracker.ietf.org/doc/draft-fbnvv-v6ops-site-multihoming/
> >>>>>>> Htmlized:
> https://datatracker.ietf.org/doc/html/draft-fbnvv-v6ops-site-multihoming
> >>>>>>> Diff:
> https://author-tools.ietf.org/iddiff?url2=draft-fbnvv-v6ops-site-multihoming-01
> >>>>>>>
> >>>>>>> Abstract:
> >>>>>>>      Carrier resilience is a typical business requirement. IPv4
> >>>>>>>      deployments have traditionally solved this challenge through
> private
> >>>>>>>      internal site addressing in combination with separate NAT
> engines
> >>>>>>>      attached to multiple redundant carriers. IPv6 brings support
> for
> >>>>>>>      true end-to-end connectivity on the Internet, and hence NAT
> is the
> >>>>>>>      least desirable option in such deployments. Native IPv6
> solutions
> >>>>>>>      for carrier resiliency, however, have drawbacks. This document
> >>>>>>>      discusses all currently-available options to organize carrier
> >>>>>>>      resiliency for a site, their strengths and weaknesses, and
> provides
> >>>>>>>      a history of past IETF efforts approaching the issue. The
> views
> >>>>>>>      presented here are the summary of discussions on the v6ops
> mailing
> >>>>>>>      list and are not just the personal opinion of the authors.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> The IETF Secretariat
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> -----------------------------------------------------------------
> >>>>>>> -
> >>>>>>> -
> >>>>>>> - IETF IPv6 working group mailing list [e-mail]
> >>>>>>> Administrative
> >>>>>>> Requests: https://www.ietf.org/mailman/listinfo/ipv6
> >>>>>>> -----------------------------------------------------------------
> >>>>>>> -
> >>>>>>> -
> >>>>>>> -
> >>>>>>
> >>>>>> ------------------------------------------------------------------
> >>>>>> -
> >>>>>> - IETF IPv6 working group mailing list [e-mail]
> >>>>>> Administrative
> >>>>>> Requests: https://www.ietf.org/mailman/listinfo/ipv6
> >>>>>> ------------------------------------------------------------------
> >>>>>> -
> >>>>>> -
> >>>>>
> >>>
> >>> --------------------------------------------------------------------
> >>> IETF IPv6 working group mailing list
> >>> [e-mail]
> >>> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> >>> --------------------------------------------------------------------
> >> --------------------------------------------------------------------
> >> IETF IPv6 working group mailing list
> >> [e-mail]
> >> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> >> --------------------------------------------------------------------
> >>
> >
> >
> > --------------------------------------------------------------------
> > IETF IPv6 working group mailing list
> > [e-mail]
> > Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> > --------------------------------------------------------------------
> > --------------------------------------------------------------------
> > IETF IPv6 working group mailing list
> > [e-mail]
> > Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> > --------------------------------------------------------------------
>
>
> _______________________________________________
> v6ops mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/v6ops
>