Re: [v6ops] I-D Action: draft-fbnvv-v6ops-site-multihoming-00.txt

[Klaus Frank <[e-mail]>]

Hi

On 04.03.2023 04:09, Brian E Carpenter wrote:
> Hi,
>
> Some comments (not really a full review):
>
> Firstly, thanks for starting this work, it is badly needed.
>
> Second, a bit off topic, but it was recently rediscovered
> where I live (New Zealand) that any amount of MHMP won't help
> you if your region is supplied by a single fibre trunk shared
> by all providers, and a landslide removes that trunk. It might
> be worth reminding readers of that. We really need MHMPMD
> (MD = multiple ducts).
You could still get Starlink as a redundant uplink for example.
>
> One of the four solutions reads:
>
> "- Dynamic PA addresses distribution and withdrawal from carriers"
>
> I think that is really hiding a more traditional solution,
> something like:
>
> - Static PA addresses from multiple carriers, chosen dynamically
>   by hosts.

No, thous two aren't the same. The first one is where the ISP also 
occasionally assigns you a different one. So your PA prefix changes 
dynamically. Or at least when your router reconnects. So it isn't a 
static PA allocation at all.

>
> Dynamic prefix assignment and withdrawal is a *new* model.
> Whether it can be made to work is an open question.
We need it to work nonetheless, as providers often tier their services 
to have dynamically allocating and therefore changing prefixes on the 
cheaper contracts and often only offer static ones either at a higher 
price at a very high price together with SD-WAN offerings...
>
> I'm ranting about this because whatever solution we propose must
> satisfy the needs of end hosts at least as well as it satisfies the
> needs of enterprise network operators and carriers. And that aspect
> isn't clear to me in the list that starts "4. Solution requirements".
> (I'd personally prefer a solution that requires no new action
> whatever by the carriers.)

That's why we listed multiple. Our assumption was that Enterprises will 
want to use their own PI prefix and either use a tunnel to a centralized 
point (similar to LISP) or have SD-WAN contracts with the ISP and be 
able to speak BGP with them to route the PI directly.

Where as smaller and home entities may either use a service provider 
(LISP), dual assignment of all the prefixes if it's a flat hierarchy 
"plug both ISP routers into the same local network", or use a 
(cheap/simple) load balance in front of the CPEs and do NAT/NPT on it.

>
> One of the requirements:
>
> "4. Sub-second convergence for the prefix deprication..."
>
> comes out of nowhere. Won't this vary widely? In a small office
> network, why not 5 seconds or 30 seconds? In a stock trading
> data centre, wouldn't they want sub-millisecond convergence?
The intent behind this was to stay within the usual timeout windows of 
higher protocols to allow for a re-transmission without a connection 
error being user-facing.
> Also cover the case of redundant topologies (>1 router per subnet).

Outside of the scope of this RFC and where relevant already covered. If 
you have two redundant routers on the subnet, then they'll route it 
towards the CPEs at some point. And at that point it is the same as if 
it was a client that sent the package. Also we mentioned the concerns 
about complex networks and what approach would allow for static 
subnetting and which ones don't.

The main issue this whole proposal is that ISPs don't offer BGP to 
customers at an acceptable price point and that if everyone had their 
own PI and would use BGP to the ISP that it would bloat the global 
routing table. Another ideal world solution would be that IP addresses 
are allocated by geographical area (and thereby allow route aggregation 
using it) and not legal entity and a RPKI like approach for the ISPs to 
assign IPs and allow customers to use any "RPKI-valid" source prefix 
through their network and route it into the internet even if it was 
assigned by another ISP.

But that's all make believe and we have to deal with the cards we were 
given, that's also why we explicitly mentioned issues with cellular and 
NAT66...

> "Privacy is furthermore assumed to be protected by [Temporary
> addressing] and is also kept outside of the carrier resiliency
> design consideration."
>
> I think that needs rewording. Some enterprises forbid temporary
> addresses, so the assumption is sometimes false. I think it would
> be safer to say that address-based privacy considerations are
> not affected by carrier resiliency in itself. (They might be
> affected by the choice to use or not use NAT66 or NPTv6, but that
> is a separate discussion.)
>
> "5.1. PI-based
> ...
> Advantages:"
>
> Isn't the #1 advantage: It works today! ?

Because it doesn't like it should. It only works in principal but has 
huge issues in practice for most people...

> "Disadvantages:
> ...
> - Bloats the Internet routing table."
>
> How about
>
> - Bloats the Internet routing table with a potential 10 million new 
> routes. ?
>
> I don't think we should make this solution sound satisfactory.

Well, that still wouldn't be a concern of the entity that wants to have 
redundancy. And ultimately we don't really have any solution that 
doesn't have issues right now.

The only real way to change this would be to completely rethink the 
address assignment (see make believe above) and redo all of it, so it's 
basically unavoidable/unsolvable at this point...

>
> "5.2. PA-based solution
> ...
> It is mandatory to implement requirement L-13 of [HNCP] section 11
> to deprecate delegated prefixes..."
>
> That reads as if you are proposing HNCP for everyone, which I am sure
> is not your intention. Can you rephrase it to sound like a property
> of the solution?

It already is, as it is listed within 5.2 and all sub sections of 5 are 
different individual solutions.

And it is in fact mandatory to the solution 5.2. So if you want to use 
the suggested PA-based solution you'll have to use it...

>
> "Potentially, the application on the host may use bind() to choose
> the source address..."
>
> Yes, but most applications don't do that, they start with getaddrinfo().
> So what is the value of this discussion of bind()?
Because it is an issue with application compatibility and the whole 
point of this proposal is to have a resilient up-link to not have 
applications loose internet link and have a solution that does not break 
applications. So this case sadly has to be considered as it is affecting 
too many applications to just be ignored. But I admit that we'd need 
better statistics for that to not just be based on individual 
experience, but we currently don't have such statistics nor a good way 
to get them...
>
> "It may be valuable to implement [Provisioning domains]..."
>
> What is the probability of that happening?
>
> "Disadvantages:
> ...
>    - Carriers may frequently change the prefix (flash renumbering), and
>      this could disrupt communication"
>
> Why is that new? Many IPv6 carriers do that for SOHO users today. I think
> they don't do it for larger enterprises, though.
It depends, I know from the top of my head of at least one enterprise 
that had within the IPv4 world it's redundant links on the cheapest 
contract available. So they'd do it there as well... And within the IPv4 
world they also didn't have any real reason to get a more expensive 
offer, as they accomplished the same SLA level by just having multiple 
redundant up-links and saved a non significant amount of money compared 
to using the more expensive contracts that didn't and because they used 
NAT44 the public IP change also didn't affect the internal network at 
all. Even for the VPN connections between the sites they just used 
dyndns and had the CPEs update the IP in DNS...
>
> "5.3. ULA with NPT
> ...
>    However, [ULA] has
>    notable limits concerning the number of sites that one prefix may
>    span, and well documented usability limitations when considering
>    address selection details across large, diverse network
>    environments."
>
> I'm not sure these limits are "well documented". If they are, please
> give a reference.
* RFC6724
* documentation of the major operating systems (Windows, MacOS, Fedora, 
RedHat, ...)
* documentation of the major firewall vendors (PaloAlto, Sophos, Cisco, ...)
* almost all it handbooks that talk about network fundamentals and also 
include IPv6
* countless blog posts with advice on how to configure IPv6
> "5.4. ULA with NAT66
> ...
> Advantages:
> ...
> - NAT may be a normative requirement in itself (this is highly
>      questionable, but nonetheless brought forward in many
>      discussions)"
>
> Note that PCI-DSS (version 4, March 2022) does not require NAT, 
> although it is mentioned as a solution for IPv4. See 
> https://docs-prv.pcisecuritystandards.org/PCI%20DSS/Standard/PCI-DSS-v4_0.pdf
That's exactly what we were referring to by "this is highly 
questionable, but nonetheless brought forward in many discussions", even 
here on the IETF lists there were discussions not even two months ago of 
working together with regulatory entities to avoid (and where it exists 
currently) remove NAT66 requirements...
>
> " 6. Conclusion"
>
> I think in general this section is premature, until the earlier sections
> have been widely reviewed and updated. So mainly I will not comment, but:
>
> "For sites having a complex topology (many links and routers), a PA-
>    based solution is not an option yet,..."
>
> Agreed, but:
>
> "... because it would need automatic
>    PA address distribution over the site and neither [HNCP] nor DHCP-PD
>    have gained market acceptance for this task."
> That is needed for flash renumbering, but NOT for a static multi-PA
> solution. There, the requirement is much simpler: prefix up/down
> status distribution over the site. That's close to a trivial problem
> to solve, if we decide to solve it. (For example, it would be easy
> to define a GRASP [RFC8990] method for it, and I'm sure there are
> ten other solutions waiting.)

Right, we kinda neglected the part of having only static PA allocation 
by the ISPs. Almost the whole draft is assuming that the PA prefix can 
and will change without any upfront warning. But even if we for now 
assume the PA allocated by every ISP is static, then we still have the 
source address selection issue. As the ISPs still would filter the 
packages that we push their way that have a Pa allocation of any of the 
other ISPs in the mix... But I agree that in the case of static PA 
allocations a customer could still do a full switch over to a different 
link. It's only a active-passive setup. Still not ideal (active-active 
with the ability for traffic engineering by the site admin), but better 
than nothing for sure.

Side note, please don't take this too serious right now, it's not fully 
worked out, just something I had to think about reading your reply to 
this: What could help even in the case of multiple dynamic PA 
allocations is if we'd have DHCP-PD solutions that could dynamically 
update their configuration based on what the CPE provides (router 
advertisement or DHCP-PD, also UPNPv6 could help if it was reliable like 
it's IPv4 equivalent). But I don't think any network equipment currently 
supports such a "feature" (but custom scripting on Linux may work 🤔 
right now), but at the end that's just a worse implementation of what we 
have right now. - Or is it? After all a solution that doesn't involve 
the ISP or CPE is exactly what we need to unblock our selves from these 
limitations...

>
> Regards
>    Brian Carpenter

Thanks for your review of our proposal.

Sincerely,
Klaus Frank

>
> On 04-Mar-23 03:59, [e-mail] wrote:
>>
>> A New Internet-Draft is available from the on-line Internet-Drafts 
>> directories.
>>
>>
>>          Title           : IPv6 Site connection to many Carriers
>>          Authors         : Nick Buraglio
>>                            Klaus Frank
>>                            Paolo Nero
>>                            Paolo Volpato
>>                            Eduard Vasilenko
>>    Filename        : draft-fbnvv-v6ops-site-multihoming-00.txt
>>    Pages           : 32
>>    Date            : 2023-03-03
>>
>> Abstract:
>>     Carrier resilience is a typical business requirement. IPv4
>>     deployments have traditionally solved this challenge through private
>>     internal site addressing in combination with separate NAT engines
>>     attached to multiple redundant carriers. IPv6 brings support for
>>     true end-to-end connectivity on the Internet, and hence NAT is the
>>     least desirable option in such deployments. Native IPv6 solutions
>>     for carrier resiliency, however, have drawbacks. This document
>>     discusses all currently-available options to organize carrier
>>     resiliency for a site, their strengths and weaknesses, and provides
>>     a history of past IETF efforts approaching the issue. The views
>>     presented here are the summary of discussions on the v6ops mailing
>>     list and are not just the personal opinion of the authors.
>>
>>
>> The IETF datatracker status page for this Internet-Draft is:
>> https://datatracker.ietf.org/doc/draft-fbnvv-v6ops-site-multihoming/
>>
>> There is also an htmlized version available at:
>> https://datatracker.ietf.org/doc/html/draft-fbnvv-v6ops-site-multihoming-00 
>>
>>
>>
>> Internet-Drafts are also available by rsync at 
>> rsync.ietf.org::internet-drafts
>>
>>
>> _______________________________________________
>> I-D-Announce mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/i-d-announce
>> Internet-Draft directories: http://www.ietf.org/shadow.html
>> or ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>
> _______________________________________________
> v6ops mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/v6ops