Re: [v6ops] I-D Action: draft-fbnvv-v6ops-site-multihoming-00.txt

[Vasilenko Eduard <[e-mail]>]

Hi Brian,
I do not see a difference between static and dynamic PA allocation.
In both cases deprecation for stale prefixes is mandatory.
Just in the case of "dynamic non-permanent" (the next prefix is different after re-connect) it should be deprecated forever.
But in the case of "dynamic permanent" or "static" (prefix would be the same after re-connect) it should be deprecated for hours till the fiber would be fixed.
Or else would be a black hole.
It is a very small difference (for how many hours to deprecate). Hence, I do not see a need for a separate section.

The question in MH context is not a "static" vs "dynamic". It is "permanent" vs "non-permanent" (the same or not after re-connect).
"Static" is always "permanent". "Dynamic" is "permanent" in 63% too. No difference at all.

PS: Comment that "static" prefixes case is covered is needed in the draft.
Eduard
-----Original Message-----
From: v6ops [mailto:[e-mail]] On Behalf Of Brian E Carpenter
Sent: Sunday, March 5, 2023 12:54 AM
To: Klaus Frank <[e-mail]>; [e-mail]
Subject: Re: [v6ops] I-D Action: draft-fbnvv-v6ops-site-multihoming-00.txt

Just a couple of extra comments where I was unclear in my first message:
On 05-Mar-23 01:55, Klaus Frank wrote:
> Hi
> 
> On 04.03.2023 04:09, Brian E Carpenter wrote:
>> Hi,
>>
>> Some comments (not really a full review):
>>
>> Firstly, thanks for starting this work, it is badly needed.
>>
>> Second, a bit off topic, but it was recently rediscovered where I 
>> live (New Zealand) that any amount of MHMP won't help you if your 
>> region is supplied by a single fibre trunk shared by all providers, 
>> and a landslide removes that trunk. It might be worth reminding 
>> readers of that. We really need MHMPMD (MD = multiple ducts).
> You could still get Starlink as a redundant uplink for example.
>>
>> One of the four solutions reads:
>>
>> "- Dynamic PA addresses distribution and withdrawal from carriers"
>>
>> I think that is really hiding a more traditional solution, something 
>> like:
>>
>> - Static PA addresses from multiple carriers, chosen dynamically
>>    by hosts.
> 
> No, thous two aren't the same. The first one is where the ISP also 
> occasionally assigns you a different one. So your PA prefix changes 
> dynamically. Or at least when your router reconnects. So it isn't a 
> static PA allocation at all.

I expressed myself badly. I think you should in fact list 5 solutions, with "Static PA addresses" before "Dynamic PA addresses".

Of course, even static prefixes are not static for ever, but for large enterprise networks a change of PA prefix would be a planned event
(RFC4192) and the MHMP problem is simpler.

I agree of course that the dynamic case must also be solved.

> 
>>
>> Dynamic prefix assignment and withdrawal is a *new* model.
>> Whether it can be made to work is an open question.
> We need it to work nonetheless, as providers often tier their services 
> to have dynamically allocating and therefore changing prefixes on the 
> cheaper contracts and often only offer static ones either at a higher 
> price at a very high price together with SD-WAN offerings...
>>
>> I'm ranting about this because whatever solution we propose must 
>> satisfy the needs of end hosts at least as well as it satisfies the 
>> needs of enterprise network operators and carriers. And that aspect 
>> isn't clear to me in the list that starts "4. Solution requirements".
>> (I'd personally prefer a solution that requires no new action 
>> whatever by the carriers.)
> 
> That's why we listed multiple. Our assumption was that Enterprises 
> will want to use their own PI prefix and either use a tunnel to a 
> centralized point (similar to LISP) or have SD-WAN contracts with the 
> ISP and be able to speak BGP with them to route the PI directly.
> 
> Where as smaller and home entities may either use a service provider 
> (LISP), dual assignment of all the prefixes if it's a flat hierarchy 
> "plug both ISP routers into the same local network", or use a
> (cheap/simple) load balance in front of the CPEs and do NAT/NPT on it.
> 
>>
>> One of the requirements:
>>
>> "4. Sub-second convergence for the prefix deprication..."
>>
>> comes out of nowhere. Won't this vary widely? In a small office 
>> network, why not 5 seconds or 30 seconds? In a stock trading data 
>> centre, wouldn't they want sub-millisecond convergence?
> The intent behind this was to stay within the usual timeout windows of 
> higher protocols to allow for a re-transmission without a connection 
> error being user-facing.
>> Also cover the case of redundant topologies (>1 router per subnet).
> 
> Outside of the scope of this RFC and where relevant already covered. 
> If you have two redundant routers on the subnet, then they'll route it 
> towards the CPEs at some point. And at that point it is the same as if 
> it was a client that sent the package. Also we mentioned the concerns 
> about complex networks and what approach would allow for static 
> subnetting and which ones don't.
> 
> The main issue this whole proposal is that ISPs don't offer BGP to 
> customers at an acceptable price point and that if everyone had their 
> own PI and would use BGP to the ISP that it would bloat the global 
> routing table. Another ideal world solution would be that IP addresses 
> are allocated by geographical area (and thereby allow route 
> aggregation using it) and not legal entity and a RPKI like approach 
> for the ISPs to assign IPs and allow customers to use any "RPKI-valid" 
> source prefix through their network and route it into the internet 
> even if it was assigned by another ISP.
> 
> But that's all make believe and we have to deal with the cards we were 
> given, that's also why we explicitly mentioned issues with cellular 
> and NAT66...
> 
>> "Privacy is furthermore assumed to be protected by [Temporary 
>> addressing] and is also kept outside of the carrier resiliency design 
>> consideration."
>>
>> I think that needs rewording. Some enterprises forbid temporary 
>> addresses, so the assumption is sometimes false. I think it would be 
>> safer to say that address-based privacy considerations are not 
>> affected by carrier resiliency in itself. (They might be affected by 
>> the choice to use or not use NAT66 or NPTv6, but that is a separate 
>> discussion.)
>>
>> "5.1. PI-based
>> ...
>> Advantages:"
>>
>> Isn't the #1 advantage: It works today! ?
> 
> Because it doesn't like it should. It only works in principal but has 
> huge issues in practice for most people...
> 
>> "Disadvantages:
>> ...
>> - Bloats the Internet routing table."
>>
>> How about
>>
>> - Bloats the Internet routing table with a potential 10 million new 
>> routes. ?
>>
>> I don't think we should make this solution sound satisfactory.
> 
> Well, that still wouldn't be a concern of the entity that wants to 
> have redundancy. And ultimately we don't really have any solution that 
> doesn't have issues right now.
> 
> The only real way to change this would be to completely rethink the 
> address assignment (see make believe above) and redo all of it, so 
> it's basically unavoidable/unsolvable at this point...
> 
>>
>> "5.2. PA-based solution
>> ...
>> It is mandatory to implement requirement L-13 of [HNCP] section 11 to 
>> deprecate delegated prefixes..."
>>
>> That reads as if you are proposing HNCP for everyone, which I am sure 
>> is not your intention. Can you rephrase it to sound like a property 
>> of the solution?
> 
> It already is, as it is listed within 5.2 and all sub sections of 5 
> are different individual solutions.

Again I expressed myself badly. I don't disagree technically, but the current text can mislead the reader to believe that you are suggesting HNCP as the solution. I'm suggesting that your text should be something
like:

"As soon as a CE router becomes aware that the path to the carrier that delegated a prefix is down, it is necessary that all more specific prefixes under that prefix are deprecated (preferred lifetime set to
zero) in all subnets. This is specified as requirement L-13 of [HNCP] but is not currently available in other routing mechanisms."

This leads directly to a requirement statement for the general case, where HNCP is not relevant.

     Brian

> And it is in fact mandatory to the solution 5.2. So if you want to use 
> the suggested PA-based solution you'll have to use it...
> 
>>
>> "Potentially, the application on the host may use bind() to choose 
>> the source address..."
>>
>> Yes, but most applications don't do that, they start with getaddrinfo().
>> So what is the value of this discussion of bind()?
> Because it is an issue with application compatibility and the whole 
> point of this proposal is to have a resilient up-link to not have 
> applications loose internet link and have a solution that does not 
> break applications. So this case sadly has to be considered as it is 
> affecting too many applications to just be ignored. But I admit that 
> we'd need better statistics for that to not just be based on 
> individual experience, but we currently don't have such statistics nor 
> a good way to get them...
>>
>> "It may be valuable to implement [Provisioning domains]..."
>>
>> What is the probability of that happening?
>>
>> "Disadvantages:
>> ...
>>     - Carriers may frequently change the prefix (flash renumbering), 
>> and
>>       this could disrupt communication"
>>
>> Why is that new? Many IPv6 carriers do that for SOHO users today. I 
>> think they don't do it for larger enterprises, though.
> It depends, I know from the top of my head of at least one enterprise 
> that had within the IPv4 world it's redundant links on the cheapest 
> contract available. So they'd do it there as well... And within the 
> IPv4 world they also didn't have any real reason to get a more 
> expensive offer, as they accomplished the same SLA level by just 
> having multiple redundant up-links and saved a non significant amount 
> of money compared to using the more expensive contracts that didn't 
> and because they used
> NAT44 the public IP change also didn't affect the internal network at 
> all. Even for the VPN connections between the sites they just used 
> dyndns and had the CPEs update the IP in DNS...
>>
>> "5.3. ULA with NPT
>> ...
>>     However, [ULA] has
>>     notable limits concerning the number of sites that one prefix may
>>     span, and well documented usability limitations when considering
>>     address selection details across large, diverse network
>>     environments."
>>
>> I'm not sure these limits are "well documented". If they are, please 
>> give a reference.
> * RFC6724
> * documentation of the major operating systems (Windows, MacOS, 
> Fedora, RedHat, ...)
> * documentation of the major firewall vendors (PaloAlto, Sophos, 
> Cisco, ...)
> * almost all it handbooks that talk about network fundamentals and 
> also include IPv6
> * countless blog posts with advice on how to configure IPv6
>> "5.4. ULA with NAT66
>> ...
>> Advantages:
>> ...
>> - NAT may be a normative requirement in itself (this is highly
>>       questionable, but nonetheless brought forward in many
>>       discussions)"
>>
>> Note that PCI-DSS (version 4, March 2022) does not require NAT, 
>> although it is mentioned as a solution for IPv4. See 
>> https://docs-prv.pcisecuritystandards.org/PCI%20DSS/Standard/PCI-DSS-
>> v4_0.pdf
> That's exactly what we were referring to by "this is highly 
> questionable, but nonetheless brought forward in many discussions", 
> even here on the IETF lists there were discussions not even two months 
> ago of working together with regulatory entities to avoid (and where 
> it exists
> currently) remove NAT66 requirements...
>>
>> " 6. Conclusion"
>>
>> I think in general this section is premature, until the earlier 
>> sections have been widely reviewed and updated. So mainly I will not comment, but:
>>
>> "For sites having a complex topology (many links and routers), a PA-
>>     based solution is not an option yet,..."
>>
>> Agreed, but:
>>
>> "... because it would need automatic
>>     PA address distribution over the site and neither [HNCP] nor 
>> DHCP-PD
>>     have gained market acceptance for this task."
>> That is needed for flash renumbering, but NOT for a static multi-PA 
>> solution. There, the requirement is much simpler: prefix up/down 
>> status distribution over the site. That's close to a trivial problem 
>> to solve, if we decide to solve it. (For example, it would be easy to 
>> define a GRASP [RFC8990] method for it, and I'm sure there are ten 
>> other solutions waiting.)
> 
> Right, we kinda neglected the part of having only static PA allocation 
> by the ISPs. Almost the whole draft is assuming that the PA prefix can 
> and will change without any upfront warning. But even if we for now 
> assume the PA allocated by every ISP is static, then we still have the 
> source address selection issue. As the ISPs still would filter the 
> packages that we push their way that have a Pa allocation of any of 
> the other ISPs in the mix... But I agree that in the case of static PA 
> allocations a customer could still do a full switch over to a 
> different link. It's only a active-passive setup. Still not ideal 
> (active-active with the ability for traffic engineering by the site 
> admin), but better than nothing for sure.
> 
> Side note, please don't take this too serious right now, it's not 
> fully worked out, just something I had to think about reading your 
> reply to
> this: What could help even in the case of multiple dynamic PA 
> allocations is if we'd have DHCP-PD solutions that could dynamically 
> update their configuration based on what the CPE provides (router 
> advertisement or DHCP-PD, also UPNPv6 could help if it was reliable 
> like it's IPv4 equivalent). But I don't think any network equipment 
> currently supports such a "feature" (but custom scripting on Linux may 
> work 🤔
> right now), but at the end that's just a worse implementation of what 
> we have right now. - Or is it? After all a solution that doesn't 
> involve the ISP or CPE is exactly what we need to unblock our selves 
> from these limitations...
> 
>>
>> Regards
>>     Brian Carpenter
> 
> Thanks for your review of our proposal.
> 
> Sincerely,
> Klaus Frank
> 
>>
>> On 04-Mar-23 03:59, [e-mail] wrote:
>>>
>>> A New Internet-Draft is available from the on-line Internet-Drafts 
>>> directories.
>>>
>>>
>>>           Title           : IPv6 Site connection to many Carriers
>>>           Authors         : Nick Buraglio
>>>                             Klaus Frank
>>>                             Paolo Nero
>>>                             Paolo Volpato
>>>                             Eduard Vasilenko
>>>     Filename        : draft-fbnvv-v6ops-site-multihoming-00.txt
>>>     Pages           : 32
>>>     Date            : 2023-03-03
>>>
>>> Abstract:
>>>      Carrier resilience is a typical business requirement. IPv4
>>>      deployments have traditionally solved this challenge through 
>>> private
>>>      internal site addressing in combination with separate NAT 
>>> engines
>>>      attached to multiple redundant carriers. IPv6 brings support 
>>> for
>>>      true end-to-end connectivity on the Internet, and hence NAT is 
>>> the
>>>      least desirable option in such deployments. Native IPv6 
>>> solutions
>>>      for carrier resiliency, however, have drawbacks. This document
>>>      discusses all currently-available options to organize carrier
>>>      resiliency for a site, their strengths and weaknesses, and 
>>> provides
>>>      a history of past IETF efforts approaching the issue. The views
>>>      presented here are the summary of discussions on the v6ops 
>>> mailing
>>>      list and are not just the personal opinion of the authors.
>>>
>>>
>>> The IETF datatracker status page for this Internet-Draft is:
>>> https://datatracker.ietf.org/doc/draft-fbnvv-v6ops-site-multihoming/
>>>
>>> There is also an htmlized version available at:
>>> https://datatracker.ietf.org/doc/html/draft-fbnvv-v6ops-site-multiho
>>> ming-00
>>>
>>>
>>>
>>> Internet-Drafts are also available by rsync at 
>>> rsync.ietf.org::internet-drafts
>>>
>>>
>>> _______________________________________________
>>> I-D-Announce mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/i-d-announce
>>> Internet-Draft directories: http://www.ietf.org/shadow.html or 
>>> ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>>
>> _______________________________________________
>> v6ops mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/v6ops
>>
>> _______________________________________________
>> v6ops mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/v6ops
_______________________________________________
v6ops mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/v6ops