Re: [v6ops] Dual ISP deployment operational issues and uncertainties

[Mark Andrews <[e-mail]>]

> On 31 Aug 2022, at 11:10, Klaus Frank <[e-mail]> wrote:
> 
> Hi Mark,
> 
> On 2022-08-31 01:27, Mark Andrews wrote:
>>> On 30 Aug 2022, at 13:47, Klaus Frank <[e-mail]> wrote:
>>> 
>>> Hi Brian ,
>>> 
>>> I see what you mean. However not using PI prefixes and using the two PA prefixes has a lot of issues Esp. when connect back in or if one needs to have static addresses (or DNS names) for some in house services like printers, NAS, servers, firewall, VPN gateways, ...
>>> 
>>> The main issues being:
>>> 
>>> a) That many devices only support specifying a single endpoint (VPNs, Printers, ...)
>> Then it is time to name and shame the products.  There is no reason other than laziness to not support multiple end points.
> 
> All products? I haven't seen any VPN gateway nor printer that supported multiple endpoint addresses. And even if the device requested multiple addresses the client connecting to it didn't.

Printers that support multiple transports support multiple endpoints.  Printers or any other device has been able to securely update the DNS for ~2 decades now.  Apple added code to do this on the Mac over decade ago when the addresses change (uses UPDATE + TSIG).  There really is no excuse that they don’t.  See RFC 2136 (DNS UPDATE) + RFC 2931 (DIG(0)) or RFC 2845 (TSIG).  We (ISC) have shipped client and server code (named (DNS server) and nsupdate (a DNS UPDATE client that supports both TSIG and SIG(0) for authenticating UPDATE request) that support these RFC’s since they where published.

Do we really have to write a RFC that says “Use RFC 2136 + RFC 2931 or RFC 2845 to update your address in the DNS”?

zone example.com. {
	…
	update-policy { grant * self *; };
};

And you would use a TSIG where the owner name matches the node name or a SIG(0) signed request where a preinstalled public KEY record at the node’s name is use to authenticate the UPDATE request by signing it with the private part of the key pair.  There are a whole suite of update policies available with the ability to extend them arbitrarily by talking to an external application over a socket.

Microsoft has been doing this with Active Directory for ages using GSS-TSIG as the authentication mechanism.

e.g.
	grant EXAMPLE.COM ms-self . ANY;

Samba clients update their address in the DNS using Kerberos as the authenticator

e.g.
	grant EXAMPLE.COM krb5-self . ANY;

In both these case EXAMPLE.COM is the REALM.  

The server can even use TCP source addresses to authenticate updates to reverse zones and limit the number of PTR records at a reverse name to 1 in this case.

zone 2.0.0.2.ip6.arpa {
	…
	update-policy {	grant * tcp-self . PTR(1); };
};

> One example with WireGuard. Even if you specify a dns name it only resolves it upon startup.

File a bug report.  Something that is long lived needs to recover.

> So in case of a failover it will just drop the connection and never try to establish a new connection instead of a silent reconnect. As I haven't tried this recently, this may even work if the IPs don't change (I.E. if both ISPs assign static IPv6-PA prefixes, but not if we also add dynamic/changing prefixes with dyndns updates).
> 
>>> b) A poor handling if a resource has multiple AAAA records for a single DNS name where one of the IPs is not available (Aka. a lot of failing connections)
>> Again it is time to name and shame the products. RFC 1123 was written in 1989 and it pointed out that servers could be multi-homed. It isn’t hard to do fast failover (Happy Eyeballs) over ALL THE ADDRESSES especially if you are making a TCP connection.  UDP is a little harder (as a DNS vendor we are often dealing with 20+ destination addresses with multiple address families over UDP and sometimes TCP and we can’t block waiting for a response over either transport). We have had non-blocking TCP stacks for decades. Nobody has ever said that you can’t try a second address while trying to connect to the first address.
>> 
>> If you want sample code for TCP see: https://users.isc.org/~marka/index.html
> 
> You're probably right here. Is this maybe mainly a library, documentation and "training" issue?
> 
> Do you know by change how different standard libraries behave when you throw in a DNS name with multiple IPs?

Unless you are using code from the last century the address lookup routines return multiple address.

> From what I've seen so far they only pick the first IP and try to connect to it. Or with Happy Eyeball pick the first IPv4 and the first IPv6.
> 
>>> So we're back at using a PI prefix (or NAT66) as the best option then...
>>> 
>>> Sincerely,
>>> Klaus Frank
>>> 
>>> On 2022-08-30 04:45, Brian E Carpenter wrote:
>>>> Klaus,
>>>> 
>>>> You (and Anoop) have discovered an issue that some people have been
>>>> concerned about for many years, and especially since the RIRs started
>>>> handing out /48 provider-independent prefixes like candy.
>>>> 
>>>> The estimate I made almost 15 years ago was that this could plausibly
>>>> lead to a "default free zone" of up to 10 million entries, based on
>>>> rough estimates of the number of small and medium enterprises in the
>>>> world, all of whom might choose to have redundant connectivity via
>>>> different ISPs.
>>>> 
>>>> The classical IPv6 answer is one that many operators don't like to hear:
>>>> don't use PI. In such a case, use two PA prefixes, one from each provider,
>>>> and run them both - each host will have addresses in two different
>>>> prefixes. Address selection policies will do the rest.
>>>> 
>>>> There are various issues with that approach, failover for existing
>>>> sessions being the worst, but I think the biggest issue is that it's
>>>> different from older practice. On the other hand, it means that neither
>>>> ISP has to deal with a PI prefix.
>>>> 
>>>> I'm sure other people will give other answers.
>>>> 
>>>> The real question is: when will this matter? Like climate change,
>>>> it has been easy to ignore it for the last 15 years.
>>>> 
>>>> Regards
>>>>    Brian Carpenter
>>>> 
>>>> On 30-Aug-22 13:38, Klaus Frank wrote:
>>>>> Hi,
>>>>> 
>>>>> from a twitter discussion I had with other admins I'd like to forward a
>>>>> problem to this mailing list.
>>>>> 
>>>>> When designing and deploying a redundant IPv6 network with two uplinks
>>>>> the current design is to use IPv6-PI space for the internal network. But
>>>>> when we would scale this approach to all businesses that have (or want
>>>>> to have) redundant uplinks it would pollute and fragment the global
>>>>> routing tables (making full table lookups harder).
>>>>> 
>>>>> Someone said that they consider using NAT66 for this. But as we all know
>>>>> NAT66 is exactly what we don't want with IPv6 ;-)
>>>>> 
>>>>> For client only networks we could just deprecate the prefix on
>>>>> switchover, but that would only allow a active-passive utilization while
>>>>> both are active. But for servers?
>>>>> 
>>>>> Also another issue is that at least here none of the standard contracts
>>>>> with ISPs contains BGP for announcing the IPv6-PI space. And only
>>>>> increasing the plan to an enterprise level with a way higher monthly fee
>>>>> is also an adoption blocker compared to just buying multiple of the
>>>>> cheaper lines with IPv4.
>>>>> 
>>>>> So therefore these questions arise:
>>>>> 
>>>>>    * Is there anything that admins can do right now (without an updated
>>>>>      or additional RFC) to solve this?
>>>>>    * Is doing a "full view" just something one shouldn't be doing anymore
>>>>>      with IPv6?
>>>>>    * Considering the above, is the best way to buy services from a
>>>>>      centralized anycast as a service provider and have a redundant
>>>>>      connection to them? (Which kinda would work against the reason why
>>>>>      we have people doing full view BGP in the first place. It also kinda
>>>>>      would introduce a new single point of failure into the High
>>>>>      Availability environment).
>>>>>    * Mobile IPv6 also looks kinda interesting in this context, but I
>>>>>      don't know of any suitable suitable real world deployment. Esp. not
>>>>>      over WAN.
>>>>>    * Am I just not seeing the forest for the trees?
>>>>> 
>>>>> Sincerely,
>>>>> Klaus Frank
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> v6ops mailing list
>>>>> [e-mail]
>>>>> https://www.ietf.org/mailman/listinfo/v6ops
>>> _______________________________________________
>>> v6ops mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/v6ops

-- 
Mark Andrews, ISC
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: [phone]              INTERNET: [e-mail]