RE: IPv6 Anycast has been killed by LINUX patch in 2016 - who cares?

[Vasilenko Eduard <[e-mail]>]

Hi Gyan,
Thanks. It is no the summary - It is a much bigger pack of information.
Especially this one is the direct answer to my original question:
NANOG link on TCP Anycast: https://archive.nanog.org/meetings/nanog37/presentations/matt.levine.pdf
As stated in the deck “don’t believe the FUD that TCP Anycast is not possible”
Eduard
From: ipv6 [mailto:[e-mail]] On Behalf Of Gyan Mishra
Sent: Tuesday, August 10, 2021 9:29 AM
To: Theodore Ts'o <[e-mail]>
Cc: Phillip Hallam-Baker <[e-mail]>; 6man WG <[e-mail]>; IETF discussion list <[e-mail]>
Subject: Re: IPv6 Anycast has been killed by LINUX patch in 2016 - who cares?


Forgive me for top posting - as this thread has forked into many different topics I try will hit a few of the topics including the subject heading that started the thread …here goes!!

Starting with the subject heading ..

“ Anycast has been killed by LINUX patch in 2016”

Tom Herbert  -> This is mid way through the thread in response to Brian and I guess basically this solves this hashing patch backs out  the 2016 patch making less aggressive hash so not recompute hash on each RTO but only after a threshold of RTOs - problem solved??

Tom Herbert-> “That's true, however we, Linux developers, certainly value the input and discussion on IETF lists. IMO, the later patch that started to recompute the hash on each RTO probably is too aggressive to be the default behavior on the Internet. I plan to post a patch that makes default less aggressive by restoring the original default behavior to recompute hash only after multiple RTOs. The rehash behavior will be configurable by sysctl and also I'll add a socket option to allow the application to control the behavior per connection thereby resolving Brian's concern”

Just so I and others understand - the main issue that sparked this thread is the RFC 7098 use of flow label for TCP application load balancing?

In the thread we transformed the topic into IPV6 flow label TCP Anycast load balancing, but the original issue was the Linux patch that changed the hashing for application IPv6 flow label hash for TCP server load balancing to aggressive each RTO which Tom is changing Default back to multiple RTOs?  Is that correct ??

RFC 7098 talks about the use of flow label for TCP flow server application load balancing.  Most load balancing vendors that I know of have their own variety of methods of load balancing such as round-Robin w/ w/o sticky Session option, source based load balancing and GEO load balancing based on an application Geo datatabase which I know of Citrix is which uses MEP Media Exchange protocol for DNS intercept, GSLB sub domain to be able to provide DNS reply from GEO application database based on client source address.

Few other points I will try to address related to ECMP load balancing as that topic came up in relation to RFC 6437 flow label.

Some general rules and history on L3 router ECMP IGP or EGP load balancing -: well as other data plane as well in relation to load balancing:

The concept of ECMP load balancing historically has been based on vendor specific proprietary algorithm ASIC or NPU based and had been flow or session based load balancing for IPv4 using basic source and destination IP  as input keys to a hashing function.

For any load balancing to work input key needs to be generated from information called keys within a packet done on a per packet basis, so that these keys within the packet can ensure that the packet gets hashed to the same path to avoid out of order packets. So the per packet input keys in the case of 5-tuple header hash is SA, DA, SP, DP, Protocol is used to generate a 20 bit “entropy” label as an input to a hashing function to provide uniform 50/50 load balancing.

 This basic “entropy” label  principle is applied to MPLS  data plane RFC 6790 SR data plane RFC 8662 and NVO3 VXLAN RFC 7348 source port entropy label same 5-tuple header hash key info generated random source port and now the same 5-tuple L3/L4 header hash keys generates the 20 bit flow label input key to a hashing function to provide uniform 50/50 load balancing.

Most all modern routers support RFC 6437 stateless uniform load balancing vendor links below of incumbent router / switch vendor stakeholders:

IPv6 flow label RFC 6437 stateless uniform 50/50 load balancing is one business driver for operators to migrate to SRv6.

Cisco XR
https://www.cisco.com/c/en/us/td/docs/routers/asr9000/software/asr9k-r6-3/ip-addresses/configuration/guide/b-ip-addresses-configuration-guide-asr9000-63x/b-ip-addresses-configuration-guide-asr9000-63x_chapter_0100.html

Juniper:
https://www.juniper.net/documentation/us/en/software/junos/sampling-forwarding-monitoring/topics/concept/hash-computation-mpcs-understanding.html

Nokia:
https://infocenter.nokia.com/public/7750SR215R1A/index.jsp?topic=%2Fcom.nokia.Gx_AVPs_Reference_Guide_21.5.R1%2Fdita_standards.html

Arista:
https://eos.arista.com/ipv6-rfc-compliance/

Huawei:
https://forum.huawei.com/enterprise/en/huawei/m/ViewThread.html?tid=605826&lang=en

BGP convergence metrics based on advertisement and withdrawals and BFD 50ms detection time and propagation times convergence within seconds:

NANOG 2013 link analysis on convergence times and route propagation benchmarks:

https://archive.nanog.org/sites/default/files/monday.general.huston.BGP.21.pdf

NANOG 2019 Analysis of internet traffic is 90% CDN traffic:

https://pc.nanog.org/static/published/meetings/NANOG76/1972/20190610_Labovitz_Internet_Traffic_2009-2019_v1.pdf

Anycast:

Anycast proximity based routing has been utilized by operators and enterprises for more almost 2 decades.

The initial uses of IPv4 Anycast was for DNS followed by NTP as well as other NMS monitoring services syslog, snmp, IPFIX.


RFC 2526 defines IPv6 Anycast which is the last  7 bits of the IID - /121-/128  Reserved  for IPv6 Anycast prefixes.

As far as I know I don’t know of any implementations or operators using the reserved Anycast range for IPv6 TCP Anycast.

I think most all IPv6 Anycast  NMS or Application deployments of Anycast have been using IPv4 style RFC 7094 proximity routing based IPv6 Anycast.

NANOG link on TCP Anycast:
https://archive.nanog.org/meetings/nanog37/presentations/matt.levine.pdf


As stated in the deck “don’t believe the FUD that TCP Anycast is not possible”

DDOS and Anycast:
Cloudfare CDN uses TCP Anycast and apparently had DDOS solution using Anycast.  Makes sense.

https://www.cloudflare.com/learning/cdn/glossary/anycast-network/

As it seems there are proven implementations and deployments of TCP Anycast both IPv4 application load balancing and IPv6 flow label RFC 6437 / RFC 7098 application load balancing.  My guess is that it works with some caveats as revealed in the NANOG archive.

I think the question came up in this thread related to L3 ECMP use of RFC 6437 for stateless uniform load balancing and also being able to use RFC 7098 application based  IPv6 GUA TCP or IPv6 Anycast TCP load balancing simultaneously.  I think that would work as the TCP load balancing is at L4 and L3 ECMP is at L3, and they both reference the per packet 5-tuple generated 20 bit key for both L3 and L4-7 load balancing.  I think what would not work in my mind is RFC 6437 stateful signaling from a edge host for flow label value as that would get overwritten by the per packet 5-tuple stateless 20 bit input key to the 50/50 load balancing hashing function.

We are thinking of using IPv6 flow label for APN edge marking, which would be stateful edge signaling from APN edge node,  however  that maybe difficult per above issue with stateless stepping on stateful.

Kind Regards

Gyan


On Mon, Aug 9, 2021 at 11:28 PM Theodore Ts'o <[e-mail]<mailto:[e-mail]>> wrote:
On Mon, Aug 09, 2021 at 10:04:45PM +0200, Robert Raszuk wrote:
> Hi Warren,
>
> There are dire proclamations that IPv6 TCP anycast cannot work....
> >
>
> Maybe I missed it but I do not recall anyone said this here.

How about the orignal message which kicked off this entire thread?
Including the hyperbolic subject line:

*** IPv6 Anycast has been killed by LINUX patch in 2016 - who cares? ***

                                          - Ted

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[e-mail]<mailto:[e-mail]>
Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
--------------------------------------------------------------------
--

[http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email]<http://www.verizon.com/>

Gyan Mishra

Network Solutions Architect

Email [e-mail]<mailto:[e-mail]>

M 301 502-1347