Re: [v6ops] MAC table shortage in IPv6 networks caused by multiple IPv6 prefixes/addresses//FW: New Version Notification for draft-liu-v6ops-running-multiple-prefixes-01.txt

[Andrew Yourtchenko <[e-mail]>]

Hi Leo,

On Thu, 10 Jul 2014, Liubing (Leo) wrote:

> Hi Andrew,
>
>> I'll take the numbers below from the maximums of the graphs, so it is not
>> scientific at all but gives the ballpark:
>>
>> IPv4: 9.46k -> 9.35k
>> IPv6: (16.60k global + 7.73k link-local ) -> (6.76k global + 7.55k link-local)
>>
>> So, yeah there is a ~1.7x bloat in the neighbor table due to privacy addresses,
>> but that does not look too dramatic, does it ?
> [Bing] Would you like to share more details of the network? E.g. it is a dual-stack or something else.

It's dualstack, the vast majority of the clients are on a single wireless 
segment (the total number of wired devices is ~300-400, so I treat them as 
noise.

The wireless clients are addressed with SLAAC, and are outside of 
my control (conference attendees).

The segment has no officially designated servers.

The configuration from the first hop router is below:

interface Vlan4
  description !!! WIRELESS CLIENTS !!! CiscoLive2014 !!!
  .. IPv4 configuration omitted ..
  ipv6 address FE80::1 link-local
  ipv6 address 2001:4D38:A:400::1/64
  ipv6 nd reachable-time 1800000
  ipv6 nd prefix default 86400 9000 off-link no-autoconfig
  ipv6 nd prefix 2001:4D38:A:400::/64 604800 86400 off-link
  ipv6 nd prefix 2001:4D38:A:6800::/64 604800 0 off-link
  ipv6 nd router-preference High
  ipv6 nd ra lifetime 9000
  no ipv6 redirects
  no ipv6 unreachables
  ipv6 verify unicast source reachable-via rx

ipv6 route 2001:4D38:A:400::/64 vlan4

Note a few bits that are specific for the conference setup, and some are 
specific to the topology:

1) maximally high lifetimes (RAs are throttled on their way to the 
clients down to 1 RA in 10 minutes).

2) clearing on-link bit: results in clients never trying to resolve the 
other clients' addresses => less memory usage on the portable devices, 
smaller ND caches there.

3) default no-autoconfig: to force the necessity to add the prefixes 
manually, to force the operator to always have the "ipv6 nd prefix ..." 
with the correct config.

4) two prefixes, one with zero preferred lifetime: there was a second VLAN 
for disaster recovery purposes, where the clients would have ended up if 
there was a catastrophic failure no the main pair of the 
wireless controllers. That VLAN had a mirrored configuration. 
Experimentally, I've found such a configuration to cause a rapid 
deprecation of the "wrong" prefix on a iOS and OSX,
resulting in minimal disruption to them in case of switchover.

5) interface route for the prefix, to send the traffic to it (configuring 
the prefix "off-link" had removed the directly connected route).

> It is quite often that a host would have at least 3 IPv6 addresses: a 
>link-local, a global and a privacy address. If ULAs are enabled, 
>DHCPv6/SLAAC co-existing, there might be more IPv6 addresses.

Right. I was well within the constraints for the first hop, so opted to 
not have DHCPv6. If I wanted a tighter control, I'd have turned off SLAAC 
and used stateful DHCPv6. Maybe I'll experiment with it this year.

> So I was just guessing, maybe the 1.7x only represent that part of the nodes is enabling IPv6?

The %% of the nodes running IPv6 was about 80-85% 
(http://2014.ciscolive-ipv6.com/munin/ipv6noc/ipv6noc/neighbors_dualstack_percentage.html 
- the dips are the the night times, a lot of internal apps were IPv4 so 
the neighbor entries went away)

I calculated this %% by taking the "show arp" and "show ipv6 neigh" 
outputs and correlating the hosts by mac address.

As I said Apple devices (they are the most aggressive today wrt the 
privacy addresses) are ~50% of the network, so maybe in another setup the 
figures would be different.

Do you have some numbers you could share ?

> Plus, an IPv6-MAC binding would cost more cache space than an IPv4-MAC binding (2-4 times due to implementation).
>
>> I re-read the original mail starting this thread and the problem description
>> there to me boils down to two issues:
>>
>> 1) one IP node has more IP addresses in version 6 than in version 4.
>> 2) an IP address takes more lookup space in version 6 than in version 4.
>>
>> Do I grok it right ?
> [Bing] Basically yes. This is one of the problems cause by multiple 
> IPv6 addresses in draft-liu-v6ops-running-multiple-prefixes.

Ok, then I think that's a tricky one - we can not decrease the number of 
bits in IPv6 address, nor get the end hosts to go back to ARP for address 
resolution :-)

Also, if iOS 8.0 indeed changes the MAC address upon the wireless 
connection, the problem is just about to get much worse and not 
IPv6-specific ;-)

So, I think there are several means to combat the lookup space churn:

1) IPv6-only networks. We can't do that today and IPv4 is relatively not 
*so* much but it's something.

2) Table maintenance/cleanup mechanisms. These appear to work today 
reasonably.

3) provisioning hardware according to anticipated scale - and either using 
larger router boxes, or splitting large L3 domains into several smaller 
segments, like Mikael mentioned.

4) using DHCPv6-only allocation of addresses, with one address per host 
assignment.

Do you think there is something more that is needed besides the above 
approaches to steer the addresses on the host ?

--a