Re: 6MAN Adoption call on raft-chakrabarti-nordmark-6man-efficient-nd-04

[Andrew 👽 Yourtchenko <[e-mail]>]

Sander thanks!

all,

first off - apologies for the length of this mail, I tried to
structure it so that first I talk about what I think the
multicast-related issues are on the WiFi (what Sander asked), and then
add a few thoughts overall that I finally have managed to put on
paper. So feel free to doze off while reading, as you see fit :-)

There are two main issues related to "too much" multicast packets on WiFi:

1) Indeed the multicast frames are transmitted at a lower rate (and as
the SO entry tells, also the mileage varies)

That said, 1Mbps rate is becoming somewhat of a corner case - usually
those rates are disabled due to their general poor throughput and
realistically it will probably be 11Mbps rate, the highest for 802.11b
devices - or 12Mbps in case you do not support 802.11b-only clients
(most of the client gear today supports either 802.11g or 802.11n).

So this becomes somewhat less of a problem, though considering that
"WiFi guys" are generally reluctant to even advertise an extra SSID
with no clients (overhead being 10pps of broadcasted beacon frames at
low data rate), you will notice every PPS helps. Andrew von Nagy has a
great calculator telling exactly how much it helps, for the extra SSID
case, at [3]. But I think we can creatively equate it to each
additional 10pps of all-hosts multicast. As you see, it's noticeable.

2) The bigger problem - the battery life:

WiFi is a pretty expensive thing to do, power-wise. To help extend the
battery, the mobile devices try very hard to shut off the radio
whenever they can. The wireless station would go to sleep, and inform
the AP about this fact - so the AP can buffer the frames for it [1].
Further optimizations in this area were introduced by 802.11e (WMM-PS)
[2], where the frames destined to workstation could be transmitted
more efficiently and radio duty cycle is reduced.

Spontaneous all-hosts multicasts interfere with these mechanisms, thus
the battery drain on the devices is much higher - especially in a
typical network where the mobile devices move, and might trigger an
RS/RA on each L2 roam between BSSIDs.

So, if a host typically gets one packet burst per minute, and, say, it
decides to put the radio to sleep 15 seconds after no packets, it will
have 25% power consumption compared to the case where the same host
needs to process a 0.1pps (one packet in 10 seconds) of all-hosts
mcast.
So, that means if the gadget survives 8 hours in a quiet home network,
now it will die after 2 hours [4].

This issue especially manifesting itself in the high-density/event
wifi deployments, generally people are very upset when their gadgets
run out of juice.


All of the above said:

both of these issues are already somewhat taken care of by the tweaks
at the data-link layer: e.g. my employer's wifi gear will replicate
the RAs as unicast packets over the radio, and also if so configured,
it can both block the excessive multicast RA sent to all-hosts, as
well as track which clients were originating the RS, and prune the
unicast over-the-air delivery to only those hosts that have just sent
"unanswered" RS messages. (This is fixable much easier at the router
by sending solicited RAs unicast, but this is a different story).
Also, both ARP and ND are proxied to some extent, given that the
controller (a box that centrally handles the traffic from APs and
their configurations) has a very good awareness of the clients and can
judge about their addresses to some extent by their traffic.

For the roaming: any WiFi network with more than 1 access-points does
a lot of roaming stuff at L2 already, tries to do it as fast as
possible, and it's even standardized in 802.11r [6], so approaching it
with an L3-only hat on as if it is a brand new problem, is suboptimal
IMHO.

I hear someone saying: "Yes, but this is all about wireless, DC is all
different story - it's all switches, they don't hold the state!" - yes
and no. The hosts in the DC are increasingly VMs, which run on top of
the hypervisor - and is there something with *more* state related to
the host than the hypervisor running such a host ?

This is my concern with adding a relatively large chunk of a protocol
on top of the existing crutches: how much does it get us compared to
what we have already, and at what cost...

To summarize:

I do think the problems this draft talks about are real. But after a
*lot* of deliberation I think the solution should be more lightweight
and more incrementally implementable/deployable than what the draft
offers wholesale.

At a non-100% deployed rate the approach in the draft will add more
headaches to the administrators and users than help IMHO.


Now, while I have a mic, let me take my napkin with some sketches ... ;-)

Maybe worth to split the problem area into smaller subsets and see
whether a "cheaper" solutions are possible. Let's first take the
"multicast RAs" problem, and assume we do the below steps:

a) Allow the environments that wish to do so, to send solicited RAs
unicast. (standard already permits to do so).

b) Have the hosts restart the  Resilient RS [5] process at 1/2 or so
of router lifetime expiry: the router now can have the heuristic to
know which hosts are supporting the "unicast RA update" mechanism
*and* did not receive the periodic RA which would have been sent
usually at 1/3 the lifetime expiry (three RAs per lifetime rule of
thumb).

c) bump up the allowable MaxRtrAdvInterval and AdvDefaultLifetime to
their maximum theoretically possible values (spec says the hosts
should already be able to handle those, I did not test though) - this
would allow to further reduce the periodic RA frequency, from 2.5
hours today to ~18.2 hours, or, if we feel adventurous, put all-ones
to be a "solicited-only RAs by default" - thus, unless another router
sends an RA with different lifetime, refreshing the router info
becomes host-driven, with an option for a router to override at any
time.

These steps take care of the "multicast RA" problem. But again, also,
each of them by itself brings incremental usefulness, and is doable on
a single device => no "chicken and egg", they gradually shrink the
issue - and are compatible with the existing tweaks that solve the
same problem (in a more layer-violating way).

Now let's think of updating the various caches while roaming:

d) we could make a tweak to allow the solicited unicast RA from (a) to
contain an arbitrary blob created by the router, and require that the
"router refresh" (b) RSes from the host to contain this blob from the
router that they are currently using as the default router.

e) Request the host to send this "refresh" RS ahead of time if they
get a notification from a data-link layer that a link has changed.
AFAIK this is something that already happens today, at least for some
hosts.

f) Define the magic pixel dust that is put into a blob by a router,
such that the receiving router could un-dust this blob in the
subsequent "refresh" RS and do the necessary notifications/updates
(this protocol would be entirely at router-switch side and would not
need any more mods by hosts).

These three would help with quicker refresh of L3/L2 data while
roaming, and also each of them brings some incremental benefits of
various sorts, is doable on a single device, and can make use of the
lower-layer optimizations if necessary.

NB: the above does not take care of the "defending the host's address
on behalf of the sleeping node" problem. I think there is a good use
in splitting it as a separate function, because "sleeping node" can be
rather a "node on the very expensive bandwidth" - so something
developed here could also apply to e.g. 3G/LTE devices, instead of
today's diode firewalls - and would not necessarily be limited to
current segment, nor hard-coupled with the router function. But the
magic pixel dust from (f) above could supposedly be a building block
for such a protocol of for a part thereof.

NB2: the above is nothing more than a napkin-sketch of what I think
may work, I have not thought about the corner cases of it, etc. If the
community thinks it would be interesting to transfer the above from a
napkin onto a draft or drafts, I could do it.

If you made it to here - thanks for reading ! :-)

[1] http://en.wikipedia.org/wiki/Delivery_traffic_indication_message

[2] http://en.wikipedia.org/wiki/IEEE_802.11e-2005#APSD

[3] https://www.dropbox.com/s/imjrugrpyyscd7/Wi-Fi%20SSID%20Overhead%20Calculator.xlsx

[4] this is a SWAG. I do not have hard data, just the theoretical
knowledge of what is at play, and the empirical observations from
various networks. The activity to measure it is on my todo list.

[5] http://tools.ietf.org/html/draft-ietf-6man-resilient-rs-02

[6] http://www.cwnp.com/cmsAdmin/uploads/802-11_rsn_ft.pdf

--a


On 1/9/14, Sander Steffann <[e-mail]> wrote:
> Hi,
>
> Op 9 jan. 2014, om 14:33 heeft Lorenzo Colitti <[e-mail]> het
> volgende geschreven:
>
>> On Thu, Jan 9, 2014 at 9:44 PM, Mikael Abrahamsson <[e-mail]>
>> wrote:
>>
>> They would be rate-limited, just like any other ND packets on any
>> implementation worth its salt. I think an NS is 72 bytes including ICMPv6
>> header; if so, then even if you send out 500 of them per second, that's 36
>> kB/s. What sort of link has a problem carrying 36 kB/s? How many NSes per
>> second can routers send out anyway?
>>
>> I don't know, people seem to indicate that there is a problem with
>> multicast pps on wifi links. I haven't seen any hard data on this, it was
>> just my interpretation that this was considered a problem.
>>
>> Knowing nothing about this, I will cite http://stackoverflow.com/a/4341485
>> , hoping that the draft authors will come along with more detailed data
>> (which hopefully they have already presented to justify their use case and
>> which I missed):
>>
>> ===
>> With unicast the access point tracks what data rate every particular
>> receiver can reliably receive at, and sends about that rate. With
>> multicast the access point does not know which receivers are interested in
>> the data, so simple access points send the data at the slowest possible
>> rate (1Mb/s). Better implemented access points may send the data at the
>> rate that the slowest connected client is using, and the best access
>> points use IGMP snooping to see who's receiving each IP multicast stream,
>> and they will choose the slowest rate out of the receivers for that
>> stream.
>> ===
>>
>> Even 1Mbps is still a fair number of ND packets though.
>
> I think the issue is that, while 1Mbps is enough for ND, it is lowering the
> speed for the whole channel. So in the time that the 72 byte packet is sent
> at 1Mbps there *could* have been 3888 bytes at 54Mbps. And I have no idea if
> the speed switch itself takes up any time.
>
> I know that Andrew Yourtchenko has some real life experience with this
> stuff, so I'll put him in CC in case he has something to add.
>
> Cheers,
> Sander
>
>