Re: [RAM] Number of DFZ routers

Thanks Ted and others for your responses.  I recognise there
is no precise way of estimating these things.  Here are some
reasons I asked these questions.

Apart from the shortage of IPv4 addresses, I think all the
routing and addressing problems we are trying to solve are
due to the demands placed on DFZ routers.

Even the IPv4 shortage would not exist, or would be much
less of a problem, if these routers could happily handle
flat routing to single IP addresses.  Then there would be
no need for route aggregation and IPv4 address space could
be assigned in much smaller pieces, as needed, so it could
be used much more efficiently than at present.

That will never be practical, but I think it is practical to
make the FIBs of the next generation of routers do "flat
routing" to /24 granularity with less expense and trouble
than any current proposal such as LISP.  This only makes
sense is if BGP can cope with ten or so times the number of
advertised prefixes as today.  Also, I think LISP would have
many uses, including for TE and for multihoming to finer
granularity than /24.

So if BGP could be radically improved, RAM-based FIBs would
make the global routing system support finer-grained address
assignment with greater efficiency of use, while supporting
existing BGP multihoming practices to a granularity of 256
IP addresses.  This would reduce the size and scope of the
problems which remain to be solved by LISP etc.

Unfortunately, no BGP experts think that such improvements
are feasible.  Maybe some unconventional ideas from
non-experts might be helpful.

Here are my rough guesses about some aspects of the total
cost burden of the Internet's DFZ routers.

Firstly, there is a current number of routers in the DFZ.
With a proposal such as mine, this would continue to grow
for a long time, as more and more SPs and AS end-user
organisations connect to the Net - and as they use more
routers per AS, for instance at more and more sites.

Secondly, the FIB cost scales with the number of interfaces
and/or total bandwidth of each router, since there is
usually one FIB per interface, per four interfaces, or per
X Gigabit/sec bandwidth.  On a border router, not all of
these will be BGP interfaces, but on a multihomed border
router, I guess the FIB of each interface needs to handle
the routes for the BGP system.

Thirdly, each router needs RAM and CPU power to cope with
all the messages received from and sent to its BGP peers.
This scales directly with the number of interfaces using
BGP, and not with the speed of those interfaces.

Finally, once the router has booted and received and
processed all the routes from its BGP peers, including
transmitting its best routes to those peers, I think the
burden on each router also scales with:

1 - The rate of updates received from peers.  Average and
    peak would be very different.  All peak load messages
    need to be handled, but delays in handling them will
    slow local and global convergence times, perhaps
    making some kinds of inherent BGP instability patterns
    harder to manage.

2 - The rate of these updates which actually change the best
    route for a prefix for this router.  This scales roughly
    with (1), but for a router with dozens of BGP peers, I
    guess most of the updates wouldn't change the current
    best route, whereas for a router with only 3 peers, a
    larger proportion of updates would cause a change to the
    best route.

3 - The number of peers to send out updates to.

4 - The CPU time spent deciding whether the update results
    in a new best route.

5 - CPU time and especially any downtime for the FIBs as
    they are updated for the new best route.

Factor (1) is broadly proportional to the to total number of
advertised prefixes, but at present, a large proportion of
updates are probably the result of bad management, rather
than things the BGP network really should handle.  See
the middle of:

  http://www1.ietf.org/mail-archive/web/ram/current/msg01117.html

I wanted to get some guesstimates of total router numbers and
some actual case reports of transit routers and the number
of interfaces they have, the speeds of those interfaces etc.

As long as there are no solid proposals to improve the
capacity of BGP in general, or routers, in the future, I
think that the solutions we are considering are intended to
either reduce the demand for more DFZ routers, more
advertised prefixes etc. and/or are intended to work with
some administrative pressure to reduce these two figures.

I think any such pressure or limits will lead to long-term
problems with competition, since the limited number of
ISPs and AS end-users who connect to the DFZ will be like
a "top tier", with many others excluded.  (Unless perhaps
the ability to have a DFZ router and/or to advertise a
prefix is handled by a market-based system with tradeable
licenses to do so.  Then anyone can do it if they pay a
price which makes others happier to achieve their goals
some other way.)

All this is a lot of trouble to go to - all because we
can't figure out a way of improving routers' capacity to
handle BGP and/or how to resolve problems with poor
convergence in the BGP system.

One line of argument might be that the costs of these
proposed architectural changes is so high that we should
consider not making them, or at least postponing them or
reducing their scope, by the simple expedient of throwing
more money at every DFZ router.  (Maybe this is a good
time to buy the stock of companies which produce TCAM and
QDR-II SRAM.)

That is one reason I asked these questions - to get some
kind of idea on how many billions of dollars the current
DFZ router system costs.

It would be good to have a similar estimate of the number
of routers which would need to be upgraded for LISP,
depending on what volume of traffic was to be handled
by LISP in the future.  I think the ITR role is quite
demanding, especially in the FIB or whatever is used
to figure out if an incoming packet is handled by
one of a probably very large number of LISP prefixes.

 - Robin

_______________________________________________
RAM mailing list
RAM@iab.org
https://www1.ietf.org/mailman/listinfo/ram