Rules of Thumb Discussion at IETF

[Kent England <[e-mail]>]

	The Rules of Thumb sub-group of the Topology Engineering
Working Group is working on Rules of Thumb for network service
providers to use in managing routing policy at their borders.  This
draft is intended for discussion at the upcoming IETF TEWG session on
Wednesday morning.  Please take the time to read this before the TEWG
meeting and ROT discussion next Wednesday.

	We are particularly interested in your feedback on what is
controversial, what is missing, and what is wrong with this draft
document.  We also solicit additional participation in TEWG ROT, in
particular if you would like to contribute additional case studies.

	The poster is the author of this draft and takes full
responsibility for errors and omissions.

___________

		    Internet Border Patrol Manual

	Executive Summary:

	We begin by making some definitions:

Administrative Domain (or Autonomous System; which?)
Border
Interior Gateway Protocol
Exterior Gateway Protocol
Stub, Multi-homed, and Transit AD Borders (according to BGP definitions)
DMZnet

	We then give some guidelines:

Keep your IGP your own
Limit interfaces involved in the EGP process
Limit other-AD routers listened to
Maintain the list of your member networks
Restrict border routes into your IGP
Keep exterior routes exterior
Send default only to stub ADs
Send minimal information to multi-homed ADs
Reconstitute EGP metrics to interior metrics
Filter traffic to plug routing leaks
Centralize your routing database

	Then we propose some Border Case Studies:

A simple Stub border
A Multi-homed border with simple back door
A Multi-homed border with transit/fallback back door
A Sub-transit domain
Co-ordination Problems with multiple backbone borders

________________


		    Internet Border Patrol Manual
				  Or
    "How I Learned to Stop Worrying and Love Global Connectivity"

Managing routing information exchange among transit networks,
mid-level and backbone, and their subscribers is one of the more
challenging aspects of the networking business today.  In the interest
of propagating to a wider audience the guidelines we have come to
believe are more than ever necessary, we present some helpful hints to
good Internet hygiene.  This mini-manual is designed to give you, the
Internet NOCnik, some guidance in developing your own policies and
procedures and help in determining configurations of your routers to
accomplish your objectives.  If these guidelines are not only widely
accepted (as they already are), but widely implemented, then much of
the Internet will be susceptable to understanding, so far as routing
goes.  Widespread adherence to these guidelines will also make one of
the newly popular Internet parlor games called "traceroute" less
interesting and will thereby reduce the ICMP processing overhead on
all our routers.

Some Useful Definitions

While some of these definitions may seem trivial, it is important to
understand exactly how these terms are used in this document.  It is
hoped this usage is as consistent as possible with common Internet
usage.

It should be remembered that the point of view taken in this manual is
that of a transit network providing service to member sites.  A
typical case would be a mid-level NSFnet network providing transit
service to its member institutions with the backbone networks
providing transit service to the mid-level.

Administrative Domain

For the purposes of this document, an Administrative Domain (AD) is an
entity which has policy, technical, and operational control over a set
of routers which use one or more IGPs within the AD and use one or
more EGPs to exchange routing information across borders with routers
of other ADs.  An AD may be identified with one or more Autonomous
System Numbers (ASNs).  Typical current usage of the term "autonomous
system" is identical to our definition of AD and is preferred by some,
but it should be remembered that autonomous system and autonomous
system number bear no relationship to one another in today's Internet.

Border

A border is a connection between two ADs capable of carrying traffic
between the two domains.  Traffic exchange between two ADs requires
both a physical connection between routers from each domain and a
routing information exchange (EGP) session.  Sessions may be pairwise,
as with EGP2, or they may be broadcast.  Broadcast sessions can be
limited to a pair of routers with administrative controls.

The case where one AD connects to another AD which has no routers
(only a LAN) is considered a simple border.

Interior Gateway Protocol

An Interior Gateway Protocol is a routing protocol operating entirely
within a single AD.  A distinction is drawn between the protocol and
the processes which instantiate the protocol.  A transit network may
use one protocol (eg, ciscoIGRP), but multiple processes or instances
of ciscoIGRP, to manage interior and exterior routes within the AD.

Exterior Gateway Protocol

An Exterior Gateway Protocol is a routing protocol used to exchange
routing information across a border.  Candidate EGPs include RIP,
ciscoIGRP, and EGP2.  Again, the distinction is made between the
protocol and the processes which instantiate the protocol.

Note that the much vaunted topology restrictions of EGP2 do not apply
when EGP2 is used as an exterior routing exchange protocol between a
pair of border routers and the EGP2 reachability information is
redistributed into the IGP at the border independently of other border
EGP2 sessions.  The topology restrictions of EGP2 are effective only
when a single router has knowledge of more than one border EGP2
session.  Otherwise, EGP2 acts just like RIP, ciscoIGRP, or any other
protocol for the simple transfer of reachability information.  The
redistribution of EGP2 data into the IGP destroys all exterior metric
information and other source information (unfortunately) except for
border priority (primary, secondary, or tertiary) which is set when
the exterior routes are distributed into the IGP.

Stub, Multi-homed, and Transit AD Borders

A stub AD border connects a single AD with a single transit AD.  This
is the typical mid-level connection to a member site.  Multiple ADs
may be aggregated at a stub border and considered together to be a
single stub AD from the point of view of the transit AD.

A multi-homed AD has more than one border and more than one path to
one or more transit ADs.  A multi-homed AD may act as transit for
another AD, or may simply use the multiple connections for better
local connectivity without transit capability.

A transit AD is, obviously, for carrying traffic from one AD to
another.  One transit network may offer transit service to its
constituent stub ADs and be granted transit service in turn by other,
wider service area (or backbone) networks, such as NSFnet.  The
meaning of transit must be taken in context.  A mid-level is a transit
network for its constituents, but may be a stub AD to its backbone
network.

In fact, use of the terms stub, multi-homed, and transit should all be
used relative to the network perspective of this document.  All
borders are treated from the point of view of a particular transit
network unless otherwise stated.

DMZnet

A DMZnet is a small network shared among several border routers for
the purpose of acting as a common LAN connection in support of
multiple, independent border exchanges.  Often, a border router does
not require a DMZnet, using an interface on a member LAN as the
physical border connection.  However, some core routers share borders
with multiple ADs; stub, multi-homed, and transit.  A DMZnet is most
useful in these cases to assure that connectivity does not become a
casualty of inter-border routing warfare.

A DMZnet can also avoid the necessity of tunneling EGP sessions
through some intermediary AD that does not participate in that
particular exchange.

[DMZnet IGP of EGP managing third party?  Ask Ross how its done.]

Border Crossing Guidelines

Keep your IGP your own

Don't let your IGP cross any borders.  Inter-protocol exchanges are
usually much more configurable than firewalling within a single
protocol process in current implementations.  Even if your neighbor
runs the same IGP you do, use an EGP to exchange routing information.
If you simply send default across your border in the IGP of the stub
AD (usually RIP), while maintaining a separate process for your IGP,
then you have separated your IGP from the stub AD.  Unfortunately,
there is usually no way to run multiple instances of RIP.

Limit interfaces involved in the EGP process

Generally the border router only needs to talk the exterior protocol
on the border interface.  There is no need to listen or advertise on
any other interfaces.

Limit other-AD routers listened to

If the EGP is broadcast based, restrict the addresses of gateways from
which routes will be received to the minimal set of routers needed to
receive all registered networks.

Maintain the list of your member networks

Carefully define the set of networks for which your domain is acting
as transit.  Advertise this set consistently to other transit ADs.

Restrict border routes into your IGP

Filter the list of networks received to accept only networks
registered at this border.  Accept the minimum set of networks
reachable through this border.  Reject all other advertisements.

Make sure that the set of member networks as advertised to your
transit networks matches the set of routes accepted at all member
borders.  This is easiest to do if you maintain a separate routing
process for interior routes, or if you can tag exterior routes.

Keep exterior routes exterior

An administrative domain must discriminate between networks that are
"members" of that domain for purposes of transit, and networks that
are members of other transit domains for which this domain does not
act as transit.  Without such discrimination, routers have difficulty
keeping secrets.

The most difficult task for the routing policy manager today is
keeping one's routers from gossiping behind your back without having
to give them long, detailed, and explicit lists of things they should
and should not be talking about (lists of networks).  We need ways to
succinctly describe our routing policy in order to avoid the error
prone necessity for explicitly describing (the same) policy at each
and every border.

Of course, some degree of manual manipulation of lists of networks by
AD is necessary in order to create fine grained policy.  But, like
configuring network management displays, it should not take the
concentration of a Zen master to implement the most rudimentary
good-neighbor policies.

The most effective way to control redistribution of exterior routes is
to identify or tag the exterior routes when they enter your AD and
avoid redistributing these routes to other ADs, thereby keeping your
AD from becoming an inadvertent transit network.

Most IGPs do not have the ability to "tag" routes.  ciscoIGRP can be
made to effectively tag exterior routes by creating a pair of routing
processes, one with interior and one with exterior routes.  External
redistribution of routes can be controlled by only redistributing the
interior routing process routes and not redistributing the exterior
routing process.

Exterior route tagging has the beneficial effect of disallowing
listening to your own AD networks from some other AD.

The other way to control exterior routes is by using explicit
filtering of network advertisements at each and every border to
eliminate redistribution of exterior routes, while allowing interior
routes to be advertised.  This is very labor intensive in most cases
and prone to error.

Send default only to stub ADs

Advertisement of default to stub ADs is useful to minimize the
bandwidth consumed on low speed WAN links, to reduce router CPU
overhead, to speed convergence of the local IGP, and to minimize the
likelihood of route "leakage" from the "stub" domain at other borders,
unbeknownst to your transit network.  In any event, route advertising
is all or none.  If all routes are not sent to a stub AD, one might as
well send only a default route.

Send minimal information to multi-homed ADs

In many cases, your transit network is not the only game in town.
Some of your clientele will have connections of their own to other
ADs, including other transit ADs, in particular application-specific
networks like ESnet or NSN and "commercial" networks like Uunet.
These clients may want more sophisticated routing policies than stub
ADs require.

Multi-homed AD borders are complex and difficult to manage properly,
so limit the routes sent to multi-homed ADs and the routes accepted,
to the minimum set that are needed to make the explicit policies that
have been negotiated work.  For example, if this multi-homed AD is to
act as fallback transit to one other AD, accept routes from these two
AD member sets only and only send the routes of the other AD, plus
default.  This kind of conservative distribution will minimize leaks.

Reconstitute EGP metrics to interior metrics

Don't accept metrics in EGP advertisements into your IGP without
translation to a metric that is recognized inside your AD as either
primary, secondary, or tertiary for selecting among candidate borders.
In general, it is best to prefer borders for specific networks
globally within your AD, so make sure that the metrics of exterior
networks differ sufficiently in IGP metric space that local variations
in distance to borders does not cause local variation in borders
chosen for specific networks.

Filter traffic to plug routing leaks

Filtering traffic based on source addresses to match the access-list
of route advertisements can avoid problems with accidental leakage of
routing information from unauthorized borders, from promiscuous use of
default, and from bogus loose source routed datagrams.

Traffic filtering may be most practical at stub borders, where the WAN
link speed is low, the router is capable, and the list of source
addresses is small.

Traffic filtering is most warranted at multi-homed AD borders, where
transit and fallback policies are in effect, but here the traffic
volume and link speeds may be higher, the list of source addresses is
much larger and the load on the routers is greater.

Traffic filtering can result in loss of connectivity for networks that
are not registered properly and delays in connectivity for newly
connected networks.  Traffic filtering can also lead to black holes
instead of sub-optimal routing where there are configuration errors in
the Internet.

However, since most borders are stub borders and most border WAN links
are slow speed, the limitation of route distribution to default only
and traffic filtering would eliminate this large number of borders
from the set of borders to be watched.  This would be an
accomplishment of first order and would allow the Internet network
managers to concentrate on the multi-homed borders, the DMZnets and
other major interconnection points like the FIXs.

Configuration Management

Centralize your routing database and avoid the necessity of
configuring the same information more than once.  Your database should
be easy to compare to authoritative databases for verification.

____________

Border Case Studies

Following are some case studies and guidelines to suit these specific
instances of real and imagined borders.

Stub border

Send default only or a candidate default-network that the stub AD can
use to generate a local IGP default.  Filter networks received and
only accept networks that have been registered with your AD
management.  Incorporate these networks into your interior routing
process, separate from exterior routes.  Reconstitute the metrics on
received routes, so that you know that this border will be the primary
path for this AD.  Filter traffic if feasible.

Multi-homed border with simple back door

If the multi-homed AD is treated as a firewalled stub AD, it can use
private back doors for simple improvements of local connectivity and
can use the transit network border for fallback to the back door
(assuming the other partner in the simple back door has a path to this
transit network) without having to have special arrangements made with
this transit network.  Therefore, you should treat simple back doors
as you do stub borders and disallow transit across the multi- homed
AD.

Multi-homed border with transit/fallback back door

If the multi-homed AD wishes to have some transit or fallback
capability for its back door partner(s), then additional routes will
have to be sent and received at this border.  If the stub firewalling
rules are used on this larger set of networks, routing leakage and
other woes will be minimized.  Traffic filtering may prove valuable in
situations such as this to back-up the route filtering.  Metric
reconstitution will have to be co-ordinated to assure that the
primary/secondary paths are properly configured.

Sub-transit domain

Some transit networks provide service for other mini-transit networks
and may share multiple interconnection points with the mini-transit
network.  The mini- transit net may use either or both borders for any
and all networks within its domain.  A typical case might be a state
network with two connections to a mid- level network.  [This is Dave
O'Leary's problem.]

[How do we do this?]

Co-ordinate multiple backbone borders

This is easier said than done.  Co-ordination among multiple backbone
networks and many mid-level networks in routing policy management is
the greatest challenge to Internet Topology Engineering today.  The
challenge is increasing with new international interconnections and
the proliferation of commercial IP network service providers.

However, we must continue to try to improve Internet topology
engineering while waiting for IWG and ORWG to save us.  Experience has
shown that nirvana can sometimes be delayed and that Internet growth
can often outstrip the estimates of the most fervent disciples of
large-scale internetworking.

One thing that will help to co-ordinate routing among the transit
networks is for each transit AD to list its transit networks in order
of preference for routes.  For example, is CSnet preferred over NSFnet
for any networks reachable via CSnet?  Is the NSFnet used as the
backbone of last resort or are the mailbridges still the great default
swamp?

Another thing that will help co-ordinate routing is for each transit
AD to manage routing information filters according to common
authoritative lists of networks by AD.  Then the transit preferences
can be interpreted to the network level.

ADs could also specify configurations that must be implemented within
other ADs to accomplish specific policies.  For example, many
mid-level networks rely on Merit configurations of NSSs to set
priorities for multiple sets of networks.  It would be helpful if the
mid-levels would describe what configurations need to be in order to
make their backbone connection behave as described.

[There must be much more that could be described here.  Perhaps the
best thing is to create more case studies?]