comments

[[e-mail]]

Hello Yakov,

Thank you for sending detailed comments on the document requesting
Proposed Standard status for IDPR.  I apologize for being so late in
responding to your questions.  Please do not take it personally.  I
was travelling for the greater part of two weeks, my office was moved
last week, and in the middle of it all I caught a bad cold.  In fact,
today is the first day this week that I have had a functioning
computer on my desk.

Below, I have supplied some answers to you questions, which I
hope will make things a little clearer.  Your original comments are
preceded with ">" marks, in order to distinguish them.

> Do you think that design of a new inter-domain routing protocol can,
> by itself, justify creation of a new network layer protocol ?

IDPR was designed to support policy-based routing.  It is not a single
"inter-domain routing protocol" but rather a suite of protocols which
together provide policy routes and message forwarding along those
routes.

I believe that traffic sources that wish to use policy-based routing
will also wish to ensure (in the absence of malicious intervention)
that their traffic travels over the policy routes they select.
Hop-by-hop message forwarding does not provide this assurance;
source-specified message forwarding is the only way to gain the
forwarding control required for true policy-based routing.  So the
answer to your question is yes, I do think that to provide
policy-based routing, we need a network layer protocol that provides
efficient source-specified forwarding.

However, the wording "a new network-layer protocol" may be a bit
of a jolt for most people, and so I have removed it.

> If the answer is yes, then we either:

> (a) have to assume that IDPR will be THE inter-domain routing protocol, or
> (b) assume that each inter-domain routing protocol may come up with its
>     own network layer that needs not be compatible with network layers
> 	  introduced by other inter-domain routing protocols.

> Which of these choices do you think are likely to happen in practice ?

I do not believe that IDPR will be the only inter-domain routing
scheme.  I have now added to the document a section that briefly
describes how IDPR can coexist with other inter-domain routing
procedures.

While it is possible that each new inter-domain routing scheme may
choose its own new network layer forwarding mechanism, I don't believe
that will be the case.  One can break down message forwarding into two
generic classes: hop-by-hop and source-specified.  An efficient
hop-by-hop mechanism exists, namely IP.  However, there was no such
mechanism available for efficient source-specified forwarding (IP
source routing is not efficient for forwarding large numbers of data
messages).  Hence, we had to develop such a mechanism for use with
IDPR.

> Section 1.

> There is a notion of transit and stub domains. How do you define a "backbone"
> domain.  Even today we have well above 10 transit domains (likely on the
> order of at least 30).  Do you expect that by 1997 (1992 + 5 years) the
> global world-wide internet will have AT MOST 99 transit domains ? (99 is as
> far as I can stretch O(10)).

Backbone domain is not synonymous with transit domain.  All backbone
domains are transit domains, but not all transit domains are backbone
domains.  While I did not provide a crisp definition of backbone, I
did say that backbone domains are those that provide transit service
for large numbers of source and destination domains.  I assumed,
perhaps incorrectly, that the readers were familiar with the current
backbone, regional, and campus hierarchy of the current Internet.  For
example, I would classify NSFNet as a backbone domain, NEARNet as a
regional domain, and BBN as a campus domain.  Yes, I do expect O(10)
backbone domains over the next five years, but I expect perhaps
O(1000) transit domains.

> Does that mean to imply that the major driving force behind IDPR is
> to accommodate these new applications that have strict requirements
> on delivery, delay, and bandwidth ?

No, there are two motivations for IDPR.  This first is to ensure that
transit domains have control over which traffic flows use their
resources.  The second is ensure that sources have access to routes
that provide the services that their applications require.

> What do you expect be the percentage of traffic/users who will need to
> support these new applications ?

I cannot predict the number of users that will need to run these
applications at any given time in the future.  As more services are
offered, more users will be able to run applications that require
these services.  In turn, this may stimulate more requirements for
such applications.

> Do you assume that IDPR, on its own, will be able to accommodate these
> "strict requirements" ?

No, I don't.  To guarantee services, we need to be able to reserve
resources and to control traffic flow rates.  These functions are
not part of IDPR.

> In the next paragraph you said "We believe that IDPR meets this
> goal... " (with respect to ensuring that traffic receives its required
> service).  I think it needs to be explained in TECHNICAL details how
> IDPR, on its own, will accomplish that, rather than just "We
> believe...".

> If, on the other hand, you do not assume that IDPR, on its own, can
> accomplish this, then you need clearly articulate what are the
> dependencies, and how much support is needed (versus available today)
> from other components.  Moreover, if not all the dependencies provide
> the required support, you need to discuss whether the required support
> is feasible, and if feasible, what needs to be done to put it in place.

My phrase "the routing procedures must be capable of simultaneously
ensuring that traffic receives its required service" is misleading and
incorrect.  Thanks for pointing this out!  The phrase should say "the
routing procedures must be capable of simultaneously ensuring that
traffic is forwarded along routes that offer the required service".
IDPR can provide routes that meet traffic service requirements, but it
alone cannot ensure that traffic forwarded along the route will
receive the required service.

> There are two issues: required services and usage restrictions.

> Usage restrictions are unlikely to be applicable to commercial
> service providers. Indeed, just look at who in the current Internet
> has AUPs (which are nothing more than the usage restrictions documents)
> and who doesn't. Thus, the only potential candidates for
> usage restriction are government subsidized networks. Thus,
> the issue of usage restriction is likely to be relevent only
> to a subset of domains. This is especially true given "privatization"
> and "commercialization" of the Internet.

While I agree that government-subsidized networks (for example,
NSFNet) are candidates for usage restriction, I don't agree that they
are the only candidates.  Examples of other types of networks that I
expect will impose usage restrictions are as follows.  Those service
providers who charge for service (the commercial service providers)
will exercise usage restrictions in the form of restricting use to
paying customers only.  Also, I can imagine a consortium of
universities or companies working together on a set of projects and
purchasing special communications links between the consortium
members for use by the consortium members only.

> Likewise, definition of required services is likely to be rather loose
> for a wide variety of applications. For example, what is the meaning
> of "ensuring required service" for e-mail, FTP, Telnet, NFS ?

For these applications, the only service requirement that I see is
that the information gets delivered intact.  I don't believe there are
stringent delay or bandwidth requirements defined for such
applications, although such requirements could be defined.

> While I have no doubts that IDPR can theoretically accommodate an Internet
> comprising of O(10^4) domains, we need to look at what are the practical
> implications (in terms of storage, computational, etc overhead) of doing
> this.  For example, what would be the implications on overhead if we'll
> deploy IDPR as a dominant inter-domain routing protocol in existing internet;
> would it require less/more storage, CPU, etc..; what would be
> the implications on overhead if we'll deploy IDPR as a dominant inter-domain
> routing protocol in an internet twice the size of the current one, but
> with addresses assigned in a hierarchical fashion.

I did provide any actual numbers that demonstrate the processing,
memory, and link bandwidth overhead of IDPR compared to other
inter-domain routing procedures, for the current Internet.  According
to RFC 1264, providing such numbers is not a requirement for
requesting proposed standard status for a routing procedure.

However, a little further on in the document (Section 4), I did
provide some size information for the routing information database and
the forwarding information database.  These are order estimates only,
to indicate those factors that affect the size of each database.  In
fact, they are worst case estimates.  The question is, is the worst
case the average case?  For the current Internet, the answer is no,
but for the future?  I don't know.  Do you?

Also, in section 2.3, I talked about moving the route computation
function to a route server rather than keeping it in the policy
gateways, in order to reduce the computational load on the policy
gateways.

> There are two aspects in the design:
> (a) ability to accommodate O(10^4) domains
> (b) ability to accommodate domains with diverse offering and requirements

> How relevant is the first aspect to the existing Internet ? How soon
> the Internet would exceed the limit that the design claims to support ?
> Likewise, how relevant is the second aspect to the existing Internet ?
> How many diverse offering do you see in the Internet today, two years
> from now ?

The claim that IDPR can accommodate O(10^4) domains applies to those
domains that are directly visible to IDPR.  In an environment
supporting a hierarchy of domains, where the hierarchy is built by
forming "confederations of domains" in your terminology or "super
domains" in IDPR terminology, many domains may be hidden from view.
In this case, IDPR would only care about those domains that are at the
top level of the hierarchy.  And according to the claim, IDPR can
support O(10^4) such domains.

Service offerings must be accompanied by mechanisms for allowing users
to take advantage of these offerings.  In the current Internet, there
aren't many different services offered.  However, I believe that this
is in part because there aren't many protocols that can distinguish
between such offerings.  In the near term, I believe that applications
that require bandwidth reservation will become prevalent.  The ability
to generate and use routes that supply the requested bandwidth and to
manage the traffic flows over such routes will be the responsibility
of Internet routing and flow control mechanisms.

> Do you have an example of an environment for which the protocol is not
> suitable, or is it suitable for any environment ?

I think IDPR will perform well in almost all environments, even those
where policy routing is not an issue.  Some may argue that in such and
environment, the overhead of path setup does not justify its
existence.  However, if paths persist for long periods of time, then
the initial overhead of path setup is not significant compared to the
life of the path.

IDPR is not as well suited to extremely dynamic environments, where
connectivity between domains is constantly changing because of failing
equipment or because of mobility of nodes.  Routes will require
frequent recalculation and paths will be setup and torn down often.
However, no routing procedures with which I'm aware (including those
used in the packet radio environment) deal gracefully with a very
dynamic Internet.  In other words, I don't think there is anything
special about IDPR that will make it perform worse than other routing
procedures in a dynamic environment.

> Section 4.

> Again, the issue is not how many domains the design is capable of
> accommodating, but what are the implications of the design on the storage
> and processing
> capabilities required by routers. For example, for an internet of 10,000
> domains how many forwarding entries need to be maintained simultaneously at
> various routers within the internet ?

Please see above.

> "Less likely" compared to what ?
> This assumption may not be correct, as IDPR aims at providing routes
> consistent with "required services", and services provided by a domain may be
> a function of intra-domain routing.

The text should say "unlikely".  True, if each intra-domain route
supported a completely different type of service, then the loss of a
particular intra-domain route would be visible at the inter-domain
level.  However, I don't expect this to be the case.  Do you?  I am
interested in your projections of what the Internet will look like and
how it will be used.  My expectation is that usually, there will be
more than one intra-domain route that will satisfactorily provide the
services required.  (Or if no redundancy is provided, that component
failure is a rare event.)

> That assumes that a domain is at LEAST 2-connected. First of all, this may,
> or may not be the common case. Even if the topology is 2-connected, switch
> failure may cause inability to select an alternative route. For example, in
> T3 NSFNET Backbone if an E-NSS that is used to attach a regional network
> fails (E-NSS is just a single machine), then the regional network
> loses its connectivity altogether. Similar situation exists on T1 NSFNET
> if an E-PSP crashes. Thus, even single temporary intra-domain failures will
> be quite visible at the inter-domain level, contrary to your assertion.

As part of the IDPR architecture, we suggest multiple policy gateways
connecting two adjacent domains to handle exactly this situation.
Domain administrators are free to choose how much policy gateway
redundancy their domains support.

> Even if you have the redundant connectivity, the services provided by such
> connectivity may be quite different from the services provided via primary
> connectivity.  For example, an E-NSS may have T3 connection to the C-NSS as
> its primary link, but only T1 connection as its secondary. Obviously, that
> services provided via T1 (in the case when T3 fails) are quite different that
> the one provided via T3 (primary).  Thus, if the IDPR route selection is
> sensitive to bandwidth, then temporary intra-domain failure will be visible
> at the inter-domain level, contrary to your assertion.

Yes, the more diverse the service offerings in a domain and the less
redundancy in the resources used to provide those offerings, the
greater the chance that an intra-domain failure will be visible at the
inter-domain level.  I don't disagree.  However, I believe that money
will drive redundancy.  People paying for a service will expect to get
that service.  The service provider, in order to attract and keep
customers, will try to provide that service almost all of the time.
And that may mean redundancy of components or extremely reliable
components or both.

> Based on these facts your expectations may not be correct. What are the
> implications of these expections on the performance of the protocol ?

Changes visible at the inter-domain level will prompt distribution of
updated routing information and establishment of new paths.

> The frequency with which policy gateways need to generate and distribute
> routing information messages depends on:

> (1) stability of inter-domain links
> (2) stability of inter-domain switches
> (3) stability of services provided within a domain

> Why do you think that even if the distribution of routing information is
> triggered only by detectable inter-domain changes, these changes are not
> going to occur often.  Moreover, what does it mean "often" (e.g. once a day,
> once per hour) ?

For me, once an hour is often; once a day is not often.  As an example
of a large network, I look at the MILNET (~200 nodes) which has
"mature" packet switches and some pretty error-prone links throughout
the world.  It seems that nodes seldom go down (many remain up for
months at a time) and when they do fail, they are often down for a
while, so they don't tend to "flap".  Links are more likely to flap
but the subset of links that do tend to flap is less than 10% of the
total number of links.  Perhaps I am being overly optimistic when I
expect that the fancy new routers and communications media, for
example of the NSFNet, should be at least as reliable as MILNET
technology.  Am I wrong in assuming such reliability?

> Conceptually this sounds fine, but I wonder how important this feature is in
> practice.  I heard several comments that suggest that if there is a traffic
> between domains A and B, then the average number of hosts involved in this
> traffic is equal to 2 (one source and one destination).  If, indeed, this
> is the case, then it does not matter whether "many different traffic flows
> may use ... the same policy... and the same path", since on average there
> will be only ONE traffic flow.

The feature is there as one of many that are meant to keep the size of
IDPR databases "manageable".  I also do not know how often it will be
used.  However, I can imagine circumstances under which it would be
useful, for example, several people on different hosts in a given
domain simulataneously accessing a service provided in another domain.
An an example, when a new RFC is announced, several people on
different hosts in the same domain may simultaneously go to the same
RFC repository (in a different domain) in order to extract a copy.
Here one path would clearly be a savings.

> Section 5:

> What is the relation between the prototype implementation and the gated
> version ?

The gated version is a single process.  It was developed by a subset
of the people that developed the original multiprocess prototype
version.  Some of the software was directly ported from the prototype;
other parts were completely rewritten.  Also, functionality missing
from the prototype has been added to the gated version.  For a
detalied description of the difference between the two IDPR
implementations, please consult Woody Woodburn at [e-mail].

> *************** An architecture for inter-domain policy routing

> As you indicated, there is a new version of the document dated March 1992.
> I think you need to give members of the group a chance to read it before
> submitting it to IESG.
> If this version is identical to the one you posted in July 1991,
> then I have quite a few substantial comments on it which I would like to
> discuss within the WG (we can do it over e-mail).
> If it is different, then I would like to read it, before making any comments.

> ************** Inter-Domain policy routing protocol specification: version 1

> This is again was dated March 1992. Again, I think you need to give members
> of the group a chance to read it before submitting it to IESG.

Actually, the March 1992 versions have not yet hit the Internet drafts
directory.  They are not significantly different from their earlier
counterparts.  The changes in the documents are for readability, not
basic protocol functionality.  (Some message formats may have changed
slightly, but those are basically the only "technical" changes.)

Yakov, if you have had comments since July 1991, why have you not sent
them to the working group list or brought them up in IETF working
group meetings?  As you know, at every IETF meeting I have begged for
comments on IDPR.  Although I have received some responses, I would
love to receive more.  (I seem to remember that you made a similar
statement last summer about earlier versions of the IDPR documents,
some of which had been in existence for a year by that time.)

Guys, it is not fair to the working group to hold out on significant
comments, only to allude to them at the time that the working group is
trying to submit the work as a standard.  Please, if anyone has any
comments on any versions of the documents, mail them in to the working
group list [e-mail].

Thanks,
m