Re: Last Call for draft-ietf-rolc-apr-00.txt

[Curtis Villamizar <[e-mail]>]

In message <[e-mail]>om>, Yakov Rekhter writes:
> Curtis,
> 
> As I promised in my previous note, here are my specific comments.
> 
> > This draft also dabbles in QoS issues, though discussion is somewhat
> > orthogonal to the issue of cut through.  QoS may be a motivation for
> > cut through, but that point is not made very clearly and a good deal
> > of misinformation about the state of IP and QoS is presented.  This
> > topic is already better covered by RFC1821.
> 
> Please refer to specific places in the text where "a good deal of misinformat
> ion
> about the state of IP and QoS is presented".

   To exploit capabilities provided by the SVC-based Data Link
   subnetworks we propose to allow SVC management to be directly
   controlled by applications,

This is a bad idea and runs counter to rsvp and int-serv directions.

   We propose certain modifications to the existing IP model in order to
   support both the applications with QoS requirements that could
   justify a dedicated SVC, and applications that would rely on the
   router-based infrastructure.

The implication throughout is that an SVC is needed to support QoS.

The QoS decision could be described as:

  - consider shared hop by hop forwarding
  - consider shared hop by hop forwarding with traffic treatment
    designed for a specific class of traffic
  - consider hop by hop forwarding with traffic treatment for the
    individual flow
  - consider cut through with a shared traffic treatment
  - consider cut through with with traffic treatment designed for a
    specific class of traffic
  - consider cut through with traffic treatment for the individual
    flow

Not necessarily in that order.  All of the "consider ..." clauses
should be predicated with "where available".

This concedes that hop by hop QoS may exist and cut through may exist.
Both can be considered where available.  Some people may think one or
the other won't be available.  Some may prefer not to consider one or
the other for some reason, perhaps a scaling, performance,
implementation quality, or other practical reason that we cannot
forsee at this point.  It is not up to anyone, whether a WG chair, IAB
member, or very vocal person in the WG to exclude the possiblity of
the work of others from consideration without sound technical basis.

> > The document proposes the the concept of subnet be replaced with APR,
> > where APR implies probably but uncertain direct reachability where
> > subnet implies direct reachability.  Removing the former definition of
> > subnet is inadvisable as it would throw prior discussions of IP into
> > terminology chaos.  There is no compelling argument for the
> > desirability of introducting the notion of a prefix region in which
> > direct reachability is probable based solely on the prefix match, but
> > is not certain.  
> 
> You are incorrect. Within an single APR direct connectivity is always
> possible, and not "probable".

Except when links are down.  So it is probable but not certain.  The
address may still be reachable by other means.  There needs to be a
better way to determine that the fabric has partitioned than failed VC
connect attempts.  These are your words:

   The
   ability to acquire the Link Layer address of the peer should not be
   viewed as an indication that the host and the peer can establish an
   SVC -- the two may be on different Data Link subnetworks, or may be
   on a common Data Link subnetwork that is partitioned.  Only the actual
   SVC establishment is sufficient to enable direct communication. If a
   host can not establish an SVC, the host may default (depending on the

> >    The original IP architecture assumes that each Data Link subnetwork
> >    is labeled with a single IP network number. 
> >    A pair of hosts with the
> >    same network number communicate directly  (with no routers); a pair
> >    of hosts with different network numbers always communicate through
> >    one or more routers. 
> >    As indicated in RFC1620, these assumptions may
> >    be violated for large data networks, and specifically for networks
> >    based on switched virtual circuit (SVC) based technologies (e.g. ATM,
> >    X.25).  The architecture works even when this assumption is violated,
> >    but it imposes constraints on communication among hosts and routers
> >    through a network, which in turn may preclude full utilization of the
> >    capabilities provided by the underlying SVC-based Data Link
> >    subnetwork.  This document describes extensions to the IP
> >    architecture that relaxes these constraints, thus enabling the full
> >    utilization of the services provided by SVC-based Data Link
> >    subnetworks. The document also specifies how the proposed extensions
> >    could be used for other types of subnetworks.
> > 
> > I feel strongly that we need to change the tone of this paragraph to
> > reflect and extension to the IP architecture rather than some sort of
> > architecture replacement.  Change "The original IP architecture" to
> > "The IP architecture".
> 
> Ok. I'll do the change.

Thanks.  Please change the tone.

> > We should use current IP terminology, which means it is OK to talk
> > about subnets and CIDR prefixes.  We should be able to say "A pair of
> > hosts covered by the same subnet prefix".  I don't think we need to
> > claim that an architectural assumption is violated.  The introduction
> > of subneting was an extension to the original class based network
> > architecture.  CIDR was a further extension.  Cut through is an
> > extension, which allows a direct connection across subnets and does
> > not violate assumptions of the IP architecure any more than subnetting
> > or CIDR or numbered or unnumbered point to point interfaces did in the
> > past.  There is a vast distinction between an architecture change and
> > a minor extension.
> 
> Using CIDR terminology is a good suggestion. I'll correct the text. 

OK.

> >   1  Background
> > 
> >    The following briefly recaptures the concept of the IP Subnet.  The
> >    Internet architecture is based on the "Catenet model" [Postel:81].
> >    The topology is assumed to be composed of links (Data Link
> >    subnetworks) interconnected via routers.  Each link has a globally
> >    unique number. An IP address of a host with an interface attached to
> >    a particular link is a tuple <link number, host number>, where host
> >    number is unique within the link. When a host needs to send an IP
> >    packet to a destination, the host needs to determine whether the
> >    destination address identifies an interface that is connected to one
> >    of the links the host is attached to ("local" decision), or not
> >    ("remote" decision).  The outcome of the "local/remote" decision is
> >    based on (a) the source address, (b) the destination address, and (c)
> >    the subnet mask associated with the source address.  If the outcome
> >    is "local", then the host resolves IP address to Link Layer address
> >    (e.g. by using ARP), and then sends the packet directly to that
> >    destination (using the Link layer services).  If the outcome is
> >    "remote", then the host uses one of its first-hop routers (thus
> >    relying on the services provided by IP routing).
> > 
> > There is no sense bringing us all the way back to Postel:81.  We can
> > and should use CIDR terminology and describe what changes need to be
> > made relative to current routing and addressing practices.  You are
> > not changing the catenet model unless these multiple subnet regions
> > can not be concatenated with existing IP internetworks, which I assume
> > is not part of this proposal.
> > 
> >    To summarize, two of the important attributes of the IP subnet model
> >    are:
> > 
> >       hosts with a common network number are assumed to be attached to a
> >       common link (subnetwork), and thus communicate with each other
> >       directly, without any routers -- "local"
> > 
> >       hosts with different network numbers are assumed to be attached to
> >       different links (subnetworks), and thus communicate with each
> >       other only through routers -- "remote"
> > 
> > Please use "subnet address prefix" rether than "network number".
> > Class based network numbers as the sole means of addressing died a
> > long time ago.
>   
> As I mentioned above, I'll fix the document to use CIDR terminology.

Thanks.  Just trying to be thorough.  It was a last call you know.

> >   2  Motivations
> > 
> >    The diversity of TCP/IP applications results in a wide range of
> >    traffic characteristics.  Some applications last for a very short
> >    time and generate only a small number of packets between a pair of
> >    communicating hosts (e.g. ping, DNS). Other applications have a short
> >    lifetime, but generate a relatively large volume of packets (e.g.
> >    FTP). There are also applications that have a relatively long
> >    lifetime, but generate relatively few packets (e.g.  Telnet).
> >    Finally, we anticipate the emergence of applications that have a
> >    relatively long lifetime and and generate a large volume of packets
> >    (e.g.  video-conferencing).
> > 
> > This is an entirely different issue, covered much more accurately in
> > RFC1821 than in this section.
> 
> This is not a different issue. This is precisely why SVC management should
> not be driven by addressing considerations, but by traffic characteristics.

RFC1821 does not propose SVC management by addressing considerations.
In fact it explicitly states:

 4.3 Mapping IP flows to ATM connections

   In general, there will be a great deal of flexibility in how one maps
   flows at the IP level to connections at the ATM level. For example,
   one could imagine setting up an ATM connection when a reservation
   message arrives at the edge of an ATM cloud and then tearing it down
   as soon as the reservation times out. However, to minimize latency or
   perhaps for economic reasons, it may be preferable to keep the ATM
   connection up for some period in case it is needed. Similarly, it may
   be possible or desirable to map multiple IP flows to a single ATM
   connection or vice versa.

   An interesting situation arises when a reservation request is
   received for an existing route across the cloud but which, when added
   to the existing reservations using that connection, would exceed the
   capacity of that connection. Since the current  ATM standards do not
   allow the QoS of a connection to be changed, there are two options:
   tear down the old connection and create a new one with the new,
   larger allocation of resources, or simply add a new connection to
   accommodate the extra traffic. It is possible that the former would
   lead to more efficient resource utilization. However, one would not
   wish to tear down the first connection before the second was
   admitted, and the second might fail admission control because of the
   resources allocated to the first. The difficulties of this situation
   seem to argue for evolution of ATM standards to support QoS
   modification on an existing connection.

I really have to ask what you are adding that wasn't said in RFC1821.

> >    The traditional IP subnet model provides a poor match for flexible
> >    and adaptive use of the SVC-based Data Link subnetworks  - the use of
> >    a subnetwork is driven by information completely unrelated to the
> >    characteristics of individual applications.  
> > 
> > This is too braod a statement and is not well clarified by the
> > example.  Why not just get to the point and state "The scaling
> > properties of internetworking routing protocols in use today, and
> > possibly routing protocols in general requires the subdivision of
> > large networks into smaller prefixes and the isolation of routers from
> > the inconsequential details of distant small changes in routing
> > through abstration.  With the current IP forwarding model IP traffic
> > follows the reverse path of the routing information, the traffic being
> > forwarded by each router that forwarded routing information.  This
> > does not lend itself to taking advantage of VCs which can be set up to
> > span multiple subnets.  There may be advantages to setting up direct
> > VCs between distant endpoints for certain classes of applications".
> 
> Your text is correct, but it ignores one of the points made in the document -
> it should not be required to always establish a VC between a pair of
> hosts on the same subnet. I'll reword the text along the lines you suggested,
> but will also include the point I made.

Yes.  I did not make that point.  Thanks.

> > There really should be mention of RSVP in this section.  The whole
> > issue of subnet cut through stems not from the fact that there is no
> > provision for QoS with IP and RSVP, just that there is a feeling among
> > some that cell switching will somehow provide much "better" service,
> > where "better" may be better QoS guarentees, or the ability to scale
> > to higher traffic capacities (mostly stemming from the belief that
> > cell switching is fundamentally better than packet switching).
> 
> That is correct and I'll make the changes.

Please consider the point I made above about shared/individual VCs,
hop-by-hop and cut-through, and special queueing treatment.  These are
three separate parameters, though related.  Various combinations may
provide better QoS or more efficient delivery of service.  One
combinations may work best for some traffic characteristics and
traffic requirements and another combination may work best for other
types of traffic.  We should not try to prejudice the technolgy
selection in a draft that tries to define the playing field.

> >   3  QoS/Traffic Driven "Local/Remote" Decision
> > 
> >    The ability of a host to establish an SVC to a peer is predicated on
> >    its knowledge  of the Link Layer address of the peer. This document
> >    assumes the existence of mechanism(s) that can provide the host with
> >    this information. Some of the possible alternatives are NHRP, ARP, or
> >    static configuration; other alternatives are not precluded.  The
> >    ability to acquire the Link Layer address of the peer should not be
> >    viewed as an indication that the host and the peer can establish an
> >    SVC -- the two may be on different Data Link subnetworks, or may be
> >    on a common Data Link subnetwork that is partitioned. Only the actual
> >    SVC establishment is sufficient to enable direct communication. If a
> >    host can not establish an SVC, the host may default (depending on the
> >    application) to sending data through routers.
> > 
> > Solid advise, but not central to "the change in IP" being discussed.
> 
> I think it is relevant to the discussion (even if not central).

It was not clear to me that the local/remote decision was the central
issue since I've assumed it to be a given.  I considered it sufficient
described by RFC1821 and the IP over ATM Framework document.

> >    One important implication of this proposal is that in contrast with
> >    the conventional IP environment, the "local/remote" decision may no
> >    longer be time invariant. While at one moment a pair of hosts (e.g. A
> >    and B) may have an SVC between them (e.g. when there is a video-
> >    conference running between the hosts) and thus will be viewed as
> >    "local" to each other, at some later point in time communication
> >    between exactly the same pair of hosts (e.g. A and B) will be done
> >    through one or more routers (after the video-conference ends, and
> >    someone would decide to run ping) and thus will viewed as "remote".
> > 
> > Routing information never has been time invariant.
> 
> So, what is your point ? The paragraph talks about the outcome of the
> "local/remote" decision being no longer time invariant.

You may confuse the issues here.  It could not become clear whether
load sensitive routing would be practical, though theoretically
appealing, until there was some experience with it.  It may also prove
that load sensitive QoS routing also proves difficult.  Discussing
such a tangential issue is a distraction from the central point.
Briefly eluding to it and leaving it wide open may be worse.  This is
not critical and maybe not even important to the central issue.

> >    In addition to being time dependent, the "local/remote" decision may
> >    yield both "local" and "remote" outcome simultaneously. This is
> >    because a set of hosts may concurrently run multiple applications,
> >    where some of these applications could justify an SVC establishment
> >    (thus resulting in a "local" outcome), while others will rely on
> >    router-based infrastructure (thus resulting in a "remote" outcome).
> > 
> > This has always been an implication of QoS.
> 
> Perhaps. Could you please provide me with some references, so I would include
> this in the document.

I was just trying to make the point that to some you are stating the
obvious.  To others you are confusing the issues.

The OSPF RFC describes QoS routing.  It is clear that different QoS
may take different paths.  The local/remote decision is QoS driven and
so is analogous.

> >    Even if when applications yields "local" outcome, depending on the
> >    nature of the applications a pair of hosts should be able to either
> >    multiplex several applications over a single SVC, or establish
> >    dedicated SVCs on a per application basis or both.  In the case where
> >    an SVC is shared among several applications care must be taken to
> >    ensure fair sharing of the resources provided by the SVC. For
> >    example, while it may be acceptable to share a single SVC for
> >    multiple FTP sessions between a pair of hosts, sharing an SVC for an
> >    FTP session and a video-conference is likely to be more problematic.
> > 
> > You mave simply state that the cut through decision isn't time
> > invariant, isn't dependent solely on destination, and may involve
> > shared or individual VCs.  Stating this without further explanation,
> > does not clarify anything, but rather may lead to further confusion.
> 
> I would put this as a separate paragraph (like a summary).

Let's see how the next draft reads.

> >   4  Address Prefix Region (APR)
> > 
> > The unfortunate conclusion of the proposal...
> > 
> >    To provide flexible and adaptive use of SVC-based Data Link
> >    subnetworks we proposed to replace the concept of a traditional IP
> >    subnet with the concept of an Address Prefix Region (APR).
> > 
> > You had better have a real good reason.  As far as I can see you don't.
> 
> Let me reiterate the reason:
> 
> With the IP subnet model the outcome of the "local/remote" decision is
> solely controlled by the src/dst addresses. In the NBMA environment
> that could provide QoS support this may be suboptimal. That is the
> reason why  the document advocates to "alter/change/extend" (pick up
> your favorite term) the classical IP subnet model.

This is fine as long as the subnet is defined to be very small, much
smaller than the NBMA.  There is no reason to try to stretch the term
subnet so a subnet becomes as big as the NBMA.

> In the previous reincarnation of this document we use the term LIS, but
> at the IP-ATM WG meeting it was stronly suggested that a new term is
> needed. That is why we picked APR.

A new term was needed for "a specific subset of the internetwork
topology in which the underlying network does not prevent all hosts or
routers from being directly reachable without IP forwarding to an
intermediate party".  This has nothing to do with addressing and any
overlap with an address prefix, whether intentional or coincidental,
does not define the region.  The term "address prefix region" is
absolutely the wrong term.  The old terms commonly used were "NBMA
network" or "subnet in the OSI sense".  You need to pick a term that
clearly reflects what you are describing.

> >    An Address Prefix Region (APR) can be formally defined by the
> >    following properties:
> > 
> >       An APR is a set of routers and hosts
> > 
> >       Every element in the set (either a host or a router) can establish
> >       direct communication (an SVC) with every other element in the set
> > 
> >       IP addresses of the hosts in the set are assigned in such a way
> >       that they can aggregated into a single IP address prefix; each
> >       element in the set knows the prefix
> > 
> >       All routers in the set advertise direct reachability to all the
> >       hosts in the set -- any router in the set is 1 IP hop away from
> >       any host in the set
> > 
> > Consider what you would have if you drop "IP addresses of the hosts in
> > the set are assigned in such a way that they can aggregated into a
> > single IP address prefix; each element in the set knows the prefix".
> > 
> > You would have a region in which direct VC reachability is possible,
> > with no definition of how the direct reachability is determined and
> > without the restriction that it be covered by a single prefix.  Adding
> > the IP prefix implies that using the IP prefix to determine whether a
> > direct VC is possible is A Good Idea.  Previous discussion on this
> > list have indicated that using the prefix to determine direct
> > reachability is not a good idea for large networks and that some other
> > mechanism should be used.  
> 
> APR allows to couple routers with hosts for the purpose of IP connectivity vi
> a routers.
> You need this coupling. How would you do this if we would drop the sentence y
> ou suggested.

I would still define "(term we haven't picked) aka NBMA network" as I
did above.  I would then state that some at least mildly broken
schemes of implementing cut-through, most notable NHRP, have not
sufficiently accounted for learning what prefixes are directly
reachable.  A proposed procedural simplification for such protocols is
to make an address prefix exactly overlap the "aka NBMA network".  You
might want to reword that sentence.  I don't want to see a hack
codified in our terminology to cover the ugliness of the favorite
protocol of this WG.

> > For small networks, it might be that using the address prefix to
> > determine direct VC reachability is very simple and proves adequate.
> > The term APR may be fine for this purpose.  This is hardly grounds to
> > label APR as an IP architecture change.
> > 
> >    For the purpose of IP unicast forwarding the role of the APR is to
> >    act as a mechanism to associate a set of hosts with one or more
> >    routers that these hosts could use to establish connectivity
> >    (reachability) with (a) destinations that are not on a common fabric,
> >    or (b) destinations for applications that don't justify an SVC. An
> >    APR would identify for a given set of hosts the set of routers that
> >    these hosts can use as their first hop (first-hop routers).
> > 
> > This paragraph implies that you are simply replacing the term subnet
> > with APR but not changing the meaning at all.  That would serve no
> > purpose except to confuse discussions.  Which is it?
> 
> I am surprised by your interpretation.  This paragraph does not say
> that hosts within a single APR should always have direct ("local")
> connectivity with each other. That is the difference with the subnet
> model.

At the top of this note you said:

    You are incorrect. Within an single APR direct connectivity is
    always possible, and not "probable".

Which is it.  It wasn't clear in the draft and still isn't.

> >    The APR could serve as a useful migration tool from the current
> >    (traditional IP subnet model) environment, as it provides backward
> >    compatibility for hosts that support only the traditional IP subnet
> >    model (e.g. hosts that support only "classic" IP over ATM model),
> >    while allowing the intermix of these hosts with modified hosts that
> >    support application driven "local/remote" decision.
> > 
> > If it is a subnet, then it certainly provides compatibility since you
> > just continue to use the same subnet model, you just give the term
> > subnet a new name and declare a new architecture when in fact nothing
> > has changed at all.  If APR means something larger than a subnet,
> > meaning that IP forwarding can occur within an APR, then you have a
> > new term but not a new architecture.  The only thing new about any of
> > this is you are introducing the possibility of doing cut through.
> > 
> >   4.1 Host Modifications
> > 
> > This section is completely worthless.  It does not take into account
> > hosts that are one or more hops off the VC cloud.  For that, the host
> > must use some sort of signalling to a router a few hops down to get
> > QoS.  If there is no router, this can just be passed down the protocol
> > stack.  (Hey - why not use RSVP for the signalling :)
> 
> The section covers only the case where both src and dest are on a common
> NBMA. You are certainly correct that this section does not cover the
> case where destination is off the NBMA. I'll add a sentence which would
> say that in this case the application should use IP level services to
> get the desired QoS.

That is not at all clear.  You need a big discalimer stating clearly
that you are only covering a limited case.  I didn't see any such
disclaimer.  The draft would be a lot more complete if you covered
more than just this case.

> >   4.2 Router Modifications
> > 
> >    When a router associated with a given APR (as defined in this
> >    document) receives an IP packet from a host in the APR that is
> >    destined to another host in the same APR, the router should forward
> >    the packet (if possible), and refrain from sending an ICMP Redirect
> >    message to the originating host.
> > 
> > That's it!?!  Routers will never need to do anything wrt SVC setup,
> > they just need to avoid sending ICMP Redirect?  This section seems as
> > worthless as the previous, if not more so.
> 
> This section addresses the changes in the routers to accommodate
> hosts within a common APR that exchange their traffic through a router.

Same comment as above.

> >   5  Transition for ATM-based subnetworks
> > 
> > I disagree entirely with this section.  I see transition from RFC1577
> > are following a different course.
> > 
> >   RFC1577 hop by hop, one SVC between adjacent routers for all traffic
> > 
> >   RFC1577 hop by hop, RSVP used to control queueing and where
> >   appropriate set up separate SVCs for a class of traffic, a set of
> >   flows, or an individual flow.
> > 
> >   RFC1577, most traffic flowing hop by hop, some one alternate VCs,
> >   some traffic taking cut through VCs.
> > 
> > Reagardless, I don't see how APRs contribute toward a migration.
> 
> APR allows an ATM-attached host to decouple its "local/remote" decision
> from the addressing information. Preserving RFC1577 model forever (as
> implied in your suggestion) would mean that a pair of hosts within a
> single LIS would always have to establish a direct VC, irrespective of
> the traffic exchanged between the hosts.

APR is a term that couples an address prefix to the "(term) aka NBMA
network".  That coupling is what I don't see as needed.  I do see the
need for cut-through and the need to have a term for "aka NBMA
network", but not the coupling with a prefix.

> >   6 Conclusions
> > 
> >    Different approaches to SVC-based Data Link subnetworks use by TCP/IP
> >    yield quite different results with respect to the ability of TCP/IP
> >    applications to efficiently exploit the functionality provided by
> >    such subnetworks.  For example, in the case of ATM both LAN Emulation
> >    and "classical" IP over ATM (RFC1577) localize host changes below the
> >    IP layer, and therefore may be good first steps in the ATM
> >    deployment.  However, these approaches are likely to be inadequate
> >    for full utilization of functionality that ATM is expected to
> >    provide.
> > 
> >    It seems that any model that doesn't allow SVC management under
> >    direct control of applications (QoS) is likely to curtail efficient
> >    use of SVC-based Data Link subnetworks.  Enabling direct connectivity
> >    for applications that could benefit from the functionality provided
> >    by SVC-based Data Link subnetworks, while relying on routers for
> >    other applications, could facilitate exploration of the capabilities
> >    provided by the subnetworks.
> > 
> > It is entirely possible that RFC1577 and RSVP will serve the needs of
> > all traffic types.  The only argument in favor of considering an
> > alternate have to do with efficiency and QoS capability of cell
> > switching vs packet forwarding.
> > 
> > Any proposal that puts SVC management at the host but provides no
> > means to communicate QoS requirements from the host to an router one
> > or two hops away is shortsighted.
> 
> This proposal is about decoupling "local/remote" from addresses and
> coupling it with QoSs. That is all what this proposal is about. Means
> of communicating QoS to routers (or to ATM switches) are outside
> the scope of this document. But certainly such mechanisms exist,
> and I'll include references to such mechanisms in the document.

You are proposing to decouple addressing and then defining a term that
couples "aka NBMA network" with an address prefix.  Fix the ommision
of hop-by-hop QoS and one hop off the NBMA procedures and get rid of
the coupling with address prefix entirely and I'll wholeheartedly
support the draft.

> Yakov.

Best regards and appologies for being critical,

Curtis