single attach doc

Richard Fox <rfox@synoptics.com> Tue, 26 February 1991 01:35 UTC

Received: from merit.edu by NRI.NRI.Reston.VA.US id aa06659; 25 Feb 91 20:35 EST
Received: Mon, 25 Feb 91 20:34:04 EST from mvis2.synoptics.com by merit.edu (5.59/1.6)
Received: by mvis2.synoptics.com (5.61/2.1G) id AA08778; Mon, 25 Feb 91 17:33:59 -0800
Message-Id: <9102260133.AA08778@mvis2.synoptics.com>
Date: Mon, 25 Feb 91 17:33:59 -0800
From: Richard Fox <rfox@synoptics.com>
To: fddi@merit.edu
Subject: single attach doc
Status: OR


A few IETF's ago there was some concern over allowing single attach stations to
attach to the secondary ring. Basically it was pointed out that allowing 
single attached stations on both rings can break IP. At the last meeting
James Reeves promised that we would post a document that attempted to
solve the problem. Enclosed you will find a draft document that
attempts to solve the problem without violating either IP or FDDI and
makes use of the Extended ARP protocol developed by this working
group.

If you have any questions/concerns please forward them to me and we
can either discuss this in St. Louis or via mail. If there are no
major problems with this document then I will publish it as an
informational RFC. 


thanks rich



----------- cut here ----------------------- 




          Internet Draft         Proxy IP Bridge                 2/10/91


               Definition of a Proxy IP Bridge for an FDDI Network

                                     9/10/90


                                   Richard Fox

                          SynOptics Communications Inc.

                                rfox@synoptics.com







          The FDDI[1,2] standard is dual counter rotating ring
          technology that allows stations to attach directly to both
          rings or to attach to a single ring through a concentrator.
          The standard does not place any limitations on which ring
          single attached stations must connect to. Thus, some stations
          may be connected on the primary ring only, while others are
          only on the secondary ring. Some configurations of an FDDI
          network topology are believed to break certain IP[3] network
          configurations by creating a discontiguous IP network where
          not all stations on the same IP subnet can communicate.  This
          document will provide a solution that  will allow any
          configuration of single attached stations to an FDDI ring such
          that IP integrity is maintained. The solution provided can be
          thought of as an IP extension of a MAC layer bridge connecting
          the two rings together or as an algorithm that is located on
          any dual MAC dual attached device. Also, the solution uses a
          mechanism called Extended ARP[4] which is an extension of the
          ARP[5] protocol to be used on technologies that allow dual
          attached stations to a network, such as FDDI.

          Distribution of this memo is unlimited.


          1.  Introduction

          The FDDI[1,2] standard is a dual counter rotating ring
          technology that allows stations to attach directly to both
          rings or to attach to a single ring through a concentrator.





          Richard Fox                                           [Page 1]





          Internet Draft         Proxy IP Bridge                 2/10/91


          The standard also allows both rings to be used for data
          traffic simultaneously, placing no limitations on which ring
          single attached stations must be connected to.  Thus, the
          configuration pictured in figure 1 and figure 2 are both legal
          FDDI network configurations.













































          Richard Fox                                           [Page 2]





          Internet Draft         Proxy IP Bridge                 2/10/91


                        ---------------               ---------------
                        |             |               |             |
                        |             |               |             |
                        | Station A   |               | Station B   |
                        |             |               |             |
                        -------|-------               -------|-------
                               |                             |
                               |                             |
                 /-------------|-----------------------------|-------------\
                 |                                                         |
                 |                                                         |
                 |     /---------------------------------------------\     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                \|/   /|\                                           /|\   \/
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     \------------------------|--------------------/     |
                 |                              |                          |
                 |                              |                          |
                 \------------------------|-----|--------------------------/
                                          |     |
                                          |     |
                                          |     |
                                      ----|-----|----
                                      |             |
                                      |             |
                                      | Station C   |
                                      |             |
                                      ---------------


                                        figure 1








          Richard Fox                                           [Page 3]





          Internet Draft         Proxy IP Bridge                 2/10/91


                        ---------------               ---------------
                        |             |               |             |
                        |             |               |             |
                        | Station A   |               | Station B   |
                        |             |               |             |
                        -------|-------               -------|-------
                               |                             |
                               |                             |
                 /-------------|-----------------------------|-------------\
                 |                                           |             |
                 |                                           |             |
                 |     /-------------------------------------|-------\     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                \|/   /|\                                           /|\   \|/
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     \------------------------|--------------------/     |
                 |                              |                          |
                 |                              |                          |
                 \------------------------|-----|--------------------------/
                                          |     |
                                          |     |
                                          |     |
                                      ----|-----|----
                                      |             |
                                      |             |
                                      | Station C   |
                                      |             |
                                      ---------------


                                        figure 2


          Dual ring technologies that allow  both rings to contain data





          Richard Fox                                           [Page 4]





          Internet Draft         Proxy IP Bridge                 2/10/91


          traffic simultaneously have many advantages. Three notable
          advantages of a dual ring technology like FDDI are load
          balancing, data aggregation, and transparent fault recovery.

          Load balancing allows traffic to be split among the two rings
          to meet some required optimization on the LAN. A common
          example of load balancing is to split traffic between the
          rings where one ring is used for high bandwidth requirements
          and the other ring is used for applications requiring good
          response time. For instance, one might restrict telnets[6]
          (good response time) to the primary ring and restrict ftps[7]
          (high bandwidth) to the secondary ring. Another example of
          load balancing is to put a companies accounting department on
          one ring and the engineering department on the other ring,
          with the dual MAC, dual attached stations being used by the
          executive management staff to oversee all company activities.

          Data aggregation is where a station takes full use of both
          rings to achieve a throughput greater than the 100 megabits of
          a single ring. For instance, two dual MAC, dual attached
          stations may set up an ftp where both rings are used to move
          the data in order to fully utilize the 200 megabits for a
          single file transfer.

          Transparent fault recovery is the ability of the network to
          reconfigure in the event of a hardware failure such that upper
          layer protocols continue to work without any outside
          intervention. In fact, the recovery should happen without the
          upper layer protocols even being aware that recovery is taking
          place. For instance when a station loses a MAC and the goes
          into wrap state, upper layer protocol connections should not
          break, but should continue to operate using a working MAC on
          the station. This feature is possible by the use of the
          Extended ARP solution.

          For upper layer protocols like TCP[8]/IP to take full
          advantage of these benefits, care must be taken to make sure
          that basic principles of the protocol are not broken by the
          configuration of the technology on the layer below. In IP,
          care must be taken to assure that the two FDDI rings do not
          fragment an IP subnet or cause an IP subnet to become
          discontiguous. Figure 3 shows an IP configuration mapped onto
          a legal FDDI network. In this example all stations belong to
          IP subnet X. However, with no bridge on the network, station A
          and station B cannot communicate, thus creating a





          Richard Fox                                           [Page 5]





          Internet Draft         Proxy IP Bridge                 2/10/91


          discontiguous IP subnet. In fact, it is this configuration of
          IP over FDDI that has caused many to advocate that all single
          attached devices, like A and B in figure 3, be connected to
          the primary ring only, as depicted in figure 1.


                        ---------------               ---------------
                        |             |               |             |
                        | Station A   |               | Station B   |
                        |             |               |             |
                        | IP Subnet X |               | IP Subnet X |
                        -------|-------               -------|-------
                               |                             |
                               |                             |
                 /-------------|-----------------------------|-------------\
                 |                                           |             |
                 |                                           |             |
                 |     /-------------------------------------|-------\     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                \|/   /|\                                           /|\   \|/
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     |                                             |     |
                 |     \------------------------|--------------------/     |
                 |                              |                          |
                 |                              |                          |
                 \------------------------|-----|--------------------------/
                                          |     |
                                          |     |
                                          |     |
                                      ----|-----|----
                                      |             |
                                      | Station C   |
                                      |             |
                                      | IP Subnet X |
                                      ---------------





          Richard Fox                                           [Page 6]





          Internet Draft         Proxy IP Bridge                 2/10/91


                                        figure 3

          Providing a pure IEEE MAC layer bridge[13] between the two
          rings of an FDDI network solves the connectivity problem of
          IP. However, it does this by simulating an FDDI wrap condition
          (ie: combining the two rings into one ring), thus compromising
          the advantages that FDDI provides. While the advantages listed
          above may be possible, they can not be guaranteed.  Instead,
          they depend on a number of events occurring in a certain
          sequence. As an example, assume that the goal is to achieve
          data aggregation so that two stations can communicate at 200
          megabits/second.  If the bridge has not learned[13] the MAC
          addresses of the two dual MAC dual attach stations it will
          forward all frames destined to these stations until the MACs
          are learned (read [13] for exact details). During this
          learning phase it is very likely that an Extended ARP[4]
          request from one ring will be bridged, or forwarded, to the
          other ring. This may cause the destination to cache the wrong
          source MAC address causing all frames sent between the two
          stations to be bridged, limiting the throughput to the
          forwarding rate of the bridge, with a maximum throughput rate
          of 100 megabits/second.

          This paper proposes an algorithm that allows any configuration
          of IP on an FDDI ring such that the three advantages of FDDI
          listed above are realized.  The algorithm may be put on a pure
          IEEE MAC layer bridge as an IP extension or on any dual MAC
          dual attached device attached directly on the FDDI trunk ring.
          The algorithm provides a contiguous view of an IP subnet in
          all of the possible configurations of an FDDI ring, including
          a configuration where single attached stations are attached to
          both the primary and secondary rings. Section 2 describes the
          various configurations that can exist in IP over FDDI with
          single attached stations.  Section 3 provides a solution to
          the problem depicted in figure 3.  Section 4 will take another
          look at section 3 with the addition of IP routers to the ring,
          and section 5 will deal with open issues and other
          implementation considerations, such as twisted rings and
          performance considerations.











          Richard Fox                                           [Page 7]





          Internet Draft         Proxy IP Bridge                 2/10/91


          2.  Configurations of Single Attached Stations in an IP/FDDI
          Environment

          Figure 2 shows a configuration of an FDDI network where there
          is a single attached station on the primary ring and one on
          the secondary ring. There are a number of IP configurations
          that can be mapped onto this configuration. This section will
          look at each of the configurations in detail, determining if
          the IP mapped configuration onto the FDDI ring breaks IP when
          the FDDI ring is in thru mode or in wrapped mode. A third
          configuration that an FDDI ring can be in is called twisted.
          This mode is discussed in section 5.

          The IP configurations that will be described are all based on
          figure 2. The configurations are:

            Case 1. All interfaces (MACs) on the primary ring belong to IP subnet X,
                  while all interfaces on the secondary ring belong to IP subnet Y.
                  This means station C belongs to two IP networks (X & Y).

            Case 2. Same as case #1 except station C (dual attached stations) use the
                  same IP subnet (X) on both rings. This means that A and C are part
                  of the same IP network but C is no longer part of the same IP
                  network as B.

            Case 3. All stations have the same IP subnet (X) regardless of which rings
                  the stations are attached to. This is equivalent to figure 3.


          2.1.  Case 1: Different Networks

          Networks comprised of different physical cables normally
          assign each physical cable a different IP subnet network
          number unless a bridge is used to connect the cables. This
          case treats the dual counter rotating rings of FDDI as two
          separate networks containing no bridges between the two rings.
          In this configuration each ring is assigned a separate IP
          network.

          This mapping of IP over FDDI does not benefit from the data
          aggregation or transparent fault recovery benefits listed in
          the introduction. However, this configuration may be widely
          used since it benefits from implicit load balancing. Load
          balancing is achieved since stations can be attached to the
          rings based on local traffic requirements.





          Richard Fox                                           [Page 8]





          Internet Draft         Proxy IP Bridge                 2/10/91


          2.1.1.  Case 1: In Thru Mode

          While in thru mode, the FDDI ring acts like two separate
          networks. For station A and station B to communicate, a path
          using routers must exist from the primary ring to the
          secondary ring (C could be a router). If no router exists,
          then A and B will not be able to communicate, even though they
          are on the same FDDI ring network. There is no IP requirement
          that mandates that stations from different IP networks must be
          able to communicate, thus the integrity of IP is maintained in
          this mapping of IP over FDDI while in thru mode.


          2.1.2.  Case 1: In Wrapped Mode

          When an FDDI ring wraps, both the primary and secondary rings
          combine to form one logical ring[1,2,9]. The affect a wrap has
          on IP in this configuration is that the two IP networks X and
          Y, which used to be on different logical rings, are now
          combined onto the same logical ring. This configuration is a
          legal IP configuration (one that is in use today), but care
          must be taken to insure that the network can still function.
          [10,11] gives some useful suggestions on how to eliminate ARP
          storms and other problems that can occur when two IP subnets
          are on the same physical cable.


          2.2.  Case 2: Dual stations using 1 IP Network

          In this case, dual MAC, dual attached stations may use the
          same IP network address for both MACs (station C in figure 1).
          This means that in figure 2, station A and station C are part
          of the same IP network, while station B is not. The benefits
          mentioned in the introduction will be realized by the dual
          MAC, dual attached stations that use the same IP address for
          both MACs, while all other stations will only realize the
          benefit as stated in section 2.1.


          2.2.1.  Case 2: In Thru Mode

          While in thru mode, the FDDI ring acts like two networks
          slightly overlapped. In other words, both rings may have
          stations from either of the two IP networks (X & Y). As long
          as all single attached stations belonging to the same IP





          Richard Fox                                           [Page 9]





          Internet Draft         Proxy IP Bridge                 2/10/91


          network are on the same ring, IP integrity is not broken.

          IP integrity is not jeopardized, since all stations on the
          same IP network are directly reachable by all other stations
          on the same IP network. If the Extended ARP protocol is used,
          dual MAC, dual attached stations can take advantage of the
          benefits listed in the introduction section, since the
          Extended ARP protocol can determine when stations are on both
          rings. However, when a dual MAC, dual attached station is
          speaking to a single attached station, the Extended ARP
          protocol will fail, and stations will default back to using
          standard ARP. Using Standard ARP will guarantee that single
          attached stations and dual MAC, dual attached stations on the
          same IP network, will be able to communicate using only the
          ring that the single attached station is connected to.


          2.2.2.  Case 2: In Wrapped Mode

          Case 2 in wrapped mode is identical to case 1 when in wrapped
          mode. The same arguments detailed in section 2.1.2 apply to
          this case. Thus, this case does not violate IP. However, when
          the ring transitions from wrapped mode back to thru mode,
          stations arp tables may now contain invalid IP to MAC address
          mappings as described in section 3.1.3. To avoid this problem
          the steps in section 3.1.3.1 should be followed.


          2.3.  Case 3: Both Rings Use Same IP Network

          In this case, all FDDI MACs are part of the same IP network
          regardless of which ring the MACs are attached. This
          configuration allows every dual MAC, dual attached station to
          take full advantage of the benefits listed in the
          introduction. This example assumes the absence of bridges.


          2.3.1.  Case 3: In Thru Mode

          While in thru mode, there will be stations on the primary ring
          that can not talk to  stations on the secondary ring without
          the aid of a MAC bridge. Since we assumed that there were no
          bridges between the two rings for this example, the IP network
          is not contiguous, thus the integrity of IP is broken.






          Richard Fox                                          [Page 10]





          Internet Draft         Proxy IP Bridge                 2/10/91


          IP is broken internally to the IP network since not all
          stations within the same IP network can speak to all other
          stations that are part of the same network. A solution that
          fixes this internal network problem is given in section 3.
          This configuration can also result in breaking IP to the
          external world. This external problem is discussed in section
          4.

          In the absence of a solution, it has been proposed that having
          only one IP network per FDDI network is allowed with the
          constraint that all single attached stations being connected
          to the primary ring only. Section 3 should show that this
          constraint is no longer valid and section 5.1 should show why
          this assumption is invalid regardless of this proposal.


          2.3.2.  Case 3: In Wrapped Mode

          In this case when the ring is wrapped, all stations can speak
          to all other stations. In other words the IP network is one
          contiguous network where all stations within the network can
          talk to all other station within the same network.

          An interesting side effect of wrapping, is now stations that
          could not communicate in thru mode, can now communicate. As
          soon as the ring transitions back to thru mode, these stations
          will again, not be able to communicate to one another without
          the aid of a bridge.


          3.  Proxy IP Bridging

          It has been advocated that single attached stations be
          required to all be attached to the primary ring in an FDDI
          environment because of the problem discussed in section 2.3.1.
          This would be a serious limitation that the FDDI community
          feels has been unjustly mandated, since one of the main
          features of FDDI is the flexibility of ring configurations and
          fault recovery. Also, section 5.1 shows why forcing this
          restriction is an INVALID thing to do. In this section, a
          proposal is presented that should solve the problem of section
          2.3.1, such that, both the IP community and the FDDI community
          can fully utilize the technology of having dual rings without
          sacrificing the integrity of IP.






          Richard Fox                                          [Page 11]





          Internet Draft         Proxy IP Bridge                 2/10/91


          The solution defined in this document provides an extension to
          a dual MAC dual attached device attached to the FDDI trunk
          ring by adding simple bridging  support for IP. It does this
          by manipulating Extended ARP[4] and ARP[5] frames in such a
          manner that it bridges IP traffic from one ring to the other
          when necessary to maintain IP integrity. A device that
          provides this type of bridging functionality is termed a proxy
          IP bridge.

          The requirements of a proxy IP bridge can be summarized as the
          following:

            -  respond to Standard ARPs for those stations that are
               single attached stations, such that the Standard ARP is
               received on the ring in which the destination station is
               not attached to. When responding to these Standard ARPs,
               the proxy IP bridge must put in its own MAC address (for
               the MAC attached to the same ring as the source of the
               ARP), and not the MAC address of the end station.

            -  forward all Standard ARPs from single attached stations
               to the other ring, replacing the source MAC address of
               the single attached station with its own MAC.

            -  forwards all IP and ARP frames that are addressed to one
               of its MACs but with an IP address that is not one of its
               own.


          3.1.  Formal Definition of a Proxy IP Bridge

          This section formally defines what comprises a proxy IP bridge
          and details the actions and events of a proxy IP bridge. The
          actions specified are for the proxy IP bridge part of a
          concentrator only. All required functionality of a
          concentrator are still in affect.


          3.1.1.  Definition of ARP Handling

          This section formally defines what comprises a proxy IP bridge
          and details the actions and events of a proxy IP bridge. It is
          assumed that that all dual MAC, dual attached stations
          implement the Extended ARP mechanism[4].  If some dual MAC,
          dual attached stations do not implement the Extended ARP





          Richard Fox                                          [Page 12]





          Internet Draft         Proxy IP Bridge                 2/10/91


          mechanism, the algorithms defined in this paper will still
          work.  However, these stations may not realize the full
          benefits of FDDI as described in the introduction.


          3.1.1.1.  ARP Request

          This section details what the proxy IP bridge does with
          received Extended ARP requests and Standard ARP requests. The
          rules in this section are to be followed as long as the ring
          is in thru mode. If the ring wraps, then section 3.1.3
          applies.

          1. On reception of an Extended ARP, the proxy IP bridge caches
          the source MAC addresses and IP address of the frame in its
          ARP table. The proxy IP bridge does not do anything more with
          an Extended ARP, since by definition, a device that sends an
          Extended ARP is dual attached and can reach the destination
          directly, without the need of any bridging functionality.

          The information contained in all received Extended ARPs, will
          overwrite any previous data stored for the source address[5]
          in the arp table.

          2. On reception of a Standard ARP the proxy IP bridge first
          looks at the source address. The following actions will be
          taken:

            A. If the source MAC address is already in the arp table and
               is marked as a dual MAC dual attached station, then the
               source can reach the destination directly and the proxy
               IP bridge will do nothing further with the ARP.

            B. If the source MAC address is already in the arp table and
               is not marked as a dual MAC dual attached station, then
               the cache entry is updated as in step 2.C.

            C. If the source MAC address is not in the arp cache, then
               the source IP address is added to the arp table with its
               MAC address for the interface that the ARP was received
               on. For the other interface (or ring) the IP address
               should be added to the arp table with the MAC address of
               the proxy IP bridge for that interface.

          3. If the destination of a Standard ARP is in the proxy IP





          Richard Fox                                          [Page 13]





          Internet Draft         Proxy IP Bridge                 2/10/91


          bridge's arp table with an entry for the interface the ARP was
          received on, then the bridge should do the following based on
          the MAC address of the arp entry:

            A. If the MAC address is not of the proxy IP bridge, then
               the source and destination of the ARP is on the same ring
               and the proxy IP bridge will do nothing further with the
               ARP request.

            B. If the MAC address is that of the proxy IP bridge, then
               the source and destination are on different rings, and
               the proxy IP bridge will respond to the arp request using
               its MAC address in the ARP response.

          4. If the destination of a Standard ARP is not in the arp
          table for the interface the arp was received on, but is in the
          arp table for the other interface or ring, then the proxy IP
          bridge will do the following:

            -  Add the destination IP address to the arp table with the
               MAC address of the proxy IP bridge for the interface that
               the Standard ARP was received on.

            -  Respond to the Standard ARP using the newly created arp
               entry.  This means that the the proxy IP bridge will be
               forwarding packets between the two rings so that the two
               single attached stations can communicate.

          5. If the destination of a Standard ARP is not in the arp
          table at all, then the proxy IP bridge will do the following:

            A. The proxy IP bridge will send out an Extended ARP to the
               destination.

            B. If an Extended ARP response is received, then the proxy
               IP bridge will add the response to the arp table and mark
               the entry as being a dual MAC dual attached station.
               Since the destination is connected to both rings, the
               source and destination will be able to directly
               communicate and the proxy IP bridge will do nothing
               further with the original Standard ARP.

            C. If an Extended ARP response is not received, then a
               standard ARP will be sent out on both rings. If a
               response to either standard ARP is received, then the





          Richard Fox                                          [Page 14]





          Internet Draft         Proxy IP Bridge                 2/10/91


               destination IP address and MAC address is added to the
               table for the interface that the ARP response was
               received on.  For the other interface the destination IP
               address is added with the MAC address of the proxy IP
               bridge for that interface. After the destination is added
               to the arp table, go to step 3.

            D. If no response to either of the Standard ARPs from step C
               is received, then the proxy IP bridge will do nothing
               further with the original Standard ARP.


          3.1.1.2.  ARP Response

          When it is determined from Section 3.1.1.1 that the proxy IP
          bridge should respond to a Standard ARP on behalf of a single
          attached station, it should always supply its appropriate MAC
          address in the ARP response packet.  The appropriate MAC
          address is the address of the MAC that is connected to the
          same ring as the source address specified in the Standard ARP
          packet. This way when the source station sends a frame to the
          destination, it will address it to the proxy IP bridge MAC
          interface that will forward the frame to the other ring.


          3.1.2.  Packet Forwarding

          Once a station has received a response to an ARP that
          completes the IP address to MAC address mapping necessary to
          send IP packets, the station will use this MAC address in the
          FDDI frame when sending IP packets to the destination IP
          address. If the MAC address used is not the destination's MAC
          address, but is the address of a proxy IP bridge, then all IP
          packets sent by the source will first be received by a proxy
          IP bridge. When a proxy IP bridge receives a packet with an IP
          address that is not its own, the following steps must be
          taken:

            A. If there is no arp entry for the destination IP address,
               then the proxy IP bridge must arp for the destination
               address as described in section 3.1.1.1 step 5. If this
               fails, then the proxy IP bridge must assume that the IP
               address is not currently reachable and the IP packet will
               be dropped.






          Richard Fox                                          [Page 15]





          Internet Draft         Proxy IP Bridge                 2/10/91


            B. If  the arp entry for the destination contains a MAC
               address that is not one of the proxy IP bridge's for an
               interface, then the proxy IP bridge will forward the IP
               packet to this interface (or ring) replacing its MAC
               address with the destination's cached MAC address.

            C. If no arp entries exist for the destination address even
               after step A, or the MAC address of the proxy IP bridge
               appears in the arp table for the interface the packet is
               being forwarded to, then the IP datagram must be dropped.


          3.1.3.  Thru to Wrap to Thru

          When the ring transitions from Thru to Wrap mode, stations
          that couldn't directly communicate (without the use of the
          proxy IP bridge) can now directly arp for one another and no
          longer need the services of a proxy IP bridge. A proxy IP
          bridge will know when the ring has wrapped when it sends
          either an Extended ARP or a Standard ARP out one interface and
          receives it on it's other interface. When the proxy IP bridge
          has detected that the ring has wrapped, it should do the
          following:

            A. It should stop building its arp table as described in
               sections 3.1.1.1 & 3.1.1.2.

            B. The proxy IP bridge should no longer answer arp requests
               that would cause the proxy IP bridge to forward packets.

            C. The proxy IP bridge should continue to forward all IP
               packets as defined in section 3.1.2, even if this
               requires the proxy IP bridge to send out an ARP to
               discover the destination.

          The reason that the proxy IP bridge should no longer try and
          discover the location of stations as in section 3.1.1.1 is
          threefold. First, as long as the ring is wrapped there is no
          need for the proxy IP bridge to forward any packets, because
          all stations can communicate directly. Secondly, any new arp
          entries will show that there is no need for the proxy IP
          bridge and this extra traffic on the network is unnecessary.
          Thirdly, entries added to the arp table while in wrap mode may
          prove to be incorrect when the station goes back to thru mode,
          since it will have entries with MAC addresses that are valid





          Richard Fox                                          [Page 16]





          Internet Draft         Proxy IP Bridge                 2/10/91


          only  when the ring is in wrap mode but are not valid in thru
          mode.

          The third point of the last paragraph has proven to be a
          problem regardless of whether the algorithms defined in this
          document exist or not.  Consider the network in figure 2. When
          the ring wraps it is quite possible that stations like C will
          have two MACs on the wrapped ring (see figure 4).  When C arps
          for A, A will get two ARP requests, one from each of C's
          MACs[4]. Depending on the order the ARPs are received, A could
          store the MAC of C which is normally not connected to the same
          ring as A is connected to. While in wrap mode this is no
          problem, but as soon as the ring goes back to thru mode, A can
          no longer communicate with C using this arp entry. When A
          tries to communicate with C it must determine if C is down, or
          the MAC address is no longer valid and the destination IP
          address must be re-ARPed. This is not a trivial problem (see
          section 3.1.3.1).
































          Richard Fox                                          [Page 17]





          Internet Draft         Proxy IP Bridge                 2/10/91


                        ---------------               ---------------
                        |             |               |             |
                        | Station A   |               | Station B   |
                        |             |               |             |
                        | IP Subnet X |               | IP Subnet X |
                        -------|-------               -------|-------
                               |                             |
                               |                             |
                 /-------------|-----------------------------|-------------\
                 |                                                         |
                 |                                                         |
                 |                                                         |
                 |                                                         |
                 |                                                         |
                 |                                                         |
                 |                                                         |
                 |                                                         |
                 |                                                         |
                \|/                                                       /|\
                 |                                                         |
                 |                                                         |
                 |                                                         |
                 |                                                         |
                 |                                                         |
                 |                                                         |
                 |                                                         |
                 |                                                         |
                 |                                                         |
                 |                                                         |
                 \------------------------|-----|--------------------------/
                                          |     |
                                          |     |
                                          |     |
                                      ----|-----|----
                                      |             |
                                      | Station C   |
                                      |             |
                                      | IP Subnet X |
                                      ---------------


                                        figure 4








          Richard Fox                                          [Page 18]





          Internet Draft         Proxy IP Bridge                 2/10/91


          3.1.3.1.  Arp Table Flushes

          When the ring goes from wrap to thru, arp tables of both dual
          attached and single attached stations may contain invalid data
          as described in section 3.1.3. This section proposes a set of
          steps that should be taken when the ring transitions from wrap
          mode to thru mode for both proxy IP bridges and single
          attached stations that rely on a proxy IP bridge when in thru
          mode.

          Whenever a proxy IP bridge flushes out an IP address from its
          arp cache, it must remove the entry out of the table for all
          interfaces.


          3.1.3.1.1.  Proxy IP Bridge: Wrap to Thru Transition

          Once the proxy IP bridge determines that the ring is wrapped
          (see section 3.1.3) the proxy IP bridge needs a method of
          determining when the ring transitions back to thru mode so
          that it can fix its arp table and restart building its arp
          table as described in sections 3.1.1.1 & 3.1.1.2.  FDDI
          currently provides no method that a station can rely on to
          determine when the ring transitions from wrap to thru. There
          are many tools that can be used such as the SRF frame[9], PMF
          protocol[9], or NIF protocol[9], but none of these tools are
          guaranteed to be implemented or contain the necessary data.
          Thus, the solution to this problem must come within the IP
          paradigm.

          When a proxy IP bridge determines that the ring is wrapped it
          must do the following:

            1. Discontinue adding new ARP entries since its services are
               not needed for new connections. Also, the proxy IP bridge
               must continue to forward all frames that are addressed to
               it as in section 3.1.2.

            2. Every A-TIME, send out an Extended ARP out both MAC
               interfaces. A-TIME is being suggested to be 30 seconds.
               An Extended ARP is sent out instead of a Standard ARP in
               case another proxy IP bridge has finished step 5 and
               cleared this station out of its arp table. In this case,
               sending an Extended ARP will get the station added to the
               other proxy IP bridge as a dual MAC, dual attached





          Richard Fox                                          [Page 19]





          Internet Draft         Proxy IP Bridge                 2/10/91


               station. Sending out a Standard ARP could result in the
               proxy IP bridge being added to anothers arp table as a
               single attached station, which would be invalid. The
               destination IP address of the Extended ARP should be that
               of the sending proxy IP bridge (itself).

            3. If the proxy IP bridge receives one of the Extended ARPs
               on the interface it wasn't sent out on, then the proxy IP
               bridge knows that the ring is still wrapped and it goes
               back to step 2.

            4. If the Extended ARPs are both received but only on the
               interfaces that they were sent out on, then the proxy IP
               bridge knows that the ring has transitioned back to thru
               mode and proceeds to step 5.

            5. Once the proxy IP bridge has determined that the ring is
               in thru mode once again, it must flush out its arp table,
               since the state of each entry can not be guaranteed to be
               correct.

            6. The proxy IP bridge rebuilds its arp table based on
               sections 3.1.1.1 & 3.1.1.2 and it operates as normal.


          3.1.3.1.2.  Single Attached Stations: Wrap to Thru
          Transition

          Stations, other than proxy IP bridges, on the ring, have no
          guaranteed method of determining when the ring goes from wrap
          to thru mode.  Neither the FDDI standard or the IP standard
          has a standard method guaranteed to determine when a
          configuration of this sort takes place.  In section 3.1.3.1.1
          a method was detailed that allows a proxy IP bridge to
          determine a wrap to thru transition, but this method does not
          work with single attached stations. For stations to make the
          transition from wrap to thru with limited loss of
          connectivity, stations should do the following:

            1. Have a very short ARP entry timeout value. This way after
               a wrap to thru transition, invalid arp entries will
               quickly be aged out of the table.

            2. With a short arp age out value, the station will depend
               on either being able to rebuild deleted arp entries





          Richard Fox                                          [Page 20]





          Internet Draft         Proxy IP Bridge                 2/10/91


               quickly by direct arp responses and by arp responses from
               proxy IP bridges.

          Single attached stations will greatly depend on the ability of
          proxy IP bridges being able to quickly respond to arp
          requests. If this requirement is not met, then stations pay a
          big price for quickly aging out arp entries. If the mean time
          between ring wraps is great, then this requirement will surely
          be met. If the mean time between ring wraps is small, then the
          penalty for aging arp entries will be insignificant to the
          penalty of the ring transition overhead.


          4.  External View of the IP Network

          Section 2 described three IP network configurations given the
          FDDI network in figure 2. The assumption was made that no
          routers were connected to the FDDI network. Case 3 in section
          2.3 describe a scenario of how stations, that are part of the
          same IP network (subnet), can not communicate to each other,
          causing the IP network to appear discontiguous internally in
          the network. The solution in section 3 fixes this problem,
          once again giving the appearance that the IP network is
          contiguous. Now the assumption of no routers must be removed,
          and the cases of section 2 revisited, to see if the external
          view of the IP network is valid.

          For an IP network to have a valid external view, all routers
          that are connected to the IP network must be able to reach all
          stations of that IP network. If a router were to fail in this
          requirement, then stations external to the network could not
          be guaranteed that all stations on the network are reachable,
          even though a router to the network advertises the network as
          reachable.


          4.1.  Dual MAC, Dual Attached Routers

          Adding dual MAC, dual attached routers to figure 2, will not
          cause the external view of the network to become invalid for
          any of the cases listed in section 2. This is true because
          being dual attached allows the router to directly speak to any
          other station that is part of the FDDI ring. Cases 1 & 2
          requires the router to route to two different IP networks.
          Using the information gathered from Extended ARP and Standard





          Richard Fox                                          [Page 21]





          Internet Draft         Proxy IP Bridge                 2/10/91


          ARP allows the router to determine easily, which MAC, of which
          ring, it should use to reach the station.

          Case 3 is similar to cases 1 & 2 except the router routes for
          one IP network and not two as in the other cases.  As in cases
          1 & 2, using Extended ARPs and Standard ARPs is sufficient to
          determine how to reach all stations on the FDDI ring.


          4.2.  Single Attached Routers

          Adding a single attached router to the FDDI ring imposes some
          problems that are not evident with a dual MAC, dual attached
          router. Cases 1 & 2 are still not a problem using the same
          arguments as stated in section 2. Case 1 the router will
          attach to one ring, and all stations that belong to the routed
          IP network can be reached on this ring. Case 2 is the same as
          case 1, except the router can not communicate with all MACs
          that are part of the IP network, since dual MAC, dual attached
          stations will have MACs on both rings, but all IP addresses or
          stations that comprise the IP network are reachable by the
          router.

          If the IP network is represented as in case 3, a single
          attached router will not be able to give a correct external
          view, since it will only be able to communicate with MACs that
          are on the same ring as its own. If there are single attached
          stations on the other ring, there is no way that the router
          will be able to route packets from the external world to this
          station without some bridge like function on the FDDI ring.
          Since a single attached router is really just like any other
          single attached station, using the procedures of section 3
          will allow the single attached router to give a valid external
          view of the IP network that it is routing to, since a proxy IP
          bridge will forward all packets that are received by the
          single attached router, which are destined for the ring that
          the router is not connected to.

          It has been shown that using the algorithms defined in this
          paper maintains IP's integrity both internally to the IP
          network and externally to the rest of the connected internet.









          Richard Fox                                          [Page 22]





          Internet Draft         Proxy IP Bridge                 2/10/91


          5.  Considerations and Issues


          5.1.  Twisted Rings

          A twisted ring is a ring that has A ports connected to A ports
          as described in [9]. This has the affect of mixing the primary
          and secondary ring as defined in [9], if A-A or B-B
          connections did not exist. In other words, single attached
          stations that would appear on the same ring in a normal trunk
          ring with only A-B, and no A-A or B-B peer connections, are no
          longer on the same ring. Thus, mandating that all single
          attached stations must be put on the primary ring only, isn't
          enough to solve the IP integrity problem, since a twisted ring
          may end up with single attached stations on both rings. Any
          assumption or protocol that assumes all single attached
          stations will be on the same ring is INVALID.

          The algorithms defined in this document makes no assumptions
          on how the ring is composed. Twisted rings are handled the
          same way as normal rings  and present no problems to the
          algorithms defined in this document.


          5.2.  Network Management Consideration

          Since the algorithms defined in this paper may not always be
          required, the procedures defined in this document should be
          able to be turned off by management control. Also, permanent
          ARP entries should be allowed to be added to the proxy IP
          bridge's arp table using management control.  This way, an
          administrator can add a priori knowledge to the proxy IP
          bridge to help speed operation when connections to certain
          machines happen quite often.


          5.3.  Extended ARP

          This paper assumes the use of the Extended ARP protocol[4].
          The protocol is still under design and may change from its
          current implementation. It is being assumed that this document
          can make use of the Extended ARP protocol even as its
          definition changes. If this assumption is no longer valid,
          then this document will be updated to state what is needed for
          this protocol to work.





          Richard Fox                                          [Page 23]





          Internet Draft         Proxy IP Bridge                 2/10/91


          5.4.  Performance Considerations

          A performance consideration to consider is the extra buffering
          that may be required for the proxy IP bridge if it stores
          Standard ARPs while it tries to discover whether it should
          respond to the ARP or not. If this is a serious problem, then
          an implementation suggestion would be to drop the Standard ARP
          when trying to discover the destination address of the ARP.
          When the source of the ARP doesn't receive a response, it may
          retry the ARP and at that time the proxy IP bridge should have
          enough information to know if it needs to respond or not. This
          makes the assumption that arps are resent more than once and
          at a rate no faster than 1 second[12]. Also section 5.2 gave
          an optional optimization that would help reduce the amount of
          ARP buffering required by the proxy IP bridge.


          5.5.  Support of Broadcast (Multicast) Packets

          The algorithms defined in this paper have solved the
          connectivity problem of two IP stations that are part of the
          same IP subnet, but are on different rings. However, there is
          more to an IP network than ARP packets and IP packets that are
          addressed to single stations. Many IP protocols are based on
          IP broadcast (or multicast) packets. The proxy IP bridge must
          be able to forward these packets when appropriate; otherwise,
          single attached stations might not be able to fully
          communicate with other devices using currently defined
          protocols based on broadcasts.

          If broadcast packets are forwarded the same way ARP packets
          are forwarded, then each proxy IP bridge will forward all
          broadcast packets due to the lack of an individual destination
          IP and MAC address in the packet. This is clearly an
          undesirable solution, since stations may receive multiple
          copies of every broadcast packet. This section defines an
          algorithm on how broadcast (or multicast) packets should be
          handled.  5.5.1 Forwarding of IP Broadcast (Multicast) Packets

          There should only be one proxy IP bridge responsible for
          forwarding broadcast packets sent by single attached stations
          per FDDI network. When a proxy IP bridge receives an IP
          broadcast packet it does the following:

            A. Determines if it is forwarding broadcast frames. If NO





          Richard Fox                                          [Page 24]





          Internet Draft         Proxy IP Bridge                 2/10/91


               then the packet is dropped and no more processing of the
               packet is required.

            B. Otherwise, the source address is looked up in the arp
               table. If the source is in the table and is marked as
               dual MAC dual attached, then no forwarding is required
               and the packet is dropped. If the source address is in
               the table but not marked as dual MAC dual attached, then
               the frame is forwarded to the other ring if the MAC
               address in the arp table for the interface the packet was
               received on is not that of the proxy IP bridge's. If the
               MAC address is that of the proxy IP bridge, then the
               broadcast packet has already been forwarded and must not
               be forwarded a second time.

            C. If the source address is not in the arp table, then the
               proxy IP bridge sends out an Extended ARP. If it gets a
               response, then the proxy IP bridge will add to response
               to the arp table and mark the entry as a dual MAC dual
               attached station. Since the station responded to the
               Extended ARP the broadcast packet does not need to be
               forwarded and may be dropped.

            D. If no response to the Extended ARP is received, then the
               station is single attached, and the proxy IP bridge will
               forward the broadcast packet. The source address should
               be cached in the arp table as in section 3.1.1.1 step
               2.C.


          6.  Conclusion

          This paper has presented a solution that allows many IP
          network configurations to be mapped onto all legal FDDI
          network configurations, such that, the integrity of IP is
          maintained without placing any limitations on FDDI. Without
          the use of some algorithms similar to those defined in this
          document, no guarantees on the integrity of IP can be made.
          There are many tools that could be used to optimize the
          solution provided, but the solution as detailed depends on no
          optionally defined protocol and requires no protocol changes
          to IP or FDDI.








          Richard Fox                                          [Page 25]





          Internet Draft         Proxy IP Bridge                 2/10/91


          7.  References


          [1]  American National Standard, Fiber Distributed Data
               Interface (FDDI) -- Token Ring Physical Protocol (PHY),
               ANSI X3.148-1988.


          [2]  American National Standard, Fiber Distributed Data
               Interface (FDDI) -- Token Ring Media Access Control, ANSI
               X3.139-1987.


          [3]  Postel, J.B., Internet Protocol, RFC 791 September 1981.


          [4]  Brown, C. and Bagnall, D., ARP Extensions for Single IP
               Subnet FDDI LANs, Draft RFC 1990.


          [5]  Plummer, D., An Ethernet Address Resolution Protocol, RFC
               826, November 1982.


          [6]  Postel, J.B. and Reynolds, J.K., Telnet Protocol
               specification, RFC 854, May 1983.


          [7]  Postel, J.B., File Transfer Protocol specification, RFC
               765, June 1980.


          [8]  Postel, J.B., DoD standard Transmission Control Protocol,
               RFC 761, January 1980.


          [9]  Draft Proposed American National Standard, Fiber
               Distributed Data Interface (FDDI) -- Token Ring Station
               Management (SMT), ANSI X3.T9.5- 1990 Revision 6.2.


          [10] McKenney, P.E., Broadcast Storms, Nervous Hosts, and Load
               Imbalances, Information Sciences and Technology Center,
               SRI International, October 1988.






          Richard Fox                                          [Page 26]





          Internet Draft         Proxy IP Bridge                 2/10/91


          [11] Mitchell, C. and Quarterman, J.S., Using ARP to Implement
               Transparent Subnet Gateways, RFC 1027, October 1987.


          [12] Braden, R.T., Requirements for Internet hosts -
               communication layers, RFC 1122, October 1989.


          [13] P802.1d MAC Bridges, IEEE Project 802, July 1989









































          Richard Fox                                          [Page 27]





          Internet Draft         Proxy IP Bridge                 2/10/91


          Table of Contents


          1 Introduction ..........................................    1
          2 Configurations of Single  Attached  Stations  in  an
               IP/FDDI Environment ................................    8
          2.1 Case 1: Different Networks ..........................    8
          2.1.1 Case 1: In Thru Mode ..............................    9
          2.1.2 Case 1: In Wrapped Mode ...........................    9
          2.2 Case 2: Dual stations using 1 IP Network ............    9
          2.2.1 Case 2: In Thru Mode ..............................    9
          2.2.2 Case 2: In Wrapped Mode ...........................   10
          2.3 Case 3: Both Rings Use Same IP Network ..............   10
          2.3.1 Case 3: In Thru Mode ..............................   10
          2.3.2 Case 3: In Wrapped Mode ...........................   11
          3 Proxy IP Bridging .....................................   11
          3.1 Formal Definition of a Proxy IP Bridge ..............   12
          3.1.1 Definition of ARP Handling ........................   12
          3.1.1.1 ARP Request .....................................   13
          3.1.1.2 ARP Response ....................................   15
          3.1.2 Packet Forwarding .................................   15
          3.1.3 Thru to Wrap to Thru ..............................   16
          3.1.3.1 Arp Table Flushes ...............................   19
          3.1.3.1.1 Proxy IP Bridge: Wrap to Thru Transition ......   19
          3.1.3.1.2 Single Attached Stations: Wrap to Thru Tran-
               sition .............................................   20
          4 External View of the IP Network .......................   21
          4.1 Dual MAC, Dual Attached Routers .....................   21
          4.2 Single Attached Routers .............................   22
          5 Considerations and Issues .............................   23
          5.1 Twisted Rings .......................................   23
          5.2 Network Management Consideration ....................   23
          5.3 Extended ARP ........................................   23
          5.4 Performance Considerations ..........................   24
          5.5 Support of Broadcast (Multicast) Packets ............   24
          6 Conclusion ............................................   25
          7 References ............................................   26













          Richard Fox                                          [Page 28]