Re: [rbridge] merging draft-tissa-trill-oam anddraft-ietf-trill-rbridge-oam

["Tissa Senevirathne (tsenevir)" <[e-mail]>]

Firstly there are few critical high-level points that needed to be stated

 

1.      OAM cannot be just detecting Liveliness and Path discovery. It is about Operational aspects of networks which include but not limited to Detecting Liveliness, Path Discovery, Fault finding, Fault Isolation etc. OAM is never constrained to ping and trace route.  Look at where OAM is heading to in MPLS and MPLS-TP.

2.      TRILL cannot be compared 1-1 with Ethernet or IP. TRILL encapsulate the user payload with the TRILL header and utilize that to forward using TRILL header and some parts of user payload (e,g, ECMP and mcast forwarding and pruning). TRILL utilize multiple paths and multi destination tress. In that regard it is similar to a tunneling protocol. Most logical comparison of TRILL OAM is RFC 4379 MPLS OAM If one takes a look at TRILL evolution, multipathing is the fundamental point of leveraging IP technology within bridging. If one cannot perform OAM functions to verify all the paths, it is a moot point to develop OAM functions.

3.      What is fundamentally required in TRILL OAM is to have OAM framework that allows to keep OAM frames as close as possible to data packet. ICMP is not utilized as a Ping or Traceroute. Motivation here is to provide such framework in backward compatible manner without introducing additional restrictions. As an example, channel draft, which define the framework of OAM channel you are proposing require to use RBridge critical alert bit. RBridges that are not supporting that cannot claim it is Channel capable. Additionally, Channel header follows right after the inner MAC header. This limit including any L3 frames. It is a common practice to select ECMP based on IP flow information and multicast prune based on IP address. (Please see rfc6326 bis for Group IP Address sub-TLV.  draft-ietf-trill-rbridge-channel, which you are proposing to use do not provide that flexibility. In your response below you have proposed several alternatives but in our mind they do not achieve what is required in OAM. Please see specific responses embedded.

4.      Current suggestion from WG members are to utilize RFC 4379 framework, as that allow to utilize lots of existing knowledge, implementation and user experience from MPLS OAM.

5.      The embedded message channel is utilized so receiver can identify OAM frames from data frames and take appropriate actions.  Without limiting what payloads can be include in the OAM packet. 

6.      You have proposed to use incremental TTL value method for multicast with invalid IP checksum. Firstly given the tree depth is different in different branches, this will invariably leak packets out. And will lead to incrementing IP invalid packet count on end stations. In hosting application this may generate unwanted support calls, and something that should not be done. In draft-tissa-trill-oam-03 section 6.3 propose to use this exact method only for unciast. Additionally because we can identify OAM frames from real data with TTL expiry due to embedded message channel, we notify that as the last hop bridge and user would not generate N+1 TTL packet.

7.      Below you have proposed to utilize multicast for path trace (trace route).  Whole purpose of OAM is data plane integrity verification. Trace route in particular, used for fault isolation. Multicast use a different forwarding logic than that of unicast. Use of Multicast to trace route unicat is not going to achieve what is needed and negate the use of OAM. Additionally there are several issues such multicast tree may not cover all links in the campus and so on. Please see specific response embedded in the relevant section

8.      You are proposing to merge the two drafts and I do not see a commonality based on your responses below.

 

Please see more specific answers below

 

From: [e-mail] <mailto:[e-mail]>  [mailto:[e-mail]] On Behalf Of Santosh Rajagopalan
Sent: Tuesday, March 06, 2012 8:49 AM
To: [e-mail] <mailto:[e-mail]> 
Cc: Santosh Rajagopalan
Subject: [rbridge] merging draft-tissa-trill-oam anddraft-ietf-trill-rbridge-oam

 

 

 

Hi,
We currently have two competing approaches to TRILL OAM, draft-ietf-trill-rbridge-oam and draft-tissa-trill-oam. Both have their merits, which calls for a revised draft with the best ideas from both.

draft-tissa-trill-oam has a dependancy on IP/ICMP/ARP etc.  The major reason for this dependency seems to be for multipath path discovery, and I have some suggestions as to how to fix this in draft-ietf-trill-rbridge-oam without using IP. draft-tissa-trill-oam also has several features which should be rightly queried from a MIB.

I think the OAM draft should concentrate on facilitating actions that would be difficult or impossible to perform in its absence. So querying counters has no place here, because you could easily figure them out by doing an SNMP query or by logging into the switch. I also think it's worthwhile to look at the features traditionally supported in ethernet OAM and to see if they have a place here.

Using draft-ietf-trill-rbridge-oam as a basis:

1) Ping/loopback: draft-ietf-trill-rbridge-oam handles this well enough (it does need a multicast addition). The purpose of ping is to verify connectivity between two rbridges, not to discover the various ways of connecting the two.  One suggestion I have here is to be able to include a cookie in the echo request message which gets reflected back by the receiver. This cookie could be used to include the sender's timestamp in the ping message. The response could then be used to measure round-trip path latency and jitter. If the sender and receiver's clocks are synchronized using some external mechanism then this can also be used to measure one-way delay and jitter. This is a feature borrowed from ethernet OAM.

[Answer] Seem the paragraph is contradicting itself. Firstly you say ping is to detect the liveliness. Then you say it can be detected to measure jitter, delay etc. If there are 10 ECMP paths and if you can only measure latency of one it does not give true picture of the information. 


2) Traceroute: IP traceroute doesn't rely on icmp for the outgoing packet (you could use ICMP, but that's a different matter). In fact, UNIX implementations usually just send UDP packets with port numbers in the 33K range, with appropriate TTL values. Note that traceroute in the IP world gives you "a" path to a destination, not "all possible paths". However, being able to enumerate all possible paths, as well as to pinpoint the particular path a flow would take are good to have features. 

[Answer] This is not a good to have feature, but a MUST, because it is fundamental to fault isolation. Take a look at MPLS OAM and its problem statement. It is no different here. If you model the way you say, running ICMP ping between RBridge suffice the need. Each Rbridge has IP stack anyway. Also, in Linux we do have –I options to trigger ICMP.


Here're the modifications needed in draft-ietf-trill-rbridge-oam to achieve this:

[Answer] All of the below is based on using multicast trees. Are you suggesting to use multicast to do trace route. Whole purpose of OAM is data plane integrity verification. Trace route in particular, used for fault isolation. Multicast use a different forwarding logic than that of unicast. Use of Multicast to trace route unicat is not going to achieve what is needed. Secondly, multicast trees will not cover all links. Case in point is having a single tree.

	a.	Multicast unpruned tree discovery: you can discover all existing unpruned multicast trees by setting the erbridge value of an rbridge channel message to the respective roots, M=1 and the dmac to "all-egress-rbridges". Just like regular ping, you can send this as an echo request message with incrementing ttl values, and the ttl expiry messages you get will tell you what the unpruned tree looks like. Changing the egress_rbridge will paint you a picture of the different trees.

[Answer] First reaction to this was am I reading this correctly. After reading this one more time it appeared I am reading it correctly. There are three main issues: 1. There can be 1000 of RBRidges, and getting a response from all of them going to overwhelm. Probably choke the CPU of the requester. 2. When you do incremental TTL value, it will leak out at the last RBridge. 3. Tree does not include all paths. Assume you have 1 tree, (Which is a perfectly a valid configuration), how are you going to cover links not in the tree.

	b.	Multicast pruned tree discovery: just do a), but replace the "all-egress-rbridges" multicast address in the inner dmac with the mac address of the multicast group. As you vary the egress_rbridge field, this will enumerate all possible pruned trees for that group. Note that draft-ietf-trill-rbridge-oam uses draft-ietf-trill-rbridge-channel, which requires the inner dmac to be all-egress-rbridges. However, the ethertype field can be used by itself to classify an OAM packet, and I am advocating that the inner dmac requirement in draft-ietf-trill-rbridge-channel be relaxed to allow for alternative inner dmacs where needed.

[Answer] Fundamental problem:. Prune tree works based on VLAN and MAC address or VLAN and IP address. Are you suggesting to try all VLAN all address here ?

	c.	Multicast pruned tree discovery for a particular flow: I recommend building a packet with an empty payload which looks like the actual flow (with the header fields from the flow being tested instead of the rbridge channel header). The IP address, mac address, L4 headers etc all match with that of the flow you're trying to debug. However, to ensure that any leaked packets get dropped, set the IP checksum to be invalid (the IP checksum is never used for the ecmp hash). Now send out successive packets with the ttl field in incrementing values. Note that you're sending a "pseudo-real" packet into the network and relying on received ttl expiry messages to figure out the path.

[Answer] Aha you are now acknowledging IP is used for ECMP hash. Now I am wondering how Channel header can facilitate that. The concept of utilizing invalid checksum is already covered in draft-tissa-trill-oam-03 Section 6.3. Secondly using TTL in multicast is a dangerous thing, as tree depth can be different and packets can leak out. You may not want to trigger invalid IP checksum counter on your hosting servers. As that may generate additional support calls. Checksum invalid SHOULD only be used to protect against unlikely case packets leaking out. Not the default. 

Question: Are you suggesting to include IP Multicast packets in the native form here or encap within channel header ? 

	d.	Unicast path discovery: the erbridge ID is that of the switch you're trying to ping. Let's send this echo request message with a fake inner DMAC. Construct the dmac so that the U/L bit is 1 and the I/G bit is 0 (basically, its a local scope unicast mac). Set the last 32 bits to a random value. This is a fake mac address, but that's ok because the network routes only on the erbridge anyway. The M bit in the TRILL header is 0. Send this as an echo request message with incrementing ttl values. This gives you "a" path from the source to the destination rbridge. Increment the dmac used earlier, and repeat. All rbridges hash at least on the inner dmac, so this will allow you to enumerate all possible paths from source to destination.

[Answer] You are making big assumption here. Firstly hash is not defined in any standard and it is implementation depended. Some devices may use SMAC:DMAC VLAN combination, Some may use IP flow information. May be low ends use just DMAC. In anycase selecting Random values are not going to gurentee a covering all paths. Secondly one may not even know how many ECMP options available on an RBridge 3 hops away. This is the reason more elaborate methods are required RFC 4379 explain How this can be done in MPLS. Based on the feedback received during the last WG session in Taipei, we have also enhanced to included this. Please see section 9.9. This needed to be done in more systematic manner rather than randomly generating DMAC. DMAC means Destination MAC. SMAC means Source MAC

	e.	Unicast path discovery for a particular flow: Similar to what we did for multicast, construct a real packet of length 0. The IP address, mac address, L4 headers etc all match with that of the flow you're trying to debug. However, to ensure that any leaked packets get dropped, set the IP checksum to be invalid. Now set the ttl field in incrementing values. Note that you're sending a "pseudo-real" packet into the network and relying on received ttl expiry messages to figure out the path.

[Answer] This same concept is in section 6.3 of draft-tissa-trill-oam.  

Note that doing this eliminates the reliance on IP/ICMP, while preserving the objectives of draft-tissa-trill-oam. The fake packets generated are technically not OAM packets, just suggestions on how to achieve a certain objective.


3) Fault notification: although neither draft talks about this, this is a part of the ethernet OAM feature set. I think draft-ietf-trill-rbridge-oam needs to mention that the expectation is that BFD will work over the rbridge channel to achieve this.

4) draft-tissa-trill-oam  has a big section on TTM. The equivalent in Ethernet OAM is performance management by checking counters and send/receive delays, etc. We can cover send/receive delay/jitter measurements using ping (as described earlier). I frankly don't see the benefit to including remote discovery of counters in this OAM proposal. The administrator can always just login to the rbridge of interest and get a read-out of the counters directly, or use snmp remotely. We shouldnt be replicating functionality which is better done elsewhere, and I think this entire section on TTM in draft-tissa-trill-oam should be taken out.





[Answer] Your understanding is incorrect. TTM  is utilized not for counters but to ensure that specific flows are reaching specifc devices. This is included as part of feedback received from the WG in Tapei. This is an adaptation of Data Driven CFM .


5) I don't see any benefits to DRB/AF discovery OAM messages. This is probably done most easily by logging into the rbridge in question and doing a "show l2 isis", or extending the SNMP MIB. 





[Answer] How do you know which device to login to use show commands/


6) draft-tissa-trill-oam has a section on MAC discovery (basically, being able to query all rbridges to see what their rbridge<->mac binding is). This is again something that should be best queried using SNMP - this is a classic MIB walk case.

[Answer] How do you know which device is having the MAC so you can do the walk ?

7) Address binding verification should be out of scope for a TRILL OAM draft, and besides, should just be queried at the rbridge with the CLI.

[Answer] Firstly how do you know which device to login to use CLI. Secondly, anything that improve usability and experience of users can be always included.

8) End-Station Attachment Point Discovery: should be out of scope of this draft, and is too vendor specific (it uses VN-tags).





[Answer] Terminology can be changed to include what is defined in VXLAN, NVGRE etc. The case in point is when there are other overlay technologies how to identify different address association. Certain VM may use the same MAC address to associate with different VXLAN segments.


best,
Sunny Rajagopalan

_______________________________________________
rbridge mailing list
[e-mail]
http://mailman.postel.org/mailman/listinfo/rbridge

 

_______________________________________________
rbridge mailing list
[e-mail]
http://mailman.postel.org/mailman/listinfo/rbridge