Re: [rbridge] merging draft-tissa-trill-oam anddraft-ietf-trill-rbridge-oam

[Santosh Rajagopalan <[e-mail]>]

I think that we're starting to talk past each other. May I suggest taking 
each of the ping/traceroute situations on a case-by-case basis to see how 
each proposal handles it? 

Basically, can one of the authors summarize the approach in 
draft-tissa-trill-oam for ping/traceroute to discover:

a. multidestination unpruned trees.

b. multidestination pruned trees (per multicast group).

c. tracing the path taken by a real multicast flow.

d. unicast path discovery for each multipath

e. unicast path discovery for real flow.

I understand this will be a restatement of information that's present in 
draft-tissa-trill-oam, and I apologize for that. I'm looking for a more 
concise presentation of the approach. I am having trouble arranging the 
information in draft-tissa-trill-oam into the categories I've suggested 
above. You can use as a template the email I sent out originally starting 
this thread.

As for the discussion below about whether properties should be queried via 
SNMP or ICMP OAM, I would simply suggest that we make a choice one way or 
the other. After all, if you add the ability to query mac addresses, DRBs 
etc to OAM (and these are already present in draft-ietf-trill-rbridge-mib
), then why stop there? Why not incorporate everything in the SNMP MIB 
into the ICMP query framework? After all, if SNMP is truly "expensive and 
intensive", why stop with DRB queries? I would then suggest doing away 
with the trill mib and using icmp as the query mechanism for all 
properties. Really, I'm ok either way, but I think we should pick one so 
that people don't end up implementing the same thing twice.

--
Sunny Rajagopalan




From:   Sam Aldrin <[e-mail]>
To:     Santosh Rajagopalan/Santa Clara/IBM@IBMUS
Cc:     "[e-mail]" <[e-mail]>
Date:   03/13/2012 11:59 AM
Subject:        Re: [rbridge] merging draft-tissa-trill-oam 
anddraft-ietf-trill-rbridge-oam
Sent by:        [e-mail]



See inline for my response to your questions.

-sam

Sent from my iPad

On Mar 12, 2012, at 3:06 PM, Santosh Rajagopalan <
[e-mail]> wrote:

1) Let's say that you are trying to find all possible paths between two 
rbridges (or liveliness) for a)unicast, b) multicast (unpruned) and c) 
multicast (pruned). My proposal  below will let you discover all three, 
without needing IP.  Since we're using the rbridge channel ethertype, 
rbridges can trap these and prevent them from getting to hosts at the 
edge. Note that we rely on unicast forwarding of rbridge channel packets 
for unicast discovery and multicast forwarding of rbridge channel packets 
for multicast discovery. This happens naturally in any trill compliant 
switch. 
%sam - Discovering all of the paths is one part and discovering a path a 
specific flow takes is another problem. What we are trying to say is 
channel draft does not allow the full flexibility, for Unicast and 
multicast combinations. As was answered in the original email by Tissa, 
that multicast pruning happens with IP VLAN combinations with channel 
drafts such things are not possible. IP invalid checksum method has danger 
of triggering invalid packet counters  and lead to administration 
nightmare in hosting providers. We encourage you to check this with 
providers. The same was attempted while doing OAM for PW's etc, and 
received a strong no-no.

2) Let's say that you already have the information from 1) above, but you 
are trying to figure out which of the many paths you know of will be used 
for a particular flow. Now, draft-tissa-trill-oam assumes that the inner 
ethernet header is never used for hashing for an IP flow. It assumes that 
as long as you keep the L3 and L4 fields the same as that of the original 
packet, hardware will hash to the same values as that from the original 
flow, even though the L2 header is different ("6.2. Identification of IP 
Flows" in  draft-tissa-trill-oam ). I do not think this is a valid 
assumption to make. The only way to make sure your probe packet takes the 
same path as a regular packet is to make it identical to the original 
packet, all the way including the L2 headers. The IP checksum is never 
used for hashing, so you can invalidate it to make sure such packets get 
dropped at the host. This is the approach I was advocating for doing path 
discovery for a particular flow. 
%sam- You have not read section 6..3 of draft-tissa-trill-oam, (Tissa 
pointed this out in his earlier response as well). Section 6.3 is all 
about using unmodified header. Please read section 6.3. Also this method 
cannot be used for Multicast as answered above will trigger invalid packet 
count.

3) The only hash behavior we can be sure of is that if you give any asic a 
non-IP packet, it will do load balancing using the only available fields, 
which is ethernet smac + dmac. I'm exploiting this property to do path 
discovery (the rbridge ethertype makes asics treat these packets as 
non-IP). Therefore, if you vary the inner dmac, you can exercise all the 
available paths. Paths you discover using this approach are applicable to 
both IP and non-IP traffic, since trill routes are common for both. The 
only hash assumption I've made is that L2 unknown ethertype (rbridge 
channel)  packets get load balanced using the inner dmac/smac. I am making 
no assumption about the fields used by the hardware for hashing IP 
packets. 
%sam- Are you suggesting ingress node to randomely change the DMAC, Point 
is how you going to randomly change a DMAC and ensure it is taking a 
specific path. ? 

4) People adopting TRILL are more likely to come from an ethernet 
background than from an MPLS background. For our target audience, it's 
important for OAM to look like what they're already familiar with, which 
is ethernet OAM. TRILL competes with SPB for the same space, and note that 
they make a virtue of being ethernet OAM compatible. Do many people on 
this list feel MPLS OAM is the way to go for us? 
%sam - that is your assumption, not necessary that others share the same 
view. OAM, irrespective of different layers, share common characteristics. 
Reusing what we learnt in other areas is very useful. Having said that, 
folks who use OAM do not see how it is implemented, rather it is for the 
implementor who cares for it. If you go by your logic, mpls OAM is not 
right because it uses IP model and mpls is not IP.
Trill as such has evolved by bringing ip and bridging technologies 
together, with best of both in each category. Just because one likes 
Ethernet OAM, doesn't mean, anything to do with Ethernet , should always 
do Ethernet OAM. The same argument was used for mpls TP OAM as well. We 
all know the result.
The proposal here is to use the proven methodology and is working. If it 
is solving problem, why reinvent the wheel all over again. Better make use 
of it. If the solution cannot work, then we can discuss about those 
points, rather than predefining the solution to fit based on what one 
feels.
As far as Qbp goes, the concept of flows and it's identification is 
different. You cannot just compare as is for TRILL. If you want, we can 
discuss in that list, to get complete perspective.

5) I'm assuming that when operators lay out a network, they have access to 
a picture that shows the interconnects (or at least have some management 
tool speaking snmp that does). So when they need to discover the DRB on a 
multi-access link, then can send an SNMP query to the rbridges connected 
on that link to get the DRB information. This also goes for finding the 
mac addresses on a given rbridge, finding the address bindings at a 
particular rbridge etc. All of these "property query" type messages are 
exactly what SNMP was made for. Some things like finding the vn-tag, vxlan 
ID, nvgre ID etc are within the domain of SNMP but out of scope for TRILL. 
What is trill-specific about these? 
%sam - if everything is available through Snmp, even performing OAM 
functions is redundant. Basing a model on Snmp is expensive and intensive. 
If we go by the argument of yours, Ethernet OAM, why do you need to export 
performance stats in band, when you can query the same using snmp?
My answer is simple. If one feels the need to export data or query data, 
we provide a mechanism without relying solely on SNMP. While we find it 
useful, everyone may not, that doesn't mean, it is not useful. If a vendor 
chooses not to implement, it is fine.

I haven't responded to any statements below which are based on a 
misunderstanding of my original email, please re-review it and I'll be 
happy to answer questions on it. 
I do not see what was not understood. You can clarify if something was not 
addressed right.

Sam

best regards, 
Sunny Rajagopalan 




From:        "Tissa Senevirathne (tsenevir)" <[e-mail]> 
To:        <[e-mail]> 
Date:        03/08/2012 09:28 AM 
Subject:        Re: [rbridge] merging draft-tissa-trill-oam 
anddraft-ietf-trill-rbridge-oam 
Sent by:        [e-mail] 



Firstly there are few critical high-level points that needed to be stated 
  
1.      OAM cannot be just detecting Liveliness and Path discovery. It is 
about Operational aspects of networks which include but not limited to 
Detecting Liveliness, Path Discovery, Fault finding, Fault Isolation etc. 
OAM is never constrained to ping and trace route.  Look at where OAM is 
heading to in MPLS and MPLS-TP. 
2.      TRILL cannot be compared 1-1 with Ethernet or IP. TRILL 
encapsulate the user payload with the TRILL header and utilize that to 
forward using TRILL header and some parts of user payload (e,g, ECMP and 
mcast forwarding and pruning). TRILL utilize multiple paths and multi 
destination tress. In that regard it is similar to a tunneling protocol. 
Most logical comparison of TRILL OAM is RFC 4379 MPLS OAM If one takes a 
look at TRILL evolution, multipathing is the fundamental point of 
leveraging IP technology within bridging. If one cannot perform OAM 
functions to verify all the paths, it is a moot point to develop OAM 
functions. 
3.      What is fundamentally required in TRILL OAM is to have OAM 
framework that allows to keep OAM frames as close as possible to data 
packet. ICMP is not utilized as a Ping or Traceroute. Motivation here is 
to provide such framework in backward compatible manner without 
introducing additional restrictions. As an example, channel draft, which 
define the framework of OAM channel you are proposing require to use 
RBridge critical alert bit. RBridges that are not supporting that cannot 
claim it is Channel capable. Additionally, Channel header follows right 
after the inner MAC header. This limit including any L3 frames. It is a 
common practice to select ECMP based on IP flow information and multicast 
prune based on IP address. (Please see rfc6326 bis for Group IP Address 
sub-TLV.  draft-ietf-trill-rbridge-channel, which you are proposing to use 
do not provide that flexibility. In your response below you have proposed 
several alternatives but in our mind they do not achieve what is required 
in OAM. Please see specific responses embedded. 
4.      Current suggestion from WG members are to utilize RFC 4379 
framework, as that allow to utilize lots of existing knowledge, 
implementation and user experience from MPLS OAM. 
5.      The embedded message channel is utilized so receiver can identify 
OAM frames from data frames and take appropriate actions.  Without 
limiting what payloads can be include in the OAM packet. 
6.      You have proposed to use incremental TTL value method for 
multicast with invalid IP checksum. Firstly given the tree depth is 
different in different branches, this will invariably leak packets out. 
And will lead to incrementing IP invalid packet count on end stations. In 
hosting application this may generate unwanted support calls, and 
something that should not be done. In draft-tissa-trill-oam-03 section 6.3 
propose to use this exact method only for unciast. Additionally because we 
can identify OAM frames from real data with TTL expiry due to embedded 
message channel, we notify that as the last hop bridge and user would not 
generate N+1 TTL packet. 
7.      Below you have proposed to utilize multicast for path trace (trace 
route).  Whole purpose of OAM is data plane integrity verification. Trace 
route in particular, used for fault isolation. Multicast use a different 
forwarding logic than that of unicast. Use of Multicast to trace route 
unicat is not going to achieve what is needed and negate the use of OAM. 
Additionally there are several issues such multicast tree may not cover 
all links in the campus and so on. Please see specific response embedded 
in the relevant section 
8.      You are proposing to merge the two drafts and I do not see a 
commonality based on your responses below. 
  
Please see more specific answers below 
  
From: [e-mail] [mailto:[e-mail]] On 
Behalf Of Santosh Rajagopalan
Sent: Tuesday, March 06, 2012 8:49 AM
To: [e-mail]
Cc: Santosh Rajagopalan
Subject: [rbridge] merging draft-tissa-trill-oam 
anddraft-ietf-trill-rbridge-oam 
  
  
  
Hi,
We currently have two competing approaches to TRILL OAM, 
draft-ietf-trill-rbridge-oam and draft-tissa-trill-oam. Both have their 
merits, which calls for a revised draft with the best ideas from both.

draft-tissa-trill-oam has a dependancy on IP/ICMP/ARP etc.  The major 
reason for this dependency seems to be for multipath path discovery, and I 
have some suggestions as to how to fix this in 
draft-ietf-trill-rbridge-oam without using IP. draft-tissa-trill-oam also 
has several features which should be rightly queried from a MIB.

I think the OAM draft should concentrate on facilitating actions that 
would be difficult or impossible to perform in its absence. So querying 
counters has no place here, because you could easily figure them out by 
doing an SNMP query or by logging into the switch. I also think it's 
worthwhile to look at the features traditionally supported in ethernet OAM 
and to see if they have a place here.

Using draft-ietf-trill-rbridge-oam as a basis:

1) Ping/loopback: draft-ietf-trill-rbridge-oam handles this well enough 
(it does need a multicast addition). The purpose of ping is to verify 
connectivity between two rbridges, not to discover the various ways of 
connecting the two.  One suggestion I have here is to be able to include a 
cookie in the echo request message which gets reflected back by the 
receiver. This cookie could be used to include the sender's timestamp in 
the ping message. The response could then be used to measure round-trip 
path latency and jitter. If the sender and receiver's clocks are 
synchronized using some external mechanism then this can also be used to 
measure one-way delay and jitter. This is a feature borrowed from ethernet 
OAM. 
[Answer] Seem the paragraph is contradicting itself. Firstly you say ping 
is to detect the liveliness. Then you say it can be detected to measure 
jitter, delay etc. If there are 10 ECMP paths and if you can only measure 
latency of one it does not give true picture of the information. 

2) Traceroute: IP traceroute doesn't rely on icmp for the outgoing packet 
(you could use ICMP, but that's a different matter). In fact, UNIX 
implementations usually just send UDP packets with port numbers in the 33K 
range, with appropriate TTL values. Note that traceroute in the IP world 
gives you "a" path to a destination, not "all possible paths". However, 
being able to enumerate all possible paths, as well as to pinpoint the 
particular path a flow would take are good to have features. 
[Answer] This is not a good to have feature, but a MUST, because it is 
fundamental to fault isolation. Take a look at MPLS OAM and its problem 
statement. It is no different here. If you model the way you say, running 
ICMP ping between RBridge suffice the need. Each Rbridge has IP stack 
anyway. Also, in Linux we do have –I options to trigger ICMP. 

Here're the modifications needed in draft-ietf-trill-rbridge-oam to 
achieve this: 
[Answer] All of the below is based on using multicast trees. Are you 
suggesting to use multicast to do trace route. Whole purpose of OAM is 
data plane integrity verification. Trace route in particular, used for 
fault isolation. Multicast use a different forwarding logic than that of 
unicast. Use of Multicast to trace route unicat is not going to achieve 
what is needed. Secondly, multicast trees will not cover all links. Case 
in point is having a single tree. 
a.        Multicast unpruned tree discovery: you can discover all existing 
unpruned multicast trees by setting the erbridge value of an rbridge 
channel message to the respective roots, M=1 and the dmac to 
"all-egress-rbridges". Just like regular ping, you can send this as an 
echo request message with incrementing ttl values, and the ttl expiry 
messages you get will tell you what the unpruned tree looks like. Changing 
the egress_rbridge will paint you a picture of the different trees. 
[Answer] First reaction to this was am I reading this correctly. After 
reading this one more time it appeared I am reading it correctly. There 
are three main issues: 1. There can be 1000 of RBRidges, and getting a 
response from all of them going to overwhelm. Probably choke the CPU of 
the requester. 2. When you do incremental TTL value, it will leak out at 
the last RBridge. 3. Tree does not include all paths. Assume you have 1 
tree, (Which is a perfectly a valid configuration), how are you going to 
cover links not in the tree. 
b.        Multicast pruned tree discovery: just do a), but replace the 
"all-egress-rbridges" multicast address in the inner dmac with the mac 
address of the multicast group. As you vary the egress_rbridge field, this 
will enumerate all possible pruned trees for that group. Note that 
draft-ietf-trill-rbridge-oam uses draft-ietf-trill-rbridge-channel, which 
requires the inner dmac to be all-egress-rbridges. However, the ethertype 
field can be used by itself to classify an OAM packet, and I am advocating 
that the inner dmac requirement in draft-ietf-trill-rbridge-channel be 
relaxed to allow for alternative inner dmacs where needed. 
[Answer] Fundamental problem:. Prune tree works based on VLAN and MAC 
address or VLAN and IP address. Are you suggesting to try all VLAN all 
address here ? 
c.        Multicast pruned tree discovery for a particular flow: I 
recommend building a packet with an empty payload which looks like the 
actual flow (with the header fields from the flow being tested instead of 
the rbridge channel header). The IP address, mac address, L4 headers etc 
all match with that of the flow you're trying to debug. However, to ensure 
that any leaked packets get dropped, set the IP checksum to be invalid 
(the IP checksum is never used for the ecmp hash). Now send out successive 
packets with the ttl field in incrementing values. Note that you're 
sending a "pseudo-real" packet into the network and relying on received 
ttl expiry messages to figure out the path. 
[Answer] Aha you are now acknowledging IP is used for ECMP hash. Now I am 
wondering how Channel header can facilitate that. The concept of utilizing 
invalid checksum is already covered in draft-tissa-trill-oam-03 Section 
6.3. Secondly using TTL in multicast is a dangerous thing, as tree depth 
can be different and packets can leak out. You may not want to trigger 
invalid IP checksum counter on your hosting servers. As that may generate 
additional support calls. Checksum invalid SHOULD only be used to protect 
against unlikely case packets leaking out. Not the default. 
Question: Are you suggesting to include IP Multicast packets in the native 
form here or encap within channel header ? 
d.        Unicast path discovery: the erbridge ID is that of the switch 
you're trying to ping. Let's send this echo request message with a fake 
inner DMAC. Construct the dmac so that the U/L bit is 1 and the I/G bit is 
0 (basically, its a local scope unicast mac). Set the last 32 bits to a 
random value. This is a fake mac address, but that's ok because the 
network routes only on the erbridge anyway. The M bit in the TRILL header 
is 0. Send this as an echo request message with incrementing ttl values. 
This gives you "a" path from the source to the destination rbridge. 
Increment the dmac used earlier, and repeat. All rbridges hash at least on 
the inner dmac, so this will allow you to enumerate all possible paths 
from source to destination. 
[Answer] You are making big assumption here. Firstly hash is not defined 
in any standard and it is implementation depended. Some devices may use 
SMAC:DMAC VLAN combination, Some may use IP flow information. May be low 
ends use just DMAC. In anycase selecting Random values are not going to 
gurentee a covering all paths. Secondly one may not even know how many 
ECMP options available on an RBridge 3 hops away. This is the reason more 
elaborate methods are required RFC 4379 explain How this can be done in 
MPLS. Based on the feedback received during the last WG session in Taipei, 
we have also enhanced to included this. Please see section 9.9. This 
needed to be done in more systematic manner rather than randomly 
generating DMAC. DMAC means Destination MAC. SMAC means Source MAC 
e.        Unicast path discovery for a particular flow: Similar to what we 
did for multicast, construct a real packet of length 0. The IP address, 
mac address, L4 headers etc all match with that of the flow you're trying 
to debug. However, to ensure that any leaked packets get dropped, set the 
IP checksum to be invalid. Now set the ttl field in incrementing values. 
Note that you're sending a "pseudo-real" packet into the network and 
relying on received ttl expiry messages to figure out the path. 
[Answer] This same concept is in section 6.3 of draft-tissa-trill-oam.   
Note that doing this eliminates the reliance on IP/ICMP, while preserving 
the objectives of draft-tissa-trill-oam. The fake packets generated are 
technically not OAM packets, just suggestions on how to achieve a certain 
objective. 

3) Fault notification: although neither draft talks about this, this is a 
part of the ethernet OAM feature set. I think draft-ietf-trill-rbridge-oam 
needs to mention that the expectation is that BFD will work over the 
rbridge channel to achieve this.

4) draft-tissa-trill-oam  has a big section on TTM. The equivalent in 
Ethernet OAM is performance management by checking counters and 
send/receive delays, etc. We can cover send/receive delay/jitter 
measurements using ping (as described earlier). I frankly don't see the 
benefit to including remote discovery of counters in this OAM proposal. 
The administrator can always just login to the rbridge of interest and get 
a read-out of the counters directly, or use snmp remotely. We shouldnt be 
replicating functionality which is better done elsewhere, and I think this 
entire section on TTM in draft-tissa-trill-oam should be taken out.



[Answer] Your understanding is incorrect. TTM  is utilized not for 
counters but to ensure that specific flows are reaching specifc devices. 
This is included as part of feedback received from the WG in Tapei. This 
is an adaptation of Data Driven CFM . 

5) I don't see any benefits to DRB/AF discovery OAM messages. This is 
probably done most easily by logging into the rbridge in question and 
doing a "show l2 isis", or extending the SNMP MIB. 



[Answer] How do you know which device to login to use show commands/ 

6) draft-tissa-trill-oam has a section on MAC discovery (basically, being 
able to query all rbridges to see what their rbridge<->mac binding is). 
This is again something that should be best queried using SNMP - this is a 
classic MIB walk case. 
[Answer] How do you know which device is having the MAC so you can do the 
walk ?

7) Address binding verification should be out of scope for a TRILL OAM 
draft, and besides, should just be queried at the rbridge with the CLI. 
[Answer] Firstly how do you know which device to login to use CLI. 
Secondly, anything that improve usability and experience of users can be 
always included.

8) End-Station Attachment Point Discovery: should be out of scope of this 
draft, and is too vendor specific (it uses VN-tags).



[Answer] Terminology can be changed to include what is defined in VXLAN, 
NVGRE etc. The case in point is when there are other overlay technologies 
how to identify different address association. Certain VM may use the same 
MAC address to associate with different VXLAN segments. 

best,
Sunny Rajagopalan 
_______________________________________________
rbridge mailing list
[e-mail]
http://mailman.postel.org/mailman/listinfo/rbridge 
 _______________________________________________
rbridge mailing list
[e-mail]
http://mailman.postel.org/mailman/listinfo/rbridge

_______________________________________________
rbridge mailing list
[e-mail]
http://mailman.postel.org/mailman/listinfo/rbridge
_______________________________________________
rbridge mailing list
[e-mail]
http://mailman.postel.org/mailman/listinfo/rbridge

_______________________________________________
rbridge mailing list
[e-mail]
http://mailman.postel.org/mailman/listinfo/rbridge