RE: Discussion on forwarding performance

[Jiangyuanlong <[e-mail]>]

I was referring always to the egress PE. In my understanding (going
deeper and deeper into implementation) an ASIC-based solution will
typically perform PW and VLAN lookup on ingress and as a result of the
lookup add internal tags to the frame representing VSI instance and
root/leaf attribute.

[JY] For an E-Tree, the outmost VLAN in a PW can only have two possible values: either root or leaf VLAN. Thus we don't need TCAM to find the corresponding VLAN. Even if TCAM is used, only two TCAM items are needed at most, no matter what is the search key length. So I can't see the scalability issue in this regards.

But now we're going too much into individual implementations so I repeat
my comment from the previous e-mail - It would be better if neutral NP
and ASIC-based solution vendors could analyze
both solutions and confirm impact on performance and scalability - and
BTW, we also need input regarding existing silicon that doesn't support
VLAN mapping.

[JY] In principle, I agree with you. However, a fair analysis is possible only when the data plane of both approaches are fully understood, but I am not sure this is the case for the Multi-PW approach, since both the forwarding plane and OAM, two most important constructs of its data plane, is still in the dark, please see http://www.ietf.org/mail-archive/web/l2vpn/current/msg03636.html and http://www.ietf.org/mail-archive/web/l2vpn/current/msg03554.html for details. 
For Dual-VLAN, the forwarding plane is very detailed in draft-jiang-l2vpn-vpls-pe-etree, and the OAM as described in RFC 6136 will apply with no change.

Thanks,
Yuanlong

DC

-----Original Message-----
From: Jiangyuanlong [mailto:[e-mail]] 
Sent: Wednesday, May 16, 2012 1:04 PM
To: Daniel Cohn; Balus, Florin Stelian (Florin); Sam Cao; Alexander
Vainshtein
Cc: [e-mail]
Subject: RE: Discussion on forwarding performance

Daniel,

I don't think we need to look up the VLAN and PW at the same time for
Dual-VLAN:
On the ingress PE, VLAN is firstly appended or translated for a incoming
Ethernet frame from AC, then MAC forwarded in VSI to one or multiple
logical ports each corresponding to a PW (the VLAN mapping may be
processed on a logical port), finally a PW label is appended and
transported over the corresponding PW.
On the egress PE, PW is popped and used to indicate a VSI (also
corresponding to a logical port, where the VLAN mapping may also be
processed), then MAC forwarded in VSI to one or multiple ports each
corresponding to an AC (where a frame with a leaf VLAN will be dropped
upon a leaf AC, actually, it is also a simple VLAN translation in IEEE,
that is, from leaf VLAN to 0xFFF, and all frames with this VLAN will be
dropped).

Therefore, double lookup of VLAN and PW seems unnecessary or even
impossible for this service IMHO. 

Regards,
Yuanlong

-----Original Message-----
From: Daniel Cohn [mailto:[e-mail]] 
Sent: Wednesday, May 16, 2012 4:49 PM
To: Jiangyuanlong; Balus, Florin Stelian (Florin); Sam Cao; Alexander
Vainshtein
Cc: [e-mail]
Subject: RE: Discussion on forwarding performance

VC ID is another name for PW label, right.
<Sasha, stop your ears now> ;-)
>From conversations with chip vendors, I understood that when multiple
lookups are required (as in the dual-VLAN example, you need to lookup PW
label to identify E-Tree instance and then VLAN ID to identify root/leaf
origin), typically a single lookup is performed on ingress with a
composite search key (in this case, including PW label and VLAN ID). 
/<Sasha, stop your ears now> ;-)
But perhaps we are getting too much into implementation here. It would
be better if neutral NP and ASIC-based solution vendors could analyze
both solutions and confirm impact on performance and scalability - and
BTW, we also need input regarding existing silicon that doesn't support
VLAN mapping.

DC

-----Original Message-----
From: Jiangyuanlong [mailto:[e-mail]] 
Sent: Wednesday, May 16, 2012 9:43 AM
To: Daniel Cohn; Balus, Florin Stelian (Florin); Sam Cao; Alexander
Vainshtein
Cc: [e-mail]
Subject: RE: Discussion on forwarding performance

Daniel, 

Not sure I fully understand what you mean by "VLAN and VCID lookups are
performed simultaneously on ingress", do you mean VLAN and PW?
Can you give more details on this requirement?

Regards,
Yuanlong


-----Original Message-----
From: Daniel Cohn [mailto:[e-mail]] 
Sent: Wednesday, May 16, 2012 2:16 PM
To: Balus, Florin Stelian (Florin); Sam Cao; Jiangyuanlong; Alexander
Vainshtein
Cc: [e-mail]
Subject: RE: Discussion on forwarding performance

Hi,

I agree that for a chip-based solution there should be no difference in
performance for the additional lookup. However from conversations with
chip vendors, I understood there might be an impact on scalability
because the double lookup would require a longer search key (assuming
VLAN and VCID lookups are performed simultaneously on ingress), thus
taking up more TCAM resources per E-Tree instance. This would depend on
specific TCAM implementation (e.g. granularity of search key length).

DC

-----Original Message-----
From: Balus, Florin Stelian (Florin)
[mailto:[e-mail]] 
Sent: Tuesday, May 15, 2012 10:53 PM
To: Sam Cao; 'Jiangyuanlong'; Daniel Cohn; 'Alexander Vainshtein'
Cc: [e-mail]
Subject: RE: Discussion on forwarding performance

Sam,

> Obviously the forwarding performance of Dual-VLAN will be much lower
than what of Multi-PW.

Sorry I could not keep up with some of the threads on this subject but
the above statement caught my eyes. We have been doing all kind of
egress operations on VLANs in the egress PE for the last 8 years and I
am not aware of any "much lower" forwarding performance. As far as I
know the chipsets used in the PEs can do this processing no problem. Do
you want to clarify which kind of hardware may have problems?

> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On Behalf
> Of Sam Cao
> Sent: Tuesday, May 15, 2012 6:32 AM
> To: 'Jiangyuanlong'; 'Daniel Cohn'; 'Alexander Vainshtein'
> Cc: [e-mail]
> Subject: RE: Discussion on forwarding performance
>
> Hi Yuanlong,
>
> 3rd mapping will make data plane complex, I think. Anyway, we reached
a
> consensus on Multi-PW implementation.
>
> If we use 3rd mapping, I agree with your opinion, maybe dual-VLAN
seems
> simple, but 2 mapping is enough. Then we can focus on data plane
> performance. While stripe PW label out on egress PE, data-plane knows
> how to forward the frames from PW, to Root AC or Leaf AC or all. But
if
> we use Dual-VLAN, after strip PW label out, data plane should strip
> VLAN-ID out and then does same forwarding work as Multi-PW does. The
> most important is, do one more operation while forwarding E-Tree
> frames. Obviously the forwarding performance of Dual-VLAN will be much
> lower than what of Multi-PW.
>
> Regards,
>
> Yuqun (Sam) Cao
> E-mail: [e-mail]
>
> -----Original Message-----
> From: Jiangyuanlong [mailto:[e-mail]]
> Sent: Tuesday, May 15, 2012 7:03 PM
> To: Sam Cao; 'Daniel Cohn'; 'Alexander Vainshtein'
> Cc: [e-mail]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> See my further comments in line.
>
> -----Original Message-----
> From: Sam Cao [mailto:[e-mail]]
> Sent: Tuesday, May 15, 2012 6:14 PM
> To: Jiangyuanlong; 'Daniel Cohn'; 'Alexander Vainshtein'
> Cc: [e-mail]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Hi Yuanlong,
>
> Thank you very much for your comments. I draw the topology you gave,
> and 7 PWs will be established if we follow 01.
>
> Root AC ---- PE 1 ----- PW 1 -------- PE 2 ----- Leaf _AC
>                |  \                 /  ||
>                |   \  /===PW 6,7===    ||
>                 PW 2  -----PW 5------\  ||
>                |  /----Root PW 3 --- \ ||
> Root _AC ---- PE 3                    PE 4 ------ Root AC
>                |   \                  /  |
>                |    ----Leaf PW 4-----   |
>             Leaf AC                    Leaf AC
> Yes, on PE 3 there are several PWs, but if you want to clarify it,
> there are only 3 types, one can carry frames originated from Root and
> Leaf AC(one endpoint of this PW should be Root-only, otherwise this is
> invalid), one only can carry frames from Root AC, and the last can
> carry frames from Leaf ACs. On second thoughts, we still can classify
> the PWs into 2 sets, one is Leaf set, and another is Root set, and the
> union of the sets Leaf and Root is the PW we called it as compatible
PW
> (Maybe this is not correct, we can find one good term on this).
>
> So this optimization still follows the original design of Dual-PW
> approach, Leaf PW only carries frames from Leaf AC; Root-PW only
> carries frames from Root AC. Is it right?
>
> I don't know whether chip can support this forwarding behavior for
> compatible PW or not, but if we implement it in NP, it is nearly same
> as VPLS. The only difference is, setup 2 mappings, one between
Leaf-ACs
> and Leaf-PWs, one between Root-ACs and Root-PWs. For traditional VPLS,
> there is only one mapping.
>
> [JY] It seems you may need a 3rd mapping: from both root & leaf AC to
a
> compatible PW.
> BTW, for the forwarding and reverse direction, the mapping may be
> asymmetric for you compatible PW.
>
> Based on my understanding, all drafts should have similar mapping.
>
> [JY] Dual-VLAN seems simpler here.
>
> Thanks,
>
> Sam
>
>
> -----Original Message-----
> From: Jiangyuanlong [mailto:[e-mail]]
> Sent: Tuesday, May 15, 2012 3:32 PM
> To: Sam Cao; 'Daniel Cohn'; 'Alexander Vainshtein'
> Cc: [e-mail]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Sam,
>
> Thanks, please see my further comments with [JY].
>
> Ok, we go on the case in your mail, where PE1 has one Root-only AC,
> AC1, and
> PE2 has one Leaf-only AC, AC2. In general we can setup 2
> "unidirectional"
> PWs (bidirectional PW, but just carry unidirectional frames), Root-PW
> which carries frames originated from AC1 and Leaf-PW which carries
> frames originated from AC2. But only one PW also can work, just as
some
> members commented. Ok, we setup one PW between PE1 and PE2, we can
call
> this PW as compatible PW or something else. Frames originated from
Root
> or leaf ACs will be carried via this PW. Its forwarding behavior is
> fully same as what traditional VPLS did, MAC learning on AC or PW in
> one E-Tree. Or say, if one PE has Root-only ACs, it will setup one PW
> with another PE, Root-only, Leaf-only or Root-Leaf-Mixed. If 2 PEs are
> root-Leaf-Mixed or other cases, then 2 PWs will be established. As you
> know, this can be done on control plane.
>
> [JY] Assume there are 2 other nodes PE3 & PE4 in this network
scenario,
> and both PE3 and PE4 have both root and leaf ACs, then there are
> compatible PWs from PE3 which may carry both root and leaf traffic (to
> PE1), which may carry only root traffic (to PE2); and further there
are
> root PW and leaf PW to PE4. Thus, the VSI on PE3 has at least 4
> different types of PW transmitting behaviours with regard to the
E-Tree
> traffic. In the reverse direction, the VSI on PE3 further has at least
> 4 different types of PW receiving behaviours. Not sure how you will
> accommodate for these PWs in both the data plane and control plane.
>
> Regards,
> Yuanlong
>
> Is this clear for you? Is this necessary to optimize PW setup in this
> case?
>
> Maybe we need to add one paragraph on forwarding behavior.
>
> Again thank you very much for your comments,
>
> Sam
>
> -----Original Message-----
> From: Jiangyuanlong [mailto:[e-mail]]
> Sent: Tuesday, May 15, 2012 10:27 AM
> To: Sam Cao; 'Daniel Cohn'; 'Alexander Vainshtein'
> Cc: [e-mail]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Please see my comments in line.
>
> -----Original Message-----
> From: Sam Cao [mailto:[e-mail]]
> Sent: Monday, May 14, 2012 8:31 PM
> To: Jiangyuanlong; 'Daniel Cohn'; 'Alexander Vainshtein'
> Cc: [e-mail]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Hi Yuanlong,
>
> We have discussed this in another thread. In fact, Daniel or I have
> given the answers to the questions you raised here for several time
:).
> If we do in this way, we will reach deadlock and can not move forward
> :). I try to explain it again.
>
> [JY] I will apologize if you had ever provided such answers before,
and
> you may just refer to the links rather than repeat the explanation.
> As you can see, I just try to understand the mechanism of 2PW and its
> implications, but the I-D does not include enough information.
>
> As you know, initial design we will setup 2 PWs all the time (if do in
> this way, I think that you have no question),
>
> [JY] I will be concerned with the operational complexity of this
> approach.
>
> but in most cases 2 PWs are not
> necessary. So in 01 version, we optimize it.
>
> "Root-only VSI <-> any VSI: only root PW required"
> "Leaf-only VSI <-> leaf-only VSI: no PWs required"
>
> In other cases 2 PWs are needed. If so, "Leaf-only -> root-only = one
> leaf PW" is not correct: On Leaf-only side, yes, one Leaf-only PW is
> created between Local Leaf type with remote Root type; on root-only
> side, one root-only PW is created. Maybe it is better to call this as
> compatible or mixed PW. So the left items you list are not correct for
> multi_PW solution.
>
> [JY] This is something new, right? To be honest, I could not
understand
> what is your point: doesn't the mixed PW as you called also consist of
> one Leaf-only PW in one direction and one root-only PW in the other
> direction?
> Furthermore, the case you took is also one case in your guideline
> "Root-only VSI <-> any VSI: only root PW required", don't you think
> that only one root PW is required following this guideline?
> Actually, I don't care much about what is the bidirectional PW or
> unidirectional PW called, but how can a VSI support such a mixed
> scenarios:
> traditional VSI assumes only one PW is required for each peer VSI, and
> its MAC leaning is based on bidirectional PW. But for 2PW, there are
> bidirectional root PW, bidirectional leaf PW, mixed PW, etc..., how to
> support these kinds of PWs in the same VSI is my top concern.
>
> BTW, I guess you also care this case we also have discussed before:
for
> example, if we configure one Leaf AC on root-only PE (then it will be
> root-leaf-mixed PE), we will teardown the compatible PW and re-setup
> PWs again with 01.
> [JY] This was discussed in the emails indeed.
>
> Regards,
>
> Yuqun (Sam) Cao
> E-mail: [e-mail]
>
> -----Original Message-----
> From: Jiangyuanlong [mailto:[e-mail]]
> Sent: Monday, May 14, 2012 7:10 PM
> To: Daniel Cohn; Alexander Vainshtein; Sam Cao
> Cc: [e-mail]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Hi Daniel,
>
> Fine, now it seems more in line with what we had proposed in 2VLAN: a
> spoke PW behaves like root/leaf AC for a PE-r.
>
> But the same problem may also apply to the root/spoke "core PW" for
> Multi-PW:
> According to Multi-PW, only one PW is required between two PEs except
> when both PEs are mixed with root and leaf, so these PWs may be formed
> by combinations of the following unidirectional PW cases (extracted
and
> adapted from Josh's email):
> Root-only -> any VSI = one root PW
> Leaf-only -> root-only = one leaf PW
> Leaf-only -> mixed = one leaf PW
> Mixed -> root-only = one (root+leaf) PW
> Mixed -> leaf-only = one (root+leaf) PW
>
> I have a concern that the forwarding plane of PE to implement this
will
> be very different from the traditional VPLS.
>
> Regards,
> Yuanlong
>
> -----Original Message-----
> From: Daniel Cohn [mailto:[e-mail]]
> Sent: Monday, May 14, 2012 5:00 PM
> To: Jiangyuanlong; Alexander Vainshtein; Sam Cao
> Cc: [e-mail]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Hi Yuanlong,
>
> Like I wrote below, root/leaf spoke PW behave like root/leaf AC, not
> like root/spoke "core PW". So no changes to forwarding plane once this
> is understood.
>
> DC
>
> -----Original Message-----
> From: Jiangyuanlong [mailto:[e-mail]]
> Sent: Monday, May 14, 2012 11:36 AM
> To: Daniel Cohn; Alexander Vainshtein; Sam Cao
> Cc: [e-mail]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Daniel, please see my comments in line.
>
> -----Original Message-----
> From: Daniel Cohn [mailto:[e-mail]]
> Sent: Monday, May 14, 2012 3:37 PM
> To: Jiangyuanlong; Alexander Vainshtein; Sam Cao
> Cc: [e-mail]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Hi Yuanlong,
>
> As I see it, for the PE-r spoke (figure 4 in RFC 4762) we establish a
> single PW per AC - root PW for root AC and leaf PW for leaf AC. With
> local configuration at the PE-rs as part of the spoke PW provisioning.
> And the PE-rs will treat root/leaf spoke PWs exactly as it treat
> root/leaf ACs. So when leaf-originated BUM traffic arrives from the
> core (over a leaf PW), it will be forwarded over all root spoke PWs
but
> not on the leaf spoke PWs.
>
> [JY] So in the reverse direction, you need to transport both root and
> leaf traffic from PE-rs over a root PW to PE-r, and transport root
> traffic over a leaf PW. It seems against the definition of root PW and
> leaf PW in the multi-PW draft. Don't this make the forwarding plane of
> PE-rs more complex?
>
> The forwarding plane is exactly the same as described in the multi-PW
> draft, where spoke PWs are treated exactly like ACs.
>
> Regards,
>
> Daniel
>
>
> -----Original Message-----
> From: Jiangyuanlong [mailto:[e-mail]]
> Sent: Friday, May 11, 2012 5:03 AM
> To: Daniel Cohn; Alexander Vainshtein; Sam Cao
> Cc: [e-mail]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Hi Daniel and all,
>
> When you set up two PWs from the PE-rs to a PE-r for each of its AC,
do
> you mean that one PW is bidirectional root PW and the other is
> bidirectional leaf PW?
> My concern is: where should the leaf traffic be filtered, on the PE-rs
> or on the PE-r?
> If filtered on the PE-r, then lots of bandwidth will be wasted (for
> example, if the PE-r is attached with 9 leafs, then the BUM traffic
> from one of its leafs will be multiplied by 8 times, and be forwarded
> by the PE-rs to the same PE-r).
> If filtered on the PE-rs, not sure how you will design its forwarding
> plane, can you give a hint?
>
> Thanks,
> Yuanlong
>
> ------------------------------
> Date: Wed, 9 May 2012 09:16:36 +0300
> From: "Daniel Cohn" <[e-mail]>
> To: "Alexander Vainshtein" <[e-mail]>,
"Sam
>       Cao" <[e-mail]>, <[e-mail]>
> Cc: [e-mail]
> Subject: RE: Discussion on E-Tree and H-VPLS
> Message-ID: <44F4E579A764584EA9BDFD07D0CA08130780CBA6@tlvmail1>
> Content-Type: text/plain;     charset="ISO-2022-JP"
>
> Hi Sasha,
>
> It's actually very simple. A frame that originated in a root AC is
> always transmitted only in root PW, no matter what the frame type
> (known/unknown unicast or broadcast). This is how the frame source
> information is propagated across the VPLS. So in the H-VPLS example,
> the PE-r will never forward a frame received over a root PW on any
leaf
> PW, only on root PWs (toward the core or toward other spokes).
>
> Hope this clarifies it, regards,
>
> Daniel
>
> -----Original Message-----
> From: [e-mail] [mailto:[e-mail]] On Behalf
> Of Alexander Vainshtein
> Sent: Wednesday, May 09, 2012 7:59 AM
> To: Sam Cao; [e-mail]
> Cc: [e-mail]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Sam, Lizhong and all,
> You've written that in the two-PW solution combined with H-VPLS the
PE-
> r must set up two PWs with each MTU-s.
>
> If this is the case, what kind of forwarding logic should be used to
> prevent sending a BUM frame received from a root-PW to a given MTU-S:
> - back to the same MTU-S on the corresponding leaf-PW?
> - twice (to both root-PW and Leaf-PW) to another MTU-s? And, BTW, how
> is the PW selected in this case?
>
> I am aware of a technique of multiple split horizon groups in PE-r
> which could be possibly used for this purpose. However, since these
> groups have to be represented explicitly in the data plane, their
> potential number is limited by the forwarding HW. Other methods could
> be probably used for the same purpose, but they would probably subject
> to similar HW-based limitations.
>
> I suspect that with the dual-PW approach, the HW would pose an
implicit
> limit, say, on a number of MTU-s that can be connected to the same
PE-r
> and that this limit could be quite low in most cases.
>
> Based on this I suspect that the H-VPLS issue as a critical drawback
of
> the dual-PW approach to E-Tree.
>
> My 2c,
>      Sasha
>
>
> ________________________________________
> From: [e-mail] [[e-mail]] on behalf of Sam
> Cao [[e-mail]]
> Sent: Wednesday, May 09, 2012 3:40 AM
> To: [e-mail]
> Cc: [e-mail]
> Subject: RE: Discussion on E-Tree and H-VPLS
>
> Hi Lizhong?
>
> Thank you very much for your comments. I updated the result on this
> question.
>
> [Lizhong] agree with the above analysis. And we did not say it is a
> technical problem, but it is an operational problem again.
> [Sam] I agree. Dual-VLAN does not make sense. I also discussed this
> with Giles and Yuanlong in another mail, and we reached agreement on
> this:
> MTU should know the access mode, VPWS or VPLS, VPWS mode should
> configure VLAN ID on MTU but VPLS mode can not. I thought this is NOT
> reasonable :), but we can figure this out in draft. It seems ok.
>
> [Lizhong] not fully understand. Do you mean,  when VPWS accessing for
> H-VPLS, the PE-r is also necessary to configure two PWs (root and leaf
> PW) for each AC access?
> [Sam] Yes.
>
> Thanks,
>
> Sam
>
>
>
>