Re: [mpls] RtgDir review: draft-ietf-mpls-egress-ptotection-framework-03

[Alexander Vainshtein <[e-mail]>]

Yimin,
Lots of thanks for a prompt response. It seems that we are aligned in our understanding of the EVPN problem, and that it deserves a dedicated document.

I do not really care about specific wording, be it “covered by a dedicated document” or “left for future study” – whatever suits your colleagues and you would do from my POV. And, just to make it clear, I do not think that these considerations should block approval and publication of the framework draft. I do not expect such a document to cover in detail all potential applications and their specific issues.

I will be waiting for the next release of the draft to confirm that my comments have been resolved.

Regards,
Sasha

Office: [phone]
Cell:      [phone]
Email:   [e-mail]

From: Yimin Shen <[e-mail]>
Sent: Thursday, December 13, 2018 4:44 PM
To: Alexander Vainshtein <[e-mail]>
Cc: [e-mail]; [e-mail]; [e-mail]; [e-mail]
Subject: Re: RtgDir review: draft-ietf-mpls-egress-ptotection-framework-03

Hi Sasha,

Thanks very much for all the analysis into the EVPN case!

We agree with your point on the BUM traffic. Given the particular forwarding mode of BUM traffic, it will suffer from duplicate packets with egress protection. EVPN definitely deserves a its own document to discuss this kind of caveats and also clarify the working case of known-unicast traffic.

What we can generally say in this framework draft is that - if there needs to be a consideration or discussion of any applicability or operation issues specific to a given type of service, it should be covered by a separate document. IOW, we don’t want to call out a particular service (like EVPN) for “further study”, because a “framework” should already imply and create the space for “future study” of all services and components if needed. We can broadly point out some categories of issues to consider (e.g. BUM traffic, unavailability of protecting forwarding state on protector), but it’s not a goal of this draft to confirm issues for a specific service and put it in the “future study” list. Would you agree with us on this approach?

Thanks again for your detailed comments. They should be noted for the future study 😊

Thanks,

-- Yimin


From: Alexander Vainshtein <[e-mail]<mailto:[e-mail]>>
Date: Thursday, December 13, 2018 at 1:01 AM
To: Yimin Shen <[e-mail]<mailto:[e-mail]>>
Cc: "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>
Subject: Re: RtgDir review: draft-ietf-mpls-egress-ptotection-framework-03

Yimin,
More of the same - yet another potential issue with egress protection of EVPN MC traffic.
When EVPN uses P2MP LSPs for delivery of MC traffic, the egress in many cases uses the top label as the context label for identifying the ingress PE and interprets the next label(s) in this context. Combining this scheme with the egress protection where the top label received by the Protector represents another context looks quite non-trivial to me and not compatible with some of the approaches for setting the bypass tunnel.
BTW, this issue is probably also relevant for MVPN using P2MP LSPs as PMSI.
My 2c.

Thumb typed by Sasha Vainshtein

________________________________
From: Alexander Vainshtein
Sent: Wednesday, December 12, 2018 11:41:11 AM
To: Yimin Shen
Cc: [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
Subject: RE: RtgDir review: draft-ietf-mpls-egress-ptotection-framework-03

Yimin and all,
To clarify my issue #4 in the previous email.


RFC 7432 in Section 12 says that “The principles behind the following procedures are borrowed from the split-horizon forwarding rules in VPLS solutions [RFC4761<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc4761&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=2-nT7xvtgxYac4wpYxwo_jh5rZM2uwTLxgRhaObwYug&m=pPZNt006XVa65-DtrxFryv0NnV76rnnNDLRSaUYnlz0&s=_lNRGBhCOgDwzbOlr60g2Uo1KXFwZ40s6b2Hc_BkeDI&e=>] [RFC4762<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc4762&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=2-nT7xvtgxYac4wpYxwo_jh5rZM2uwTLxgRhaObwYug&m=pPZNt006XVa65-DtrxFryv0NnV76rnnNDLRSaUYnlz0&s=6X6YWm8roGZppe3mOskc3651p9ofde52PWeiOx-3ef0&e=>].”



However, RFC 4761 and RFC 4762 define the split-horizon rules differently:


-     RFC 4761 in Section 4.2.5 states that “Split horizon forwarding rules apply to broadcast and multicast packets, as well as packets to an unknown MAC address”

-     RFC 4762 in Section 4.4 states that “a PE MUST NOT forward traffic from one PW to another in the same VPLS mesh”, it does not differentiate between BUM and “known unicast” traffic.

While RFC 7432 is aligned with RFC 4761 in its definition of the split-horizon rules, the rule introduced in RFC 4762 is much simpler to implement, and it is not explicitly prohibited by RFC 7432.


Regards,
Sasha

Office: [phone]
Cell:      [phone]
Email:   [e-mail]<mailto:[e-mail]>

From: Alexander Vainshtein
Sent: Wednesday, December 12, 2018 11:20 AM
To: 'Yimin Shen' <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
Subject: RE: RtgDir review: draft-ietf-mpls-egress-ptotection-framework-03

Yimin,
Lots of thanks for the message.

I think that use of the protection framework described in the draft for protection of EVPN-based MP2MP services requires clarification of several issues. I am not sure I can provide a full list, but, at least, the following should be considered:

1.       It seems that in order for egress protection to work, Ethernet links connecting the primary Egress PE and the Protector to the customer site must participate in a common multi-homed Ethernet Segment. If this is correct, it should be specified explicitly

2.       If the multi-homing Ethernet Segment above operates in the Single-Active mode, and if the Primary PE is the DF on this segment for the specific service you try to protect, then, to the best of my understanding, the proposed protection scheme will not protect anything, because the Protector will declare the relevant VLANs as blocked to the CE (e.g., using MVRP as described in Section 8.5 of RFC 7432. The Protector will only unblock these VLANs when it is elected as the DF – but this will not happen until it will learn that the primary Egress PE or the link that connects it to the CE have failed. (If the Primary PE was not a DF, there would be nothing to protect).

3.       If the If the multi-homing Ethernet Segment above operates in the All-Active mode, and if the Primary PE is the DF on this segment for the specific service you try to protect, then:

a.       In the case of the primary PE failure “known unicast” traffic will be  protected by the proposed scheme

b.       The Protector will discard any BUM traffic it received for this service until it is elected as the DF.
Whether this behavior would count as protection for the service in question or not depends on the significance of multicast traffic in this service.

4.       RFC 7432 defines in Section 12 that Ethernet frames with unknown unicast Destination MAC addresses received from another PE MUST NOT ever be forwarded by the receiving PE to any other PE, only locally. While RFC 7432 restricts this to just unknown unicast frames, I suspect that most implementations do not forward EVPN-encapsulated frames they receive  to other PEs in any case (I can be wrong here, of course) just to be on the safe side when it comes to Ethernet loop prevention. Such implementations would have problems with using the proposed protection scheme for protection against failures of the attachment circuit where the primary Egress PE acts as the PLR.

I do not know if the list above is complete and whether all the issues I’ve raised are relevant, but I think that applicability of the egress protection  framework to EVPN requires some additional thought (and, eventually, a dedicated document). Simply saying that “EVPN can be supported in a similar manner as Layer 3 VPN” could be misleading IMHO.

Hopefully these notes would be useful.

Regards,
Sasha

Office: [phone]
Cell:      [phone]
Email:   [e-mail]<mailto:[e-mail]>

From: Yimin Shen <[e-mail]<mailto:[e-mail]>>
Sent: Wednesday, December 12, 2018 2:44 AM
To: Alexander Vainshtein <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
Subject: Re: RtgDir review: draft-ietf-mpls-egress-ptotection-framework-03

Hi Sasah,

a.       L2VPN services that use EVPN technology would neither suffer from an empty FIB in the protecting MAC-VRF nor use EVPN application labels for MAC learning. But I cannot say whether they do or do not introduce some other issues with the proposed framework – this would require serious in-depth analysis. Meanwhile such services look to me like another case when an explicit definition of “left FFS” or “to be addressed in a separate document” would be very much in place.

EVPN can be supported in a similar manner as Layer 3 VPN, although this document doesn’t provide a detailed example.

Thanks,

-- Yimin


From: Alexander Vainshtein <[e-mail]<mailto:[e-mail]>>
Date: Thursday, November 29, 2018 at 3:50 AM
To: Yimin Shen <[e-mail]<mailto:[e-mail]>>
Cc: "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>
Subject: RE: RtgDir review: draft-ietf-mpls-egress-ptotection-framework-03

Yimin,
Lots of thanks for a prompt and very encouraging response.

It seems that all my comments would be resolved with the proposed changes. I will be waiting for the next revision of the draft to confirm that.

Please see some remarks inline below.

Regards,
Sasha

Office: [phone]
Cell:      [phone]
Email:   [e-mail]<mailto:[e-mail]>

From: Yimin Shen <[e-mail]<mailto:[e-mail]>>
Sent: Thursday, November 29, 2018 2:53 AM
To: Alexander Vainshtein <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>
Cc: [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
Subject: Re: RtgDir review: draft-ietf-mpls-egress-ptotection-framework-03

Hi Sasha,

Thanks very much again for the detailed review and the constructive comments and suggestions!

Please see [yshen] inline below.

Thanks,
-- Yimin


From: Alexander Vainshtein <[e-mail]<mailto:[e-mail]>>
Date: Tuesday, November 20, 2018 at 4:41 AM
To: "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>
Cc: "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>, "[e-mail]<mailto:[e-mail]>" <[e-mail]<mailto:[e-mail]>>
Subject: RtgDir review: draft-ietf-mpls-egress-ptotection-framework-03
Resent-From: <[e-mail]<mailto:[e-mail]>>
Resent-To: <[e-mail]<mailto:[e-mail]>>, Jeyananth <[e-mail]<mailto:[e-mail]>>, <[e-mail]<mailto:[e-mail]>>, <[e-mail]<mailto:[e-mail]>>, <[e-mail]<mailto:[e-mail]>>, <[e-mail]<mailto:[e-mail]>>, <[e-mail]<mailto:[e-mail]>>, <[e-mail]<mailto:[e-mail]>>, <[e-mail]<mailto:[e-mail]>>, <[e-mail]<mailto:[e-mail]>>, <[e-mail]<mailto:[e-mail]>>, <[e-mail]<mailto:[e-mail]>>, <[e-mail]<mailto:[e-mail]>>, Loa Andersson <[e-mail]<mailto:[e-mail]>>
Resent-Date: Tuesday, November 20, 2018 at 4:41 AM


Hello,

I have been selected as the Routing Directorate reviewer for this draft. The Routing Directorate seeks to review all routing or routing-related drafts as they pass through IETF last call and IESG review, and sometimes on special request. The purpose of the review is to provide assistance to the Routing ADs. For more information about the Routing Directorate, please see ​http://trac.tools.ietf.org/area/rtg/trac/wiki/RtgDir<https://urldefense.proofpoint.com/v2/url?u=http-3A__trac.tools.ietf.org_area_rtg_trac_wiki_RtgDir&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=2-nT7xvtgxYac4wpYxwo_jh5rZM2uwTLxgRhaObwYug&m=xu90MfAW_jm6lYh6hyEgpJehTmsKq0dHrkE5tND9658&s=lmNRlXjT4KG6xOton77vla3JlOuHmrmtA-DTbeS-eI8&e=>

Although these comments are primarily for the use of the Routing ADs, it would be helpful if you could consider them along with any other IETF Last Call comments that you receive, and strive to resolve them through discussion or by updating the draft.
Document: draft-ietf-mpls-egress-protection-framework-03
Reviewer: Alexander (“Sasha”) Vainshtein
Review date: 19-Nov-18
IETF LC End Date: Not known
Intend status: Standards Track

Summary:
I have some minor concerns about this document that I think should be resolved before publication.

Comments:
The draft is well written, requires from the reader has good understanding of multiple technologies including:

  *   context label spaces and context labels RFC 5331
  *   local and remote LFA mechanisms (RFC 5286 and RFC 7490)
  *   Segment Routing Mirror SIDs (RFC 8402) and more.
I doubt it is suitable reading for a beginner, but I also doubt any required background could be skipped.

This is a framework document, with at least one “specialization” of this framework already being published as RFC 8104<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc8104&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=2-nT7xvtgxYac4wpYxwo_jh5rZM2uwTLxgRhaObwYug&m=xu90MfAW_jm6lYh6hyEgpJehTmsKq0dHrkE5tND9658&s=ZQnSkVQtt1y5mvUspbxj3ex47FimKytILN6VNLAAXIk&e=>.  I assume that other “specialization” documents are on the way.

I have hold a very constructive off-list discussion with the authors before posting this review. Some of my comments have been already acknowledged. I would like to thank the authors and, especially, Yimin, for cooperation.

Caveat:
I have done an unsolicited review of the precursor individual draft<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mail-2Darchive_web_mpls_current_msg17033.html&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=2-nT7xvtgxYac4wpYxwo_jh5rZM2uwTLxgRhaObwYug&m=xu90MfAW_jm6lYh6hyEgpJehTmsKq0dHrkE5tND9658&s=zeQV7i8GATWDPu-c_ILYWpjPkln0Wz5-0D6Al_gpPFU&e=>. From my POV, the document has been quite in good shape even then, so I may be considered as positively biased (the current version acknowledges my comments). I have noticed that the majority of my then comments have been successfully resolved in the current version of the draft.

[yshen] Thanks again!

Major Issues: No major issues found

Minor Issues:

1)      The document seems to be ambiguous  with regard to expected coverage of L2VPN services by the framework it defines:
a.       On one hand, the requirement for the framework in the 2nd bullet in Section 4 says that it MUST “accommodate existing and future IP/MPLS services, including layer-2 VPNs, layer-3 VPNs, hierarchical LSP, and others”
b.       On the other hand, the overview of the framework (in Section 1) says that “When a PLR does local repair, the protector is responsible for performing "context label switching" for rerouted MPLS service packets and "context IP forwarding" for rerouted IP service packets, to allow the service packets to continue to reach the service destinations”. i.e., MAC-based switching seems to be excluded from the protector responsibility.
c.       Section 3 of the draft only defines the terms “Context label switching”  and “Context IP forwarding”. This also suggests that Context Layer-2 switching is not covered by the framework

[yshen] Actually, this framework does allow a context label to point to a layer-2 switching table, if the packets coming to a protector have a layer-2 header (rather than a label or IP header) below the context label, which could be the case of MAC address table.  In theory, a context label may point to any type of look-up table. IOW, depending the use case, any meaningful context-based “layer-N” forwarding/switching is allowed by this framework. We will add a section to the end of draft to make it future compatible.[[Sasha]] Sounds good. I will be waiting for the actual text in the next revision of the draft.

2)      I have some doubts regarding applicability of the egress protection framework defined in the draft  to VPLS-based (RFC 4761 and/or RFC 4762)  services because:
a.       With VPLS, CE multi-homing always operates in single-active mode. As a consequence:

                                                               i.      Prior to failure of the primary egress, the VSI that represents the service in the Protector PE would not learn any MAC addresses

                                                             ii.      If the PLR redirects “known unicast” traffic intended for the primary egress PE to the protector, the VSI that locally represents protected service there would flood this traffic as “unknown unicast” and, as such, probably subject it to rate limiting.

                                                           iii.      From the POV of the customer of the VPLS service,  this means that redirecting traffic to the protecting VSI, by and of itself would not fully restore the service, since substantial part of the customer traffic would be dropped until MAC learning in the protecting VSI is completed
b.       It is also worth noting that in VPLS received application labels do not only define egress VSI as the context for the MAC-based switching but also identify remote ingress VSI for MAC learning. Extending such identification to cover also application labels of the redirected traffic looks quite non-trivial to me
c.       I see these concerns as minor issues because, as explained above, it is not clear to me whether the framework is supposed to cover VPLS services. An explicit caveat (possibly defining applicability of the framework to VPLS services as “left FFS”, or to be discussed in a separate document”) would suffice to resolve them IMHO
d.       L2VPN services that use EVPN technology would neither suffer from an empty FIB in the protecting MAC-VRF nor use EVPN application labels for MAC learning. But I cannot say whether they do or do not introduce some other issues with the proposed framework – this would require serious in-depth analysis. Meanwhile such services look to me like another case when an explicit definition of “left FFS” or “to be addressed in a separate document” would be very much in place.

[yshen] We will mention VPLS and EVPN for future study, as well as services where no other router is able to learn or establish its own connectivity with service destination in advance of a primary PE failure, to serve as a protector.[[Sasha]] This should suffice I think.

3)      The example of applying the egress protection framework to BGP/MPLS IP VPN (RFC 4364) in Section 8 is explicitly restricted to scenario when both the primary egress VRF and the protector VRF use per-VRF label allocation scheme.
a.       With this scheme, the usage of the proposed egress protection framework is quite straightforward.
b.       However, RFC 4364 defines several label allocation schemes and states that the label allocation scheme can be selected independently in each PE without any impact on interoperability. It is not clear to me, to which extent IP VPN services that use different label allocation schemes in different egress PEs would be covered by the proposed framework
c.       Since we are only speaking about an example in a framework document, there is no need to describe a complete solution here. Simply noting the need for elaboration with regard to other VPN application label allocation schemes  (including the possible mismatch in the primary and protector egress PEs) and sending the reader to a (future) dedicated document would suffice IMHO.

[yshen] We will clarify that the primary PE (PE2) may allocate VPN labels by using per-VRF, per-route, or per-interface mode. The protector (PE3) should set up the nexthop for each label route in pe2.mpls table as below:
-          If the VPN label is a per-VRF label, the nexthop of the label route should point to the protection VRF.
-          If the VPN label is a per-route label, the nexthop of the label route should be based on the protector’s own connectivity with the IP prefix.
-          If the VPN label is a per-interface label, there are two cases. (A) If the distribution of IP prefixes over the interfaces on the primary PE aligns completely with the distribution of IP prefixes over the interfaces on the protector, the nexthop of the label route should point to the corresponding interface of the IP prefix on the protector. (B) Otherwise (or for simplicity), the nexthop of the label route should point to the protection VRF.
[[Sasha]] Would it not be simpler to say that, regardless of the way the primary egress PE allocates its VPN application labels, the Protector should always treat them as pointing to the relevant VRF and performing context IP forwarding? The logic that matches routes learned by the primary egress PE and by the Protector would be complicated and eventually could fall back to the same scheme – so what is the gain?

4)      Section 5.8 describes 3 possible approaches to advertisement of the Context ID IP address in IGP. But it does not define any of these approaches as “MUST to implement”. This looks to me like an opening for interoperability issues between implementations that support different approaches. On the other hand, I am not sure whether a framework document is the right place to select a MUST to implement option, especially when the expected areas of applicability are quite different:
a.       The “proxy mode” approach imposes minimal (if any) specific requirements on the routing and TE domain (i.e., no dedicated extensions in IGP,  just support of some FRR mechanism by the signaling protocol used for the tunnel setup)
b.       The “alias mode” seems to require support of at least Segment Routing extensions to the relevant IGP in the routing domain as well as support of the Mirror SID in all potential PLRs. Therefore I think that it is a native scheme to be used in the SR-MPLS environments, but problematic in the environments that do not use SR-MPLS for setup of any LSPs
c.       The definition of the “stub link mode” is accompanied by a somewhat vague caveat: “The correctness of the egress-protected tunnels and the bypass tunnels relies on the path computations for the any-cast IP  address performed by the ingress routers and PLR.  Therefore, care MUST be taken for the applicability of this approach to a given network topology”.  Some clarification here would be welcome IMHO, especially since no analog of this mode could be found in RFC 8104.
d.       I have looked up Section 4.3 in RFC 8104 that discusses two approaches (equivalents of the Proxy and Alias modes in this draft) to advertisement of Context ID, and found the following text there (the critical fragment is highlighted):
   The mechanism in this document intends to be flexible on the approach
   used by a network, as long as it satisfies the above requirements for
   the transport tunnel path and bypass tunnel path.  In theory, the
   network can use one approach for context ID X and another approach
   for context ID Y.  For a given context ID, all relevant routers,
   including primary PE, protector, and PLR, must support and agree on
   the chosen approach.  The coordination between the routers can be
   achieved by configuration.
Adding such ( or similar) text to Section 5.8 looks to me as a reasonable compromise between being excessively strict and being excessively flexible in a framework document.

[yshen] We will add similar text to the draft. As a framework, the draft presents and discusses a number of options in a general manner. These options are technically equal and all feasible, although they each may have pros and cons. This is also based on our goal to make egress protection applicable to a variety of networks of different technologies. Hence, the draft does not mandate any option as a MUST. That decision of which option to use in a given network is left to the specific technology deployed and the vendors involved.
[[Sasha]] This should do it IMHO.

Nits:

  1.  RFC 2119 is mentioned in the text, but does not appear as a Normative reference (BTW, I have pointed to this fact in my unsolicited review more than a year ago, but it still remains unresolved)
  2.  RFC 8174 is neither mentioned in the text (as seems to be the norm these days) nor appears as a Normative reference.

[yshen] We will add these RFCs to the normative reference section.
[[Sasha]] And to refer to them in the “Specification of Requirements” section.

Regards,
Sasha

Office: [phone]
Cell:      [phone]
Email:   [e-mail]<mailto:[e-mail]>


___________________________________________________________________________

This e-mail message is intended for the recipient only and contains information which is
CONFIDENTIAL and which may be proprietary to ECI Telecom. If you have received this
transmission in error, please inform us by e-mail, phone or fax, and then delete the original
and all copies thereof.
___________________________________________________________________________

___________________________________________________________________________

This e-mail message is intended for the recipient only and contains information which is
CONFIDENTIAL and which may be proprietary to ECI Telecom. If you have received this
transmission in error, please inform us by e-mail, phone or fax, and then delete the original
and all copies thereof.
___________________________________________________________________________

___________________________________________________________________________

This e-mail message is intended for the recipient only and contains information which is
CONFIDENTIAL and which may be proprietary to ECI Telecom. If you have received this
transmission in error, please inform us by e-mail, phone or fax, and then delete the original
and all copies thereof.
___________________________________________________________________________

___________________________________________________________________________

This e-mail message is intended for the recipient only and contains information which is 
CONFIDENTIAL and which may be proprietary to ECI Telecom. If you have received this 
transmission in error, please inform us by e-mail, phone or fax, and then delete the original 
and all copies thereof.
___________________________________________________________________________