Re: [Idr] draft-wang-idr-vpn-prefix-orf-00.txt- Adoption call (2/4/2022 to 2/18/2022)

["Jakob Heitz (jheitz)" <[e-mail]>]

The reality here is that someone leaks the GRT into his VPN.
PE3 may prefer to filter that GRT and carry on anyway, because the real routes are important.
PE2 doesn't care and kicks him out.
Still, it's important that the PE1 operator knows.
If we think the existing network monitoring tools are enough, then I'm fine with it.

The other thing is that (contrary to the claims in the draft),
https://datatracker.ietf.org/doc/html/rfc5292 can do this ORF already.
RFC5292 says nothing about configuration of the feature.
Configuration of the feature is not an interoperability issue,
therefore it's not covered in an RFC.
This draft amounts to a feature request of a router vendor.
The IETF should not be used to submit feature requests.

Regards,
Jakob.

From: Robert Raszuk <[e-mail]>
Sent: Thursday, February 10, 2022 12:40 PM
To: Jakob Heitz (jheitz) <[e-mail]>
Cc: Gyan Mishra <[e-mail]>; UTTARO, JAMES <[e-mail]>; [e-mail]; [e-mail]; Susan Hares <[e-mail]>
Subject: Re: [Idr] draft-wang-idr-vpn-prefix-orf-00.txt- Adoption call (2/4/2022 to 2/18/2022)

Hey Jakob,

Happy Birthday !

> I would be happy to support a new message in BGP (possibly attached
> to a REFRESH message) to achieve a good solution.

When I wrote last paragraph I was thinking about the same - but stopped upon two issues:

a) can not find any advantage on signalling this inband vs out of band (ie via mgmt channel including NNI mgmt)

b) what is PE1 supposed to do if as you say the very same routes are working fine on PE3 and just PE2 has issues ? PE1 can not shut the session to CE, PE1 can not stop sending them to RR etc ... so what's the point ?

I was even crazy enough to think that maybe we could stuff Flowspec v2 with such message, but again (a) won again.

Best,
R.






On Thu, Feb 10, 2022 at 9:27 PM Jakob Heitz (jheitz) <jheitz=[e-mail]<mailto:[e-mail]>> wrote:
Good problem.
Bad solution.

Consider an originator PE, PE1, an RR and two receiving PEs, PE2, PE3.

PE1----RR----PE2
        |
        +----PE3

PE1 sends the routes. PE2 hits a limit, but PE3 does not.
The RR I have drawn may be multiple RRs, a confederation or even a multi AS network.
PE2 should do some combination of the following:
- drop the routes from PE1,
- accept no more routes from PE1 (may cause loops)
- send a notification to PE1.
In particular, RR must not block routes from PE1, because PE3 still wants them.

An ORF does not satisfy the above.
I would be happy to support a new message in BGP (possibly attached to a REFRESH message) to achieve a good solution.

Regards,
Jakob.

From: Idr <[e-mail]<mailto:[e-mail]>> On Behalf Of Gyan Mishra
Sent: Thursday, February 10, 2022 10:23 AM
To: UTTARO, JAMES <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>; Susan Hares <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>
Subject: Re: [Idr] draft-wang-idr-vpn-prefix-orf-00.txt- Adoption call (2/4/2022 to 2/18/2022)


Verizon has had this issue with the rogue PE and the workaround is to set the maximum prefix high as well as per VPN maximum prefix but then that allows the flood to occur and defeats the purpose of maximum prefix to manage resources.  So it’s an industry wide known problem but just had not been brought to light until this draft.

Kind Regards

Gyan

On Thu, Feb 10, 2022 at 1:13 PM Gyan Mishra <[e-mail]<mailto:[e-mail]>> wrote:
Jim

The draft focuses on both EVPN and IP VPN as in scope to the draft as Aijun mentioned.

What I was mentioning was the scenario with China Telecom and the other two providers related to their specific use case being EVPN related however this solution applies to both EVPN and IP VPN.  I believe they may had IP VPN issues as well as the issue as described below can be problematic and impact any operator.  This is not an operations or design issue.

The solution provides granularity to filter on the source PE flooding excessive routes which could be EVPN or IP VPN.  The PE does not have to be a weak PE and could be any PE with plenty of memory and CPU as well.

The Crux of issue is the flood of unwanted routes by a rogue PE sourcing the unwanted flooding of RT-2 or IP prefix flooding.

**  If the PE-CE maximum prefix threshold and per VPN maximum prefix threshold gets hit due to the flood of routes now that impacts the legitimate valid RT-2 or IP prefixes.  As a result all customers could be impacted that are part of the VPN or MAC VRF. So that’s the major issue this draft is solving in trying to clamp down and squash the offending rogue PE**. I mentioned this before that if you set the maximum prefix high or to the maximum you might as well not set it as it’s not helpful in the prevention of flood of routes.   You can think of this like a DDOS attack by a rogue PE impacting all customers that are part of the VPN importing the RT.

The issue with weak PE or PE being overwhelmed with the flood of routes as well is an issue and this draft addresses any of those possible scenarios.


I mentioned EVPN  RT-2 as you may have 1000s of MAC/IP routes depending on how flat your subnet is per RT-5 so the numbers go up pretty quickly and easy to flood.  Also with

The issue is worse inter-as options b as all RTs have to be accepted with knob retain-rt-all on inter-as PE ASBR for option-b and RTs cannot be filtered on RR for option-c same issue for both EVPN and IP VPN.

The draft discusses why existing mechanisms PE-CE maximum prefix and per VPN maximum prefix does not help this issue.

Kind Regards

Gyan

On Thu, Feb 10, 2022 at 6:44 AM UTTARO, JAMES <[e-mail]<mailto:[e-mail]>> wrote:
Gyan,

              So, the draft is targeted to a future where network services migrate to SRV6/EVPN, not a lack of functionality in the current set of tools for existing networks except in the case of China Mobile ( I would like to understand what is unique in this network that is driving this ask ). If that is the case then the draft should focus on the deficiencies of the current tool set for these FOUs. Not sure I get the statement in re RT-2 EVPN routes, is a distinction being made between these and let’s say RT-5?

Thanks,
              Jim Uttaro

From: Gyan Mishra <[e-mail]<mailto:[e-mail]>>
Sent: Thursday, February 10, 2022 12:35 AM
To: Aijun Wang <[e-mail]<mailto:[e-mail]>>
Cc: Susan Hares <[e-mail]<mailto:[e-mail]>>; UTTARO, JAMES <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
Subject: Re: [Idr] draft-wang-idr-vpn-prefix-orf-00.txt- Adoption call (2/4/2022 to 2/18/2022)

Hi Jim

I don’t think there are many SRv6 deployments worldwide and I believe China is one of the leaders on proliferation of IPV6 by operators  as well as now SRv6.

Most operators around the world have large IPv4 backbones and it will take time to migrate.

So as Aijun is stating as more operators deploy SRv6 / EVPN and have inter-as EVPN and  with SR-TE it makes it so easy stretch EVPNs so you end up with these cases.

The issue I believe is with  EVPN Type 2 route flood inter-as and not IP VPN prefixes.

Aijun

Is that correct ?

Kind Regards

Gyan

On Wed, Feb 9, 2022 at 9:31 PM Aijun Wang <[e-mail]<mailto:[e-mail]>> wrote:
Hi, Jim:
Actually, draft https://datatracker.ietf.org/doc/html/draft-wang-idr-vpn-routes-control-analysis/<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-wang-idr-vpn-routes-control-analysis/__;!!BhdT!2fA_b9k4tDNDtZyioqm3O8hinKjqjXnzYV1IMVwk7WZuJsfddfyj_cAYW_od73I$> describes more possible scenarios for the potential VPN prefixes overflow.
With the trending/deployment of EVPN/SRv6 technologies, such problems will be emerged  inevitably because it is more easy to deploy the inter-AS VPN services.
Even for the intra-AS scenario, as that discussed in the current draft, it is possible for the misconfiguration, or misbehavior of PE; or the increase of VPN prefixes in one VRF crashes some PEs in the network and influences other existing VPN services on such PEs.

VPN Prefix ORF mechanism gives us the capabilities to isolate the problem per VRF, not per BGP session, or per PE.
For standard activity, we should look forward to the trend, not only focus on the current status/experiences.

And, there are at least three operators that have widespread networks have interested in the solution, I think it will also benefit your service deployment in future.

Best Regards

Aijun Wang
China Telecom

From: UTTARO, JAMES <[e-mail]<mailto:[e-mail]>>
Sent: Thursday, February 10, 2022 12:10 AM
To: [e-mail]<mailto:[e-mail]>; Aijun Wang <[e-mail]<mailto:[e-mail]>>
Cc: Susan Hares <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>
Subject: RE: RE: [Idr] draft-wang-idr-vpn-prefix-orf-00.txt- Adoption call (2/4/2022 to 2/18/2022)

Comments In-Line

Thanks,
              Jim Uttaro

Here is the thread..

This draft is trying to solve problems existing in networks today.
[Jim U>] Have you canvassed operators? I would be interested in the “problems” that have been identified in networks today. Pls provide data and not conjecture.
[LZQ] Is what you said conjected? If not, I believe this is a real problem. And by the way, I am from China Mobile, an operator, I don't need to canvass other operators.

I understood your first statement as indicating more than just the China Mobile Network. If this work is directed towards an industry wide problem than you should canvass to see if other operators require this specification.  You may be experiencing this problem on your network due to the architecture/design of your network. Feel free to reach out to others who do not seem to be experiencing your networks problem to better understand why this draft is unnecessary..



From: [e-mail]<mailto:[e-mail]> <[e-mail]<mailto:[e-mail]>>
Sent: Wednesday, February 9, 2022 10:37 AM
To: UTTARO, JAMES <[e-mail]<mailto:[e-mail]>>; Aijun Wang <[e-mail]<mailto:[e-mail]>>
Cc: Susan Hares <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>
Subject: Re: RE: [Idr] draft-wang-idr-vpn-prefix-orf-00.txt- Adoption call (2/4/2022 to 2/18/2022)

Comments in-line beginning with [LZQ]

________________________________
Best Regards,
Zhenqiang Li
[e-mail]<mailto:[e-mail]>

From: UTTARO, JAMES<mailto:[e-mail]>
Date: 2022-02-09 21:33
To: Aijun Wang<mailto:[e-mail]>
CC: [e-mail]<mailto:[e-mail]>; Hares<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
Subject: RE: [Idr] draft-wang-idr-vpn-prefix-orf-00.txt- Adoption call (2/4/2022 to 2/18/2022)
Comments In-Line

Thanks,
              Jim Uttaro

From: Aijun Wang <[e-mail]<mailto:[e-mail]>>
Sent: Wednesday, February 9, 2022 7:07 AM
To: UTTARO, JAMES <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>; Hares <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>
Subject: Re: [Idr] draft-wang-idr-vpn-prefix-orf-00.txt- Adoption call (2/4/2022 to 2/18/2022)

Hi, Jim:
Currently, we depended only the “maximum prefixes limit” on the PE router, so when the VPN routes surpasses the threshold, the BGP sessions with the peer will be broken.
[Jim U>] Not true. Additional routing state will be throttled. There is a notification setting and a threshold setting. The only time this was encountered was when a customer mistakenly re-distributed the GRT into their VPN. They were actually quite appreciative of our notification as it helped them quickly isolate the problem and fix it.
[LZQ] Is what you said conjected? If not, I believe this is a real problem. And by the way, I am from China Mobile, an operator, I don't need to canvass other operators.


It will influence all the VRFs that shares within the BGP sessions.
[Jim U>] What does this mean? Is this specific to Option B?
Haven’t you encountered such problems in your network?
[Jim U>] No.
We are trying to find the better solution than the current tools.
[Jim U>] Not required.
Aijun Wang
China Telecom

在 2022年2月9日，19:54，UTTARO, JAMES <[e-mail]<mailto:[e-mail]>> 写道：

Comments In-Line..

Thanks,
              Jim Uttaro

From: Idr <[e-mail]<mailto:[e-mail]>> On Behalf Of [e-mail]<mailto:[e-mail]>
Sent: Wednesday, February 9, 2022 5:13 AM
To: Hares <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>
Subject: Re: [Idr] draft-wang-idr-vpn-prefix-orf-00.txt- Adoption call (2/4/2022 to 2/18/2022)

This draft is trying to solve problems existing in networks today.
[Jim U>] Have you canvassed operators? I would be interested in the “problems” that have been identified in networks today. Pls provide data and not conjecture.
For the solution, I think it is difficult for the receiving router to decide which PE should be blocked when its VRF routes reach a threshould. Why not block any PE sending routes to the overflowing VRF?

________________________________
Best Regards,
Zhenqiang Li

[e-mail]<mailto:[e-mail]>

From: Susan Hares<mailto:[e-mail]>
Date: 2022-02-04 20:39
To: [e-mail]<mailto:[e-mail]>
Subject: [Idr] draft-wang-idr-vpn-prefix-orf-00.txt- Adoption call (2/4/2022 to 2/18/2022)
Resending the adoption call – with the appropriate header

From: Idr [mailto:[e-mail]] On Behalf Of Susan Hares
Sent: Friday, February 4, 2022 7:33 AM
To: [e-mail]<mailto:[e-mail]>
Subject: [Idr] draft-wang-idr-vpn-prefix-orf-00.txt : (2/4/2022 to 2/18/2022)

This begins a 2 week WG adoption call  for
draft-wang-idr-vpn-prefix-orf-00.txt
https://datatracker.ietf.org/doc/draft-wang-idr-vpn-prefix-orf<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-wang-idr-vpn-prefix-orf__;!!BhdT!yPUYUuirQ1P6jY-6LfD7vJRLsQR3n8-cWzq-Id3eLmvKj3ISHGjnW-6e6po$>.

In your discussion on adoption, please consider the following questions:

1) Does this draft describe problems existing in networks today?

2) Does the solution describe address these problems?
The solution has both a general solution and BGP encodings.

3) Would the solution be useful in networks today?

Thank you, Susan Hares
_______________________________________________
Idr mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/idr<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!BhdT!z587vpWgIAY6_KETP44Fo8e1ZASFQCIfEodjzi5G17QCO0dTApqL8DQmB0y2p3Q$>
_______________________________________________
Idr mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/idr<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/idr__;!!BhdT!2fA_b9k4tDNDtZyioqm3O8hinKjqjXnzYV1IMVwk7WZuJsfddfyj_cAYXaxyV6I$>
--

[http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email]<https://urldefense.com/v3/__http:/www.verizon.com/__;!!BhdT!2fA_b9k4tDNDtZyioqm3O8hinKjqjXnzYV1IMVwk7WZuJsfddfyj_cAYuAfegQY$>

Gyan Mishra

Network Solutions Architect

Email [e-mail]<mailto:[e-mail]>

M 301 502-1347

--

[http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email]<http://www.verizon.com/>

Gyan Mishra

Network Solutions Architect

Email [e-mail]<mailto:[e-mail]>

M 301 502-1347

--

[http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email]<http://www.verizon.com/>

Gyan Mishra

Network Solutions Architect

Email [e-mail]<mailto:[e-mail]>

M 301 502-1347

_______________________________________________
Idr mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/idr