Re: [Idr] Adoption and IPR call for draft-wang-idr-vpn-prefix-orf-03.txt (8/16 to 8/30)

[Wei Wang <[e-mail]>]

Hi Igor,


   When the number of routes comes from PE3 pass the quota and also the VPN1 VRF's prefix limit, PE1 should send VPN Prefix ORF entry to RR and RR will withdraw the excessive routes. So we can imagine 2 cases:
   1) If the speed of withdraw quick enough, the number of routes will drop below the VRF's prefix limit. Then, the routes comes from PE2 can be installed normally.
   2) If the speed of withdraw is lower than the speed of receiving new routes, or relying solely on local discard, the VPN1 VRF is still full. Then, the new routes will be dropped. This case also shows the necessity of mechanism for VPN Prefix control, to prevent the resource exhausted by one “Rogue” resources.


   The above is our consideration about the example you stated, and we think it will only happen in case 2). If we miss any points, please let us know, thanks!


Best Regards,
Wei
------------------ Original ------------------
From:                                                                                                                        "Igor Malyushkin"                                                                                    <[e-mail]&gt;;
Date:&nbsp;Mon, Aug 29, 2022 06:43 PM
To:&nbsp;"Aijun Wang"<[e-mail]&gt;;
Cc:&nbsp;"Wei Wang"<[e-mail]&gt;;"Susan Hares"<[e-mail]&gt;;"idr@ietf. org"<[e-mail]&gt;;
Subject:&nbsp;Re: [Idr] Adoption and IPR call for draft-wang-idr-vpn-prefix-orf-03.txt (8/16 to 8/30)



Hi Aijun,

Thanks! As I understood we will have some solution based on ORF in the near future, which helps an RR to understand what routes are excessive. Ok. Now I see that previously installed routes won`t be deleted, it`s good.

Taking back to quotas. Let me discuss the example from Section 4.1.1 again.
After the prefix limit    of VPN1 VRF is exceeded, due to there is no other VRFs on PE1 need    the VPN routes with RT1, PE1 will generate a BGP ROUTE-REFRESH    message contains a VPN Prefix ORF entry, and send to RR.

At this moment VPN1 VRF on PE1 is full, and a quota for PE3 in this VRF is also past. We can say that we cannot install any new routes to the VRF (no matter who`s the source of them). In this example, the VPN ORF message prevents RR to send excessive routes to PE1 only from PE3. What if PE2 also starts sending new routes with RT1? Please consider that the number of these routes may not exceed PE2`s quota, but the VRF prefix limit is exceeded by PE3.







пн, 29 авг. 2022 г. в 11:17, Aijun Wang <[e-mail]&gt;:


Hi, Igor:

&nbsp; &nbsp; The current procedures of VPN Prefixes ORF are following the rules defined in RFC5291.

 But as I stated before (https://mailarchive.ietf.org/arch/msg/idr/Gm0NYRerh_Iorn_LYIJherKYSUc/), there are some rooms to enrich the ORF mechanism for the filtered routes-----instead of withdrawn all the filtered routes, it is possible to withdrawn only the excessive routes for VPN Prefixes ORF(the routes that pass the quota value). 

&nbsp;&nbsp; &nbsp; Because it is related to the RFC5291 and is just the small part of VPN Prefixes ORF mechanism, we want to discuss it later, then omit the description in the current version.

&nbsp;

Best Regards

&nbsp;

Aijun Wang

China Telecom

&nbsp;

From: Igor Malyushkin <[e-mail]&gt; 
Sent: Sunday, August 28, 2022 6:10 PM
To: Aijun Wang <[e-mail]&gt;
Cc: Wei Wang <[e-mail]&gt;; Susan Hares <[e-mail]&gt;; idr@ietf. org <[e-mail]&gt;
Subject: Re: [Idr] Adoption and IPR call for draft-wang-idr-vpn-prefix-orf-03.txt (8/16 to 8/30)

&nbsp;

Hi Aijun,

I`m glad to hear that I got lost if we are not going to drop any prefixes, let`s see.


&nbsp;


Please see my inline.


&nbsp;


&nbsp;

вс, 28 авг. 2022 г. в 06:30, Aijun Wang <[e-mail]&gt;:


Hi, Igor:

&nbsp;


I think you got lost again.


One correction to your follow statement: the proposed mechanism doesn’t drop all routes from the rogue PE, it drop just the overwhelming VPN routes from the rogue PE.



[IM] Let me quote section 4.1.1 (you have used it as the example previously):

After the prefix limit&nbsp;&nbsp; of VPN1 VRF is exceeded, due to there is no other VRFs on PE1 need&nbsp;&nbsp; the VPN routes with RT1, PE1 will generate a BGP ROUTE-REFRESH&nbsp;&nbsp; message contains a VPN Prefix ORF entry, and send to RR.&nbsp; RR will&nbsp;&nbsp; withdraw and stop to advertise such offending VPN routes (RD31, RT1,&nbsp;&nbsp; source PE is PE3) to PE1.

RR will withdraw&nbsp;and stop advertising. What actually will it withdraw? In brackets, you specify (RD31, RT1, PE3) to PE1 as offending VPN routes. Here is why I said that you what to delete all&nbsp;routes of PE3 from PE1`s VRF. If I'm wrong (which is actually good) and RR will withdraw only extra routes (routes that PE1 received after the VRF prefix limit has been reached) please, confirm it. 


&nbsp;


Section 8 (which is also based on Section 4.1.1) shows us that PE1 generates a VPN ORF message with an IMMEDIATE when-to-refresh option and a DENY math option. Quoting RFC5291 (section 3):

The semantics of DENY is to ask the peer&nbsp;&nbsp; not to pass updates for the set of routes that match the ORF entry.

Section 6 of this document also states:

If the When-to-refresh indicates IMMEDIATE, then after processing all&nbsp;&nbsp; the ORF entries carried in the message the speaker re-advertises to&nbsp;&nbsp; the peer routes from the Adj-RIB-Out associated with the peer that&nbsp;&nbsp; have the same AFI/SAFI as what is carried in the message, and taking&nbsp;&nbsp; into account all the ORF entries for that AFI/SAFI received from the&nbsp;&nbsp; peer.&nbsp; The speaker MUST re-advertise all the routes that have been&nbsp;&nbsp; affected by the ORF entries carried in the message, but MAY also re-&nbsp;&nbsp; advertise the routes that have not been affected by the ORF entries&nbsp;&nbsp; carried in the message.

So my understanding here is when RR receives the VPN ORF message with RD31 and some TLVs (RT1, PE3`s IP address) it will reevaluate the Adj-RIB-Out for PE1, find such routes, and discard (withdraws) all routes with the specified attributes (RD31, RT1, PE3) towards PE1 because of DENY option. Please confirm that I got lost here too.


&nbsp;


And one analogy to the similar situation: if one group keep dial your phone and let your normal customer cannot connect you, what will you do? Do nothing? It is certainly not the right approach.


Currently, there are various methods to identify the overwhelming phone number and let the service provider reject them directly.


The phone itself can also discard them locally, but as discussed before, doing so can not relief the receiver from its control plane pressure.



[IM] Thanks for the analogy. But at this moment it looks to me that you are going to ban a whole phone company that owns these annoying numbers. 


&nbsp;


Or, let’s change the question: what will you do in such situations? How to keep to &nbsp;deliver the traffic correctly under the overwhelming control plane pressure?



[IM] I won't surprise you to say that L3VPN (just as an example) here for ages, will I? PE-CE session limit and VRF prefix limit save us from most problems. Maybe having a PE under overwhelming control plane, again and again, is a matter of bad gear placement? I mean exceptional problems happen sometimes, yes. We have robust enough mechanisms for the rest.


&nbsp;


We are eager to hear the more optimal solution as the WG has consensus the problems exists and needs to be solved.



[IM] Let me clarify my attitude. I agree that stopping receiving routes when a VRF prefix limit on a PE is reached is good. But only if and only if the VRF prefix limit is reached. I don't think that we need to drop any routes from a VRF that were installed into the VRF before a VRF limit is reached. It`s absolutely inappropriate.


&nbsp;


If none, the VPN prefixes ORF mechanism is the right approach, although there may exist some points need to be fine tuned which can be accomplished after its adoption.


&nbsp;

Aijun Wang

China Telecom







On Aug 27, 2022, at 20:43, Igor Malyushkin <[e-mail]&gt; wrote:




Hi Aijun,

Thank you for the clarification, it helps a lot. Now I see what I've missed. I thought we will stop receiving extra routes when some limit is reached (quota or VRF, no matter). But a careful reading shows me that you are going to remove all routes received from a rogue PE which is not appropriate from my point of view.

Thus, I'm not going to proceed with this thread any further and I consider this solution should be abandoned. I don't support its adoption. Networks should deliver traffic, not drop it.



&nbsp;

сб, 27 авг. 2022 г. в 02:53, Aijun Wang <[e-mail]&gt;:


Hi, Igor and Robert:

&nbsp;


Let me reply you together via this mail. 


It seems that we can move forward, let’s move together then:


&nbsp;


The reason that we want to identify the rogue PE on the DUT, is that the VPN routes on DUT comes from several source PEs within the same VPN.


&nbsp;


If we push back all the VPN routes within the exceeding VRF, the communication between the DUT and all other source PEs will be broken within the affected VPN. But if we push back only the overwhelming VPN routes from the rogue PE, only the communication between DUT and the rogue PE within the affected VPN are broken, other VPNs on the rogue PE, or the communication with other normal behavior source PEs(all VPNs) will not be influenced.


This is we called “precise control”, and is the most reasonable responses when such thing happens.


&nbsp;


Let me explain more details the procedures that Igor’s concern:


1) If one source PE send the routes over its quota, but the total number doesn’t exceed the VRF limit, No ORF message will be sent out. https://datatracker.ietf.org/doc/html/draft-wang-idr-vpn-prefix-orf#section-4.1.1&nbsp;has explained this, as the followings contents:

&nbsp;When routes associated with <RD31,PE3&gt; tuple past the quota but the prefix limit of VPN1 VRF is not exceeded, PE1 sends a warning message to the operator, and the VPN Prefix ORF mechanism should not be triggered.

&nbsp;


2) The considerations that the quota value needn’t change frequently along with the introduction of new PE:


If the quota are ten times the PE-CE limit, and there is only 2 PEs at first, then in theory, the operator need only adjust the quota value when the PEs number reach 10. Maybe it will be helpful to give/recommended one formulas to calculate the quota value based on the <“PE-CE limit”, numbers of PEs, Resources Limit on DUT&gt;? Currently we think the operator can have their freedom to plan their networks.


&nbsp;


How about the following:


Quota=MIN[(Margins coefficient)*<PE,CE limit&gt;*<Number of PEs within the VPN, includes the possibility expansion in futures&gt;, Resources Limit on DUT]


&nbsp;


Anyway, the above recommendations can be tuned more reasonable after the proposed mechanism is adopted. I think this is easier work for the operator to accomplish this task via their management plane.

Aijun Wang

China Telecom







On Aug 27, 2022, at 06:56, Igor Malyushkin <[e-mail]&gt; wrote:




Hi Aijun,

Thanks for a more detailed explanation. It is really helpful.

Imagine that we have several sources and quotas for them on a destination PE (DUT). Let's say these quotas are slightly bigger than the actual number of received routes from every source PE. Also, we have a VRF prefix limit, which is more than the sum of these quotas (we need to accommodate some routes from local CEs). At this moment everything is ok. Then some source PE accidentally sends more routes (for an unknown reason, we identify such PEs as rogues, thanks for the term!). The number of routes from this rogue PE is more than its quota on DUT, but less than the difference between the actual routes inside the VRF of DUT and the VRF prefix limit. In the other words, we have a space for these routes and we can install them, but we will drop them. This example shows us that careful calculation and setting of these quotas are required. As it was previously stated in this thread by the authors of the draft the situation when someone needs to use these quotas is exceptional rather than usual. I worry that just setting some default values for the quotas is dangerous. Thus, we have some operation burden (lots of new limits that should be carefully selected). Also, I still don't understand how increasing the number of PEs in a VPN does not require the renumbering of the quotas` values on every PE of this VPN. It is also an extra burden (not everybody has a management tool or something for that).

So, in the end, my question is why we can't stop receiving all routes for VRF when its prefix limit is reached.&nbsp; It doesn't require identifying any source (but requires identifying a VPN), and it doesn't require some quotas setting. VRF prefix limit is widely used and does not require some extra configuration.


&nbsp;

сб, 27 авг. 2022 г. в 00:09, Aijun Wang <[e-mail]&gt;:


Hi, Igor:

&nbsp;


If every source behavior normally, there will be no VRF limit exceeds, the operator will allocate enough value on the receiving device to accommodate the necessary routes.


The aim of the VPN Prefixes ORF mechanism is to present the side effect from the rogue PE. 


In such case, we should identify which source PE or source VRF introduce the overwhelming VPN routes(via the predefined Qutoa), then push back only these overflowed routes. 


Other routes via the same BGP sessions on the receiving device will not be influenced.


Wish the above explanations can give you more helpful to get the essence of this proposal, and also glad to know more of &nbsp;your suggestions.

Aijun Wang

China Telecom







On Aug 26, 2022, at 20:03, Igor Malyushkin <[e-mail]&gt; wrote:




Hi Aijun,

Thanks for the quick response!
VRF limit does not prevent a receiving router from parsing updates, yes. Is my understanding correct that it is the main problem that your draft tries to solve?
Quoting this text I tried to say that it is enough to say other routers to push back (with VPN Prefixes ORF for sure) when the VRF limit is reached. No need to split the VRF limit further on any quotas.


&nbsp;

пт, 26 авг. 2022 г. в 13:55, Aijun Wang <[e-mail]&gt;:


Hi, Igor:

&nbsp;

What you quoted has stated the VRF Limit mechanism is not enough to alleviate the receiving router from parsing the overflowed BGP updates. 


We need to push back theses overflowed routes via the VPN Prefixes ORFF mechanism, which needs the quota to judge which VPN routes breaks its threshold.


Wish the above explanation can help you get the key motivation of this draft.

Aijun Wang

China Telecom







On Aug 26, 2022, at 19:05, Igor Malyushkin <[e-mail]&gt; wrote:




Hi Wei,

Thanks for your comments.


&nbsp;


It seems we are going around the same point again and again. I understand what quota gives us, but I don't think that it is necessary (or mandatory) and that it should be a core feature of your solution. Neither draft-wang-idr-vpn-routes-control-analysis nor draft-wang-idr-vpn-prefix-orf gives us a good motivation for that (please point me to it if I'm wrong).


Let me quote draft-wang-idr-vpn-prefix-orf:

5) Configure the Maximum Prefix for each VRF on edge nodes
 When a VRF overflows, it stops the import of routes and log the extra&nbsp;&nbsp; VPN routes into its RIB.&nbsp; However, PEs still need to parse the BGP&nbsp;&nbsp; updates.&nbsp; These processes will cost CPU cycles and further burden the&nbsp;&nbsp; overflowing PE.

It does not matter for what reason a VRF`s prefix limit has been overflowed (at least I don't see any discussion in the documents about it). All we need in this case is to stop receiving routes into this VRF whatever the source of them. Maybe it is possible to describe two options in your draft? One is based on the VRF prefix limit solely and another is for its slicing by source PEs (as you see it). The first is mandatory.


&nbsp;


&nbsp;


пт, 26 авг. 2022 г. в 04:27, Wei Wang <[e-mail]&gt;:


Hi Igor,


&nbsp; &nbsp; We think the quota value should be set based on the resouce limit of the receiver device and the number of route sources(PEs) within the same VPN. The aim of the quota is to ensure the resouce is shared/used properly among the sources.


&nbsp; &nbsp; The quota value usually be set much higher than PE-CE limit. It will certainly have enough margin to accomodate the possible future expansion and need no shrink when one or some the PEs are taken out of the VPN.



&nbsp;


Best Regards,


Wei


------------------ Original ------------------


From: "Igor Malyushkin" <[e-mail]&gt;;


Date:&nbsp;Thu, Aug 25, 2022 05:30 PM


To:&nbsp;"Aijun Wang"<[e-mail]&gt;;


Cc:&nbsp;"Robert Raszuk"<[e-mail]&gt;;"Wanghaibo (Rainsword)"<rainsword.wang=[e-mail]&gt;;"Susan Hares"<[e-mail]&gt;;"idr@ietf. org"<[e-mail]&gt;;


Subject:&nbsp;Re: [Idr] Adoption and IPR call for draft-wang-idr-vpn-prefix-orf-03.txt (8/16 to 8/30)



&nbsp;


Hi Aijun,

Thanks for the comments, please see the inline below.


&nbsp;

чт, 25 авг. 2022 г. в 08:30, Aijun Wang <[e-mail]&gt;:


Hi, Igor:

&nbsp;

The setting of quota value is the matter of management issue, and should be decoupled from the control plane. 




[IM] From my point of you in your specification, this is the only option.


If you select to let the source PEs advertise such value, it is possible that they change it along with their overflow VPN routes.




[IM] Well, if the limit between a source and destination PEs should be linked as you are suggesting, increasing some value for one VRF cannot be made without of review other VRFs among a common VPN. No matter if the limit is set manually on a destination PE or dynamically synced from a source. A network design should be consistent.


And from the POV of the receiver PE, before you setting the prefix limit under each VRF, you should have the well estimated quota/value from each PE or each VRF.&nbsp; 




[IM] That is where we do not agree with each other. From my perspective, if we are talking about local PE protection then setting a limit under a VRF is a matter of the number of all VRFs under the PE and some memory limits of the PE. And the VRF limit, in this case, does not be the same as the sum of all incoming routes, it can be slightly bigger. If we see warnings (say, we've just exceeded 80%) we are considering increasing this value (if the memory limit allows it) or updating the hardware. Your suggestion requires reviewing the limits for all VRFs every time we add or remove a VRF-PE pair (not to mention configuring of these limits). And at the end of the day, you actually reach the same local memory limit. I want to remember that the solution you are suggesting actually does not depend on how we set the prefix limit for the VRF. When it is reached we just signal to stop sending. 


As stated before, in most of the situation, the per <PE&gt; or per <RD, PE&gt; quota will be the same value, the operator will not let the design of its network too complex to operate, but the standard should provide the finer control capabilities.




&nbsp;[IM] I believe the standard should provide finer control for a reason. Let's imagine that without these quotas the solution will not work, but it will.


&nbsp;This is same as the usage of RD within the network. There is no scenario that the operator to allocate different RDs for one VPN on the same PE. What we often do is that we allocate different RDs for the same VPN on different PEs. Then RD is the right value to distinguish the VPNs on one PE.




&nbsp;[IM] All I wanted is to point to the authors that from the parent standards POV an RD is not a unique ID for a VRF (under a single PE). Some scenarios may emerge in the future.


&nbsp;

&nbsp;

Best Regards

&nbsp;

Aijun Wang

China Telecom

&nbsp;

From: [e-mail] <[e-mail]&gt; On Behalf Of Igor Malyushkin
Sent: Thursday, August 25, 2022 3:14 AM
To: Robert Raszuk <[e-mail]&gt;
Cc: Wanghaibo (Rainsword) <rainsword.wang=[e-mail]&gt;; Susan Hares <[e-mail]&gt;; [e-mail]
Subject: Re: [Idr] Adoption and IPR call for draft-wang-idr-vpn-prefix-orf-03.txt (8/16 to 8/30)

&nbsp;

For sure it is design-depended. If every VRF has its own RT it will work. But I think if we have a mechanism that allows us to attach several RTs to a route we are in a trouble. Guys are just typing to cover this case.

&nbsp;


If we talk in general about a whole solution I also think that the way with the new AFI/SAFI is much better. I also don't understand the benefits behind quotas per VRF-PE pair, but if it is really worth the time I expect to see the propagation of these quotas from source PEs instead of a manual configuration. I think it can be easily introduced into a solution with the new AFI/SAFI.



&nbsp;

ср, 24 авг. 2022 г. в 21:05, Robert Raszuk <[e-mail]&gt;:


Igor,

&nbsp;


RT can uniquely distinguish src vrf. It is simply a matter of proper configuration. No ned protocol extension is required. 


&nbsp;


Thx,


R.



&nbsp;

On Wed, Aug 24, 2022 at 9:03 PM Igor Malyushkin <[e-mail]&gt; wrote:


Hi Haibo,

&nbsp;


2) It is unpractical to set the quota value for <RT&gt;, or <RT, PE&gt; under VRF, because RT can't uniquely distinguish one VRF on one PE.


What if a VRF has several RDs for its routes? RFC4364 and RFC4659 don't restrict this behavior and even more, it explicitly describes it. So, RD also can't uniquely distinguish one VRF. Looks like we need a different marker for all routes belonging to the same source VRF. Say, VPN ID community or something like that.


&nbsp;



&nbsp;

ср, 24 авг. 2022 г. в 18:26, Wanghaibo (Rainsword) <rainsword.wang=[e-mail]&gt;:


Hi Sue and WG,

&nbsp;

I support this adoption.

This draft proposes the mechanism to control the overflow of VPN routes within one VRF from influencing other VPNs on the same device automatically. 

&nbsp;

The updated contents have accommodated the suggestions and addressed the comments raised within the WG discussions. Some additional concerns can be addressed after the adoption.

&nbsp;

I am not aware of any undisclosed IPR to this draft.

&nbsp;

To make the actual progress of this draft, we should avoid to discuss the solved points back and forth. For example, RTC mechanism is not suitable for the scenarios that described in this adoption draft, because:

1) RTC has no any automatic detection mechanism to determine which RT should be withdrawn now.

2) It is unpractical to set the quota value for <RT&gt;, or <RT, PE&gt; under VRF, because RT can't uniquely distinguish one VRF on one PE.

3) It is dangerous to propagate the RT based filter rule unconditionally in the intra-domain or inter-domain wide, as that done in current RTC mechanism.

&nbsp;

The conclusion, RTC is not the right direction to accomplish the goal.

&nbsp;

Regards,

Haibo

&nbsp;

From: Idr [mailto:[e-mail]] On Behalf Of Susan Hares
Sent: Tuesday, August 16, 2022 11:56 PM
To: [e-mail]
Subject: [Idr] Adoption and IPR call for draft-wang-idr-vpn-prefix-orf-03.txt (8/16 to 8/30)



&nbsp;

This begins a 2 week WG Adoption call for draft-wang-idr-vpn-prefix-orf-03.txt

https://datatracker.ietf.org/doc/draft-wang-idr-vpn-prefix-orf/

&nbsp;

The authors believe that they have addressed the concerns raised in 

the previous 2 WG adoption calls.

&nbsp;

The WG should consider if:

&nbsp;

1) The revised text answers the previous concerns regarding 

the scope of this draft? 

&nbsp;

2) Does the revised text provide a useful function for 

networks? 

&nbsp;

3) Are there any additional concerns regarding the new text? 

&nbsp;

Each of the authors should send an IPR statement for 

this version of the draft. 

&nbsp;

The adoption call was moved to 8/29 to 8/30 to allow questions 

to be asked at the IDR interim meeting on 8/29/2022 (10am – 12pm EDT). 

&nbsp;

Cheers, Sue Hares &nbsp;




_______________________________________________
Idr mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/idr


_______________________________________________
Idr mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/idr