Re: [Idr] Adoption and IPR call for draft-wang-idr-vpn-prefix-orf-03.txt (8/16 to 8/30)

[Robert Raszuk <[e-mail]>]

Aijun,

On the "rogue PE" there are two independent protection mechanisms today in
all L3VPN deployments.

#1 PE-CE prefix limit
#2 VRF limit

Both knobs offer sufficient protection from any VPN overflow from any CE.

Note that routes which pass PE-CE prefix limit, but exceed VRF limit and
coming from CE *should not be advertised to IBGP.*

So literally if you setup your network correctly there is no fear of any
other PE overflows in any of the VPNs.

That of course does not mean you should not monitor the network. So when
monitoring detects resource starvation operator can make a proper action to
address it. Often by reallocating some VPNs to different PEs or purchasing
more PEs.

I do not think detecting this on "DUTs" is right place. Worse even if you
detect it the protocol automation should shut down VRF sourcing those
routes - and sure such VRF can be identified by RD. But applying a local
filter via ORF on RRs is completely wrong idea.

Imagine if one PE indeed goes south and generates and advertises 1M VPN
routes from a single VRF not one but all PEs importing even single RT will
be impacted. So all those PEs will now push ORF to RRs. Just imagine load
of such RRs when they were working fine having all peers served by single
update generation to now having per peer policy and doing individual update
generation.

The right design is to stop the storm and not to keep opening lot's of
little umbrellas over each PE on a per RD+TLV basis.

Thx,
R.


On Sat, Aug 27, 2022 at 2:54 AM Aijun Wang <[e-mail]>
wrote:

> Hi, Igor and Robert:
>
> Let me reply you together via this mail.
> It seems that we can move forward, let’s move together then:
>
> The reason that we want to identify the rogue PE on the DUT, is that the
> VPN routes on DUT comes from several source PEs within the same VPN.
>
> If we push back all the VPN routes within the exceeding VRF, the
> communication between the DUT and all other source PEs will be broken
> within the affected VPN. But if we push back only the overwhelming VPN
> routes from the rogue PE, only the communication between DUT and the rogue
> PE within the affected VPN are broken, other VPNs on the rogue PE, or the
> communication with other normal behavior source PEs(all VPNs) will not be
> influenced.
> This is we called “precise control”, and is the most reasonable responses
> when such thing happens.
>
> Let me explain more details the procedures that Igor’s concern:
> 1) If one source PE send the routes over its quota, but the total number
> doesn’t exceed the VRF limit, No ORF message will be sent out.
> https://datatracker.ietf.org/doc/html/draft-wang-idr-vpn-prefix-orf#section-4.1.1 has
> explained this, as the followings contents:
>
>
> When routes associated with <RD31,PE3> tuple past the quota but the prefix limit of VPN1 VRF is not exceeded, PE1 sends a warning message to the operator, and the VPN Prefix ORF mechanism should not be triggered.
>
>
> 2) The considerations that the quota value needn’t change frequently along
> with the introduction of new PE:
> If the quota are ten times the PE-CE limit, and there is only 2 PEs at
> first, then in theory, the operator need only adjust the quota value when
> the PEs number reach 10. Maybe it will be helpful to give/recommended one
> formulas to calculate the quota value based on the <“PE-CE limit”, numbers
> of PEs, Resources Limit on DUT>? Currently we think the operator can have
> their freedom to plan their networks.
>
> How about the following:
> Quota=MIN[(Margins coefficient)*<PE,CE limit>*<Number of PEs within the
> VPN, includes the possibility expansion in futures>, Resources Limit on DUT]
>
> Anyway, the above recommendations can be tuned more reasonable after the
> proposed mechanism is adopted. I think this is easier work for the operator
> to accomplish this task via their management plane.
>
> Aijun Wang
> China Telecom
>
> On Aug 27, 2022, at 06:56, Igor Malyushkin <[e-mail]> wrote:
>
> 
> Hi Aijun,
>
> Thanks for a more detailed explanation. It is really helpful.
>
> Imagine that we have several sources and quotas for them on a destination
> PE (DUT). Let's say these quotas are slightly bigger than the actual number
> of received routes from every source PE. Also, we have a VRF prefix limit,
> which is more than the sum of these quotas (we need to accommodate some
> routes from local CEs). At this moment everything is ok. Then some source
> PE accidentally sends more routes (for an unknown reason, we identify such
> PEs as rogues, thanks for the term!). The number of routes from this rogue
> PE is more than its quota on DUT, but less than the difference between the
> actual routes inside the VRF of DUT and the VRF prefix limit. In the other
> words, we have a space for these routes and we can install them, but we
> will drop them. This example shows us that careful calculation and setting
> of these quotas are required. As it was previously stated in this thread by
> the authors of the draft the situation when someone needs to use these
> quotas is exceptional rather than usual. I worry that just setting some
> default values for the quotas is dangerous. Thus, we have some operation
> burden (lots of new limits that should be carefully selected). Also, I
> still don't understand how increasing the number of PEs in a VPN does not
> require the renumbering of the quotas` values on every PE of this VPN. It
> is also an extra burden (not everybody has a management tool or something
> for that).
>
> So, in the end, my question is why we can't stop receiving all routes for
> VRF when its prefix limit is reached.  It doesn't require identifying any
> source (but requires identifying a VPN), and it doesn't require some quotas
> setting. VRF prefix limit is widely used and does not require some extra
> configuration.
>
> сб, 27 авг. 2022 г. в 00:09, Aijun Wang <[e-mail]>:
>
>> Hi, Igor:
>>
>> If every source behavior normally, there will be no VRF limit exceeds,
>> the operator will allocate enough value on the receiving device to
>> accommodate the necessary routes.
>> The aim of the VPN Prefixes ORF mechanism is to present the side effect
>> from the rogue PE.
>> In such case, we should identify which source PE or source VRF introduce
>> the overwhelming VPN routes(via the predefined Qutoa), then push back only
>> these overflowed routes.
>> Other routes via the same BGP sessions on the receiving device will not
>> be influenced.
>> Wish the above explanations can give you more helpful to get the essence
>> of this proposal, and also glad to know more of  your suggestions.
>>
>> Aijun Wang
>> China Telecom
>>
>> On Aug 26, 2022, at 20:03, Igor Malyushkin <[e-mail]> wrote:
>>
>> 
>> Hi Aijun,
>>
>> Thanks for the quick response!
>> VRF limit does not prevent a receiving router from parsing updates, yes.
>> Is my understanding correct that it is the main problem that your draft
>> tries to solve?
>> Quoting this text I tried to say that it is enough to say other
>> routers to push back (with VPN Prefixes ORF for sure) when the VRF limit is
>> reached. No need to split the VRF limit further on any quotas.
>>
>> пт, 26 авг. 2022 г. в 13:55, Aijun Wang <[e-mail]>:
>>
>>> Hi, Igor:
>>>
>>> What you quoted has stated the VRF Limit mechanism is not enough to
>>> alleviate the receiving router from parsing the overflowed BGP updates.
>>> We need to push back theses overflowed routes via the VPN Prefixes ORFF
>>> mechanism, which needs the quota to judge which VPN routes breaks its
>>> threshold.
>>> Wish the above explanation can help you get the key motivation of this
>>> draft.
>>>
>>> Aijun Wang
>>> China Telecom
>>>
>>> On Aug 26, 2022, at 19:05, Igor Malyushkin <[e-mail]>
>>> wrote:
>>>
>>> 
>>> Hi Wei,
>>>
>>> Thanks for your comments.
>>>
>>> It seems we are going around the same point again and again. I
>>> understand what quota gives us, but I don't think that it is necessary (or
>>> mandatory) and that it should be a core feature of your solution.
>>> Neither draft-wang-idr-vpn-routes-control-analysis
>>> nor draft-wang-idr-vpn-prefix-orf gives us a good motivation for that
>>> (please point me to it if I'm wrong).
>>> Let me quote draft-wang-idr-vpn-prefix-orf:
>>>
>>> 5) Configure the Maximum Prefix for each VRF on edge nodes
>>>
>>>  When a VRF overflows, it stops the import of routes and log the extra
>>>    VPN routes into its RIB.  However, PEs still need to parse the BGP
>>>    updates.  These processes will cost CPU cycles and further burden the
>>>    overflowing PE.
>>>
>>> It does not matter for what reason a VRF`s prefix limit has been
>>> overflowed (at least I don't see any discussion in the documents about it).
>>> All we need in this case is to stop receiving routes into this VRF
>>> whatever the source of them. Maybe it is possible to describe two options
>>> in your draft? One is based on the VRF prefix limit solely and another is
>>> for its slicing by source PEs (as you see it). The first is mandatory.
>>>
>>>
>>> пт, 26 авг. 2022 г. в 04:27, Wei Wang <[e-mail]>:
>>>
>>>> Hi Igor,
>>>>     We think the quota value should be set based on the resouce limit
>>>> of the receiver device and the number of route sources(PEs) within the same
>>>> VPN. The aim of the quota is to ensure the resouce is shared/used properly
>>>> among the sources.
>>>>     The quota value usually be set much higher than PE-CE limit. It
>>>> will certainly have enough margin to accomodate the possible future
>>>> expansion and need no shrink when one or some the PEs are taken out of the
>>>> VPN.
>>>>
>>>> Best Regards,
>>>> Wei
>>>> ------------------ Original ------------------
>>>> *From:* "Igor Malyushkin" <[e-mail]>;
>>>> *Date:* Thu, Aug 25, 2022 05:30 PM
>>>> *To:* "Aijun Wang"<[e-mail]>;
>>>> *Cc:* "Robert Raszuk"<[e-mail]>;"Wanghaibo
>>>> (Rainsword)"<rainsword.wang=[e-mail]>;"Susan Hares"<
>>>> [e-mail]>;"idr@ietf. org"<[e-mail]>;
>>>> *Subject:* Re: [Idr] Adoption and IPR call for
>>>> draft-wang-idr-vpn-prefix-orf-03.txt (8/16 to 8/30)
>>>>
>>>> Hi Aijun,
>>>>
>>>> Thanks for the comments, please see the inline below.
>>>>
>>>> чт, 25 авг. 2022 г. в 08:30, Aijun Wang <[e-mail]>:
>>>>
>>>>> Hi, Igor:
>>>>>
>>>>>
>>>>>
>>>>> The setting of quota value is the matter of management issue, and
>>>>> should be decoupled from the control plane.
>>>>>
>>>> [IM] From my point of you in your specification, this is the only
>>>> option.
>>>>
>>>>> If you select to let the source PEs advertise such value, it is
>>>>> possible that they change it along with their overflow VPN routes.
>>>>>
>>>> [IM] Well, if the limit between a source and destination PEs should be
>>>> linked as you are suggesting, increasing some value for one VRF cannot be
>>>> made without of review other VRFs among a common VPN. No matter if the
>>>> limit is set manually on a destination PE or dynamically synced from a
>>>> source. A network design should be consistent.
>>>>
>>>>> And from the POV of the receiver PE, before you setting the prefix
>>>>> limit under each VRF, you should have the well estimated quota/value from
>>>>> each PE or each VRF.
>>>>>
>>>> [IM] That is where we do not agree with each other. From my
>>>> perspective, if we are talking about local PE protection then setting a
>>>> limit under a VRF is a matter of the number of all VRFs under the PE and
>>>> some memory limits of the PE. And the VRF limit, in this case, does not be
>>>> the same as the sum of all incoming routes, it can be slightly bigger. If
>>>> we see warnings (say, we've just exceeded 80%) we are considering
>>>> increasing this value (if the memory limit allows it) or updating the
>>>> hardware. Your suggestion requires reviewing the limits for all VRFs
>>>> every time we add or remove a VRF-PE pair (not to mention configuring of
>>>> these limits). And at the end of the day, you actually reach the same local
>>>> memory limit. I want to remember that the solution you are suggesting
>>>> actually does not depend on how we set the prefix limit for the VRF. When
>>>> it is reached we just signal to stop sending.
>>>>
>>>>> As stated before, in most of the situation, the per <PE> or per <RD,
>>>>> PE> quota will be the same value, the operator will not let the design of
>>>>> its network too complex to operate, but the standard should provide the
>>>>> finer control capabilities.
>>>>>
>>>>  [IM] I believe the standard should provide finer control for a reason.
>>>> Let's imagine that without these quotas the solution will not work, but it
>>>> will.
>>>>
>>>>>  This is same as the usage of RD within the network. There is no
>>>>> scenario that the operator to allocate different RDs for one VPN on the
>>>>> same PE. What we often do is that we allocate different RDs for the same
>>>>> VPN on different PEs. Then RD is the right value to distinguish the VPNs on
>>>>> one PE.
>>>>>
>>>>  [IM] All I wanted is to point to the authors that from the parent
>>>> standards POV an RD is not a unique ID for a VRF (under a single PE). Some
>>>> scenarios may emerge in the future.
>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Best Regards
>>>>>
>>>>>
>>>>>
>>>>> Aijun Wang
>>>>>
>>>>> China Telecom
>>>>>
>>>>>
>>>>>
>>>>> *From:* [e-mail] <[e-mail]> *On Behalf Of *Igor
>>>>> Malyushkin
>>>>> *Sent:* Thursday, August 25, 2022 3:14 AM
>>>>> *To:* Robert Raszuk <[e-mail]>
>>>>> *Cc:* Wanghaibo (Rainsword) <rainsword.wang=
>>>>> [e-mail]>; Susan Hares <[e-mail]>;
>>>>> [e-mail]
>>>>> *Subject:* Re: [Idr] Adoption and IPR call for
>>>>> draft-wang-idr-vpn-prefix-orf-03.txt (8/16 to 8/30)
>>>>>
>>>>>
>>>>>
>>>>> For sure it is design-depended. If every VRF has its own RT it will
>>>>> work. But I think if we have a mechanism that allows us to attach several
>>>>> RTs to a route we are in a trouble. Guys are just typing to cover this case.
>>>>>
>>>>>
>>>>>
>>>>> If we talk in general about a whole solution I also think that the way
>>>>> with the new AFI/SAFI is much better. I also don't understand the benefits
>>>>> behind quotas per VRF-PE pair, but if it is really worth the time I expect
>>>>> to see the propagation of these quotas from source PEs instead of a manual
>>>>> configuration. I think it can be easily introduced into a solution with the
>>>>> new AFI/SAFI.
>>>>>
>>>>>
>>>>>
>>>>> ср, 24 авг. 2022 г. в 21:05, Robert Raszuk <[e-mail]>:
>>>>>
>>>>> Igor,
>>>>>
>>>>>
>>>>>
>>>>> RT can uniquely distinguish src vrf. It is simply a matter of
>>>>> proper configuration. No ned protocol extension is required.
>>>>>
>>>>>
>>>>>
>>>>> Thx,
>>>>>
>>>>> R.
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Aug 24, 2022 at 9:03 PM Igor Malyushkin <[e-mail]>
>>>>> wrote:
>>>>>
>>>>> Hi Haibo,
>>>>>
>>>>>
>>>>>
>>>>> 2) It is unpractical to set the quota value for <RT>, or <RT, PE>
>>>>> under VRF, because RT can't uniquely distinguish one VRF on one PE.
>>>>>
>>>>> What if a VRF has several RDs for its routes? RFC4364 and RFC4659
>>>>> don't restrict this behavior and even more, it explicitly describes it. So,
>>>>> RD also can't uniquely distinguish one VRF. Looks like we need a different
>>>>> marker for all routes belonging to the same source VRF. Say, VPN ID
>>>>> community or something like that.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ср, 24 авг. 2022 г. в 18:26, Wanghaibo (Rainsword) <rainsword.wang=
>>>>> [e-mail]>:
>>>>>
>>>>> Hi Sue and WG,
>>>>>
>>>>>
>>>>>
>>>>> I support this adoption.
>>>>>
>>>>> This draft proposes the mechanism to control the overflow of VPN
>>>>> routes within one VRF from influencing other VPNs on the same device
>>>>> automatically.
>>>>>
>>>>>
>>>>>
>>>>> The updated contents have accommodated the suggestions and addressed
>>>>> the comments raised within the WG discussions. Some additional concerns can
>>>>> be addressed after the adoption.
>>>>>
>>>>>
>>>>>
>>>>> I am not aware of any undisclosed IPR to this draft.
>>>>>
>>>>>
>>>>>
>>>>> To make the actual progress of this draft, we should avoid to discuss
>>>>> the solved points back and forth. For example, RTC mechanism is not
>>>>> suitable for the scenarios that described in this adoption draft, because:
>>>>>
>>>>> 1) RTC has no any automatic detection mechanism to determine which RT
>>>>> should be withdrawn now.
>>>>>
>>>>> 2) It is unpractical to set the quota value for <RT>, or <RT, PE>
>>>>> under VRF, because RT can't uniquely distinguish one VRF on one PE.
>>>>>
>>>>> 3) It is dangerous to propagate the RT based filter rule
>>>>> unconditionally in the intra-domain or inter-domain wide, as that done in
>>>>> current RTC mechanism.
>>>>>
>>>>>
>>>>>
>>>>> The conclusion, RTC is not the right direction to accomplish the goal.
>>>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>>
>>>>> Haibo
>>>>>
>>>>>
>>>>>
>>>>> *From:* Idr [mailto:[e-mail] <[e-mail]>] *On
>>>>> Behalf Of *Susan Hares
>>>>> *Sent:* Tuesday, August 16, 2022 11:56 PM
>>>>> *To:* [e-mail]
>>>>> *Subject:* [Idr] Adoption and IPR call for
>>>>> draft-wang-idr-vpn-prefix-orf-03.txt (8/16 to 8/30)
>>>>>
>>>>>
>>>>>
>>>>> This begins a 2 week WG Adoption call for
>>>>> draft-wang-idr-vpn-prefix-orf-03.txt
>>>>>
>>>>> https://datatracker.ietf.org/doc/draft-wang-idr-vpn-prefix-orf/
>>>>>
>>>>>
>>>>>
>>>>> The authors believe that they have addressed the concerns raised in
>>>>>
>>>>> the previous 2 WG adoption calls.
>>>>>
>>>>>
>>>>>
>>>>> The WG should consider if:
>>>>>
>>>>>
>>>>>
>>>>> 1) The revised text answers the previous concerns regarding
>>>>>
>>>>> the scope of this draft?
>>>>>
>>>>>
>>>>>
>>>>> 2) Does the revised text provide a useful function for
>>>>>
>>>>> networks?
>>>>>
>>>>>
>>>>>
>>>>> 3) Are there any additional concerns regarding the new text?
>>>>>
>>>>>
>>>>>
>>>>> Each of the authors should send an IPR statement for
>>>>>
>>>>> this version of the draft.
>>>>>
>>>>>
>>>>>
>>>>> The adoption call was moved to 8/29 to 8/30 to allow questions
>>>>>
>>>>> to be asked at the IDR interim meeting on 8/29/2022 (10am – 12pm
>>>>> EDT).
>>>>>
>>>>>
>>>>>
>>>>> Cheers, Sue Hares
>>>>>
>>>>> _______________________________________________
>>>>> Idr mailing list
>>>>> [e-mail]
>>>>> https://www.ietf.org/mailman/listinfo/idr
>>>>>
>>>>> _______________________________________________
>>>>> Idr mailing list
>>>>> [e-mail]
>>>>> https://www.ietf.org/mailman/listinfo/idr
>>>>>
>>>>> _______________________________________________
> Idr mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/idr
>