[Idr] draft-dunbar-idr-sdwan-port-safi: One SDWAN Port property advertisement per BGP UPDATE or multiple Ports property packed in one BGP UPDATE ?

[Linda Dunbar <[e-mail]>]

Haibo,

Thank you very much for pointing out the issue of packing multiple WAN Port properties into one BGP UPDATE.
Alternatively, we can take the same approach by draft-ietf-idr-segment-routing-te-policy-07 where only one SR Policy is carried by one BGP UPDATE.
The Advantage is making it easier for advertising WAN property changes, such as ISP service agreement changes or decommissioned for the WAN port.
The disadvantage is Multiple UPDATE if the SDWAN edge node has multiple ports.

Do people have any preference?

Thanks, Linda

From: Wanghaibo (Rainsword) <[e-mail]>
Sent: Thursday, October 24, 2019 1:11 AM
To: Linda Dunbar <[e-mail]>; Robert Raszuk <[e-mail]>; [e-mail]
Subject: RE: [Idr] Solicit feedback to BGP UPDATE encoding options for SDWAN WAN port properties propagation among SDWAN edges (pros & cons to draft-dunbar-idr-sdwan-port-safi and alternatives)

Hi Linda,

         I think in this method, the routes update will be more complex.
         If we use like this, when A1’s port is down, how to do withdraw? Advertise a new route with type A2 & A3 ?
         Also, when A1’s information is update, we must advertise a route with all A1, A2 and A3.


Regards,
 Haibo

From: Idr [mailto:[e-mail]] On Behalf Of Linda Dunbar
Sent: Thursday, October 24, 2019 4:52 AM
To: Linda Dunbar <[e-mail]<mailto:[e-mail]>>; Robert Raszuk <[e-mail]<mailto:[e-mail]>>; [e-mail]<mailto:[e-mail]>
Subject: Re: [Idr] Solicit feedback to BGP UPDATE encoding options for SDWAN WAN port properties propagation among SDWAN edges (pros & cons to draft-dunbar-idr-sdwan-port-safi and alternatives)

Thanks to everyone’s feedback and discussion. We will re-organize the draft to explain multiple options of encodings.  The description of the options can also help people (future people) to better understand how the MP-NLRI Path Attribute and Tunnel Path Attribute are used together.

Before the detailed encoding discussion, let’s first look the needed information to be advertised by SDWAN Edge:

As described in the [SDWAN-BGP-USAGE], the key characteristics of Hybrid SDWAN includes:

  *   CPE have its WAN ports connected to different ISPs/NSPs, some of which are untrusted network, others are trusted.
For a packet destined towards a Destination, if forwarded out of a WAN port connected to public network, encrypted might be needed, if forwarded out of a WAN port connected to private network, the packet can be forwarded natively without encryption.


  *   A SDWAN node (shorten as SDW in the figure) can only communicate with its authorized peers, not to everyone. To minimize configuration on SDWAN edge nodes, it is the RR (SDWAN controller) that controls the distribution of SDWAN BGP UPDATE.

Therefore, the SDWAN BGP UPDATE messages are only between SDWAN edge and its RR.



  *   The network between SDWAN edge and their RR are over untrusted network.


Centralized-------------------------------------------
Controller (RR)                                       |
|                                                     |
|                     +---+                           |
|        Peer Group 1 |RR |   Peer Group 2            |
|       +======+====+=+   +======+====+=====+         |
|      /      /     | +---+      |     \     \        |
|     /      /      |            |      |     \       |
|  +-+-+  +-+--+  +-+-+        +-+-+  +-+-+  +-+-+    |
|  |SDW|  | SDW|--|SDW|        |SDW|  |SDW|  |SDW|    |
---| 1 |  |  2 |  | 3 |        |4  |  | 5 |  | 6 |----|
   +---+  +----+  +---+        +---+  +---+  +---+
Tenant 1                   Tenant 2

Figure 1: SDWAN WAN Port Properties Advertisement via RR


Using Figure 1 as illustration, a SDWAN node needs to propagate the following properties to its RR:

  *   Node information, including Node ID and Site ID,
  *   WAN Port information, under each Port, need to have more detailed description of the port, such as
     *   the types of the Wide Area Network, if it is a private MPLS L3/L2 VPN, Public Internet, LTE, etc.
     *   Service provider identifier for the network facing the WAN port
     *   If facing untrusted network, IPsec information about the IPsec tunnels terminated at the port
     *   If the port is assigned with the private IP address, its NAT properties
  *   Attached client routes, which is same as in VPN or EVPN.
  *   Node Level IPsec information, if the SDWAN node needs to establish Node Level IPsec tunnel, regardless which WAN ports the packets are sent/received.

RR is responsible for propagate the received SDWAN UPDATE to a group of authorized peers via encrypted channels. For example, the RR propagates the UPDATE received from SDW-1 to SDW-2 & SDW-3.


Using MP-NLRI and Tunnel Path attributes, a SDWAN node with 3 WAN Ports (A1 connected to Internet, A2 & A3 connected to MPLS) can send out the following UPDATE to advertise its 3 WAN ports A1/A2/A3 properties:

Border Gateway Protocol – UPDATE Message

Path Attribute - MP_REACH_NLRI  /* no including detailed routes in NLRI field if the routes are not attached */

      Address family identifier (AFI)

     Subsequent address family identifier (SAFI)

Next hop network address  /* for the SDWAN-A loopback address */

** need to add an extra field to indicate Site-ID of this SD-WAN node **



Path Attribute – Tunnel-Encap    (Type Code=23)

/*for A1*/

Tunnel-Type = Untrusted-Internet  /* need IANA assignment */

Remote endpoint subTLV /* for A1 address  or A1 port ID*/

Encoding of A1’s properties in subTLVs  /* see later section for detailed encoding */

Ipsec properties for Ipsec tunnel terminated at A1

NAT properties if A1 has private address



/* for A2 */

Tunnel-Type = MPLS-VPN

Remote endpoint subTLV /* for A2 address */



/* for A3 */

Tunnel-Type = MPLS-VPN

Remote endpoint subTLV /* for A3 address */

The SDWAN-A loopback address might be internal to the peer group and not routable in the WAN.

What do people think? If no objection, I will add the text to the draft.

Thanks, Linda

From: Idr <[e-mail]<mailto:[e-mail]>> On Behalf Of Linda Dunbar
Sent: Tuesday, August 13, 2019 3:01 PM
To: Robert Raszuk <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>
Subject: Re: [Idr] Solicit feedback to BGP UPDATE encoding options for SDWAN WAN port properties propagation among SDWAN edges (pros & cons to draft-dunbar-idr-sdwan-port-safi and alternatives)

Robert

Yes, it is “Overlay BGP”. Thank you very much for suggesting the name. I will incorporate it into the draft.

Linda

From: Robert Raszuk <[e-mail]<mailto:[e-mail]>>
Sent: Tuesday, August 13, 2019 2:06 PM
To: Linda Dunbar <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>
Subject: Re: [Idr] Solicit feedback to BGP UPDATE encoding options for SDWAN WAN port properties propagation among SDWAN edges (pros & cons to draft-dunbar-idr-sdwan-port-safi and alternatives)

Hi Linda,

> note: the BGP for SDWAN Edges is running at different layers than the BGP for underlay networks, i.e. not “FLAT” BGP

So I was under impression that you are to discover the interested SDWAN peers with this extension - well apparently not if what you are defining is an overlay BGP.

But then if you have to discover the peers dynamically in some other way prior to establishing said BGP sessions - why not to add this information as part of auto discovery mechanism ?

Many thx,
R.






On Tue, Aug 13, 2019 at 8:07 PM Linda Dunbar <[e-mail]<mailto:[e-mail]>> wrote:
Robert,

Thanks for the quick feedback. It is interesting that you mentioned LISP, I had “LISP” listed as the 4th option in my first version of the analysis, then decided to remove it because the discussion is in IDR group. Legitimate question on WHY BGP?

In addition to looking into your suggested “out of box thinking” that BGP carrying some short reference the actual information should be kept outside of it in one of the new global databases,  here are some of the Compelling reasons of using BGP to distribute SDWAN edge properties among peers that might be spread across the globe:
(note: the BGP for SDWAN Edges is running at different layers than the BGP for underlay networks, i.e. not “FLAT” BGP. They are among SDWAN edges, not for exposing to underlay provider as you stated EBGP. When the underlay network service providers use SDWAN to temporarily expand bandwidth in some segments, they have more reason to use BGP to minimize amount of learning & configuration of introducing new protocols in their environment)


     *   BGP already widely deployed as sole protocol (see RFC 7938). Even if not for this purpose of propagating SDWAN WAN port properties, the BGP base protocol implementation is supported by virtually all switches/routers (virtual & physical).  Even AWS VPC export the BGP routes.
     *   Wide acceptance – minimal learning (which is very important requirement for operations)
     *   Robust and simple implementation,
     *   Reliable transport
     *   Guaranteed in-order delivery
     *   Incremental updates
     *   No flooding and selective filtering
     *   RR already has the capability to apply policies to communications among peers.
Bottom line:  It is much easier to add one function than adding a brand-new protocol stack.

Alternative: extending LISP, NHRP, DSVPN/DMVPN

     *   In addition to more proposal changes needed, NHRP/DSVPN/DMVPN don’t scale well.
     *   More learning, more barrier to be deployed, just think how many decades of painful journey deploying IPv6.
     *
Prior extension of BGP for non-client routes reachability: Flowspec, BGP LS, Segment routing policies, etc

Linda

From: Robert Raszuk <[e-mail]<mailto:[e-mail]>>
Sent: Tuesday, August 13, 2019 3:16 AM
To: Linda Dunbar <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>
Subject: Re: [Idr] Solicit feedback to BGP UPDATE encoding options for SDWAN WAN port properties propagation among SDWAN edges (pros & cons to draft-dunbar-idr-sdwan-port-safi and alternatives)

Hi Linda,

I must up front say that your efforts in this area are very useful ! Thank you for this work.

But in the same time I need to ask why are we so much hostage of BGP and attempt to propagate all of this opaque information in BGP ? SDWANs by design are to span globe and out of the information you would inject into global BGP only tiny fraction (if even that much) would benefit - for rest of the users it will be unneeded data.

My suggestion here is to think outside of the box and while BGP could carry some short reference the actual information should be kept outside of it in one of the new global databases. See today I operate a global SDWAN and it works just fine when my controller is distributed in AWS cloud.

I would be actually afraid to advertise anything in my EBGP sessions to my providers and it may just attract attacks and hackers towards my sides without any real value to the sites I am interconnecting.

So perhaps we should spend this energy on building secure NNI to the interface of such public database rather to keep loading more and more service info into flat BGP ?

Many thx,
R.

PS. Natural question POPs up - what's wrong with using LISP mapping plane for it ? Sure some are allergic to LISP just like some were allergic to MPLS, but is this really a right reason not to explore full spectrum of options ?

On Tue, Aug 13, 2019 at 12:48 AM Linda Dunbar <[e-mail]<mailto:[e-mail]>> wrote:
IDR participants:

Many thanks to the feedback from IETF105 discussion on draft-dunbar-idr-sdwan-port-safi, especially the hallway discussions with Ali Sajassi, John Drake, Keyur Patel and Sue Hares on other possible encoding options. I put together an analysis of multiple BGP UPDATE encoding options to achieve SDWAN WAN port properties propagation among SDWAN edges. Please see the slides in the IDR WIKI page: https://trac.ietf.org/trac/idr/attachment/wiki/WikiStart/<https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftrac.ietf.org%2Ftrac%2Fidr%2Fattachment%2Fwiki%2FWikiStart%2F&data=02%7C01%7Clinda.dunbar%40futurewei.com%7Ca7b9ea26ff544510462308d75848f8d5%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637074942762363098&sdata=dl0yy0Jw3T%2BjI9keT4JObS8EACnReZ4g4eqs2hjdNLM%3D&reserved=0>

The Goal is for WAN Ports Property Propagation across SDWAN nodes in different domains
[cid:image001.png@01D58A50.536BD410]


The constraints are those SDWAN edges can be spread across different geographical locations, their connection to RR can be over untrusted networks, and they might not know the reachable addresses for the peers they need to communicate (therefore needing RR to propagate).

There are many ways to skin the cat… different encoding for BGP Update Messages

Option 1: Extending Tunnel-Encap with existing IP’s SAFI to achieve WAN port registration.
[cid:image002.png@01D58A50.536BD410]


  *   Pros:

     *   no new SAFI introduced, the update messages can traverse existing routers

  *   Cons:

     *   Same IPv4/IPv6 SAFI NLRI carries the WAN port information that is very different from clients’ routes attached to the C-PEs.
     *   The receivers (RR) has to do extra processing to differentiate the UPDATE messages  from the attached routes UPDATE messages.

Option 2: Tunnel-Encap with SDWAN NLRI for SDWAN WAN Ports Prosperities & Policies described by draft-dunbar-idr-sdwan-port-safi-02
[cid:image003.png@01D58A50.536BD410]


  *   Pros:

     *   Clean design and processing on the receivers (RRs). Simpler processing to differentiate the UPDATE messages  from the attached routes UPDATE messages.

  *   Cons:

     *   New NLRI is introduced, the update messages can’t traverse existing routers

        *   Since the the Tunnel UPDATE message with the new SDWAN NLRI/SAFI is strictly between SDWAN edge nodes and their respective RR(s) via a secure tunnel, the SDWAN UPDATE messages are not going to traverse existing routers. Therefore, it doesn’t cause any issues.

Option 3: Using the new SAFI introduced for BGP labeled Colored Unicast  described by draft-szarecki-idr-bgp-lcu-traffic-steering

[cid:image004.png@01D58A50.536BD410]


  *   Pros:

     *   leverage the newly proposed NLRI for carrying Traffic Color across domains
     *   Similar goal as SDWAN needing to propagating WAN port properties across domain/geolocations

  *   Cons:

     *   Need to attach the attributes which haven’t been specified by the draft yet.


Need to ask merging the content from draft-dunbar-idr-sdwan-port-safi-02..


We are looking for feedback to those analysis and options.

Thank you very much
Linda Dunbar
_______________________________________________
Idr mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/idr<https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fidr&data=02%7C01%7Clinda.dunbar%40futurewei.com%7Ca7b9ea26ff544510462308d75848f8d5%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637074942762373091&sdata=XMcMgcQzzdqk42A0eBbzhLOpCn%2BRpUavKGMeFPfilOE%3D&reserved=0>