Re: [Lsr] Why not leverage Network conditions to optimize balancing among multiple App Layer Load Balancers? as proposed by draft-dunbar-lsr-5g-edge-compute-ospf-ext

[Linda Dunbar <[e-mail]>]

Gyan,

To a router, having multiple servers with the same (ANYCAST) address attached to different egress routers (A-ER) is same as having multiple paths to reach the (ANYCAST) address.

You are absolutely correct that there are many tools to influence the path section, such as the routing distance, TE metrics, policies, etc.

draft-dunbar-lsr-5g-edge-compute-ospf-ext proposes to add another component to influence the path selection: the "Site-Cost"  which is influenced by  "site-capacity + load measurement + Preference + xxx". The "site-Cost" can be raw measurements collected by the egress routers based on the instruction from a controller, or informed by the App Controller periodically.

In the past, ANYCAST has been predominantly used for multiple servers in geographically far apart locations so that routing distance alone can always nail down to one specific ANYCAST server.

3GPP TR23.748 (5G Edge Computing) is proposing to use multiple servers (or multiple App Layer Load Balancers) with the same ANYCAST address in their Local IP Data Network to avoid the single point of failure and the bottleneck at the App Layer Load Balancer for mission critical applications.

Thank you very much for your comments. I have made some changes to the text. Please see the revision: https://datatracker.ietf.org/doc/draft-dunbar-lsr-5g-edge-compute-ospf-ext/


Linda

From: Gyan Mishra <[e-mail]>
Sent: Tuesday, March 9, 2021 12:08 AM
To: Liyizhou <[e-mail]>
Cc: Christian Hopps <[e-mail]>; Linda Dunbar <[e-mail]>; [e-mail]
Subject: Re: [Lsr] Why not leverage Network conditions to optimize balancing among multiple App Layer Load Balancers? as proposed by draft-dunbar-lsr-5g-edge-compute-ospf-ext

Linda and authors

Some thoughts regarding load balancing draft.

Anycast in my experience has been used predominantly in my experience within operators networks with BGP overlay,  using BGP best path selection and most cases boils down to lowest IGP metric tie breaker shortest path for the service Anycast proximity route which you can also with unique RD in overlay and can take advantage of iBGP multipath equal cost load balancing over an operator vaccine or 4G/5G RAN xhaul or internet.

The nice thing about Anycast with BGP overlay you as are automatically proximity based routing load balancing inherent to Anycast routing.

Point here is we are using BGP best path selection but it does boils down to IGP lowest metric tie breaker but you can use iBGP multipath to further optimize the routing for cloud computing.

We have so many tools in our operators toolbox to optimize routing SR or Flex-Algo, SDN etc am wondering if some form of SDN or SD-WAN overlay could provide the Dyncast type Dynamic Anycast solution.

I want to the wiki page for Dyncast.  The presentation is not available yet.  Will check tomorrow.

Thanks

Gyan


On Mon, Mar 8, 2021 at 11:36 PM Liyizhou <[e-mail]<mailto:[e-mail]>> wrote:
Hi,

Sorry to chime in.

There are certainly some higher layer application/protocols to employ. At the same time, there are some advantages of network layer approaches as well in my mind.

When talking about edge computing, there are some unique characteristics. The number of edge sites could be large or huge in future in a big city. Edges are geographically scattered which could be a few, or tens of, or a hundred kilometers away from each other, and each site has limited computing resources which could be a small cluster. Application layer based approach normally would rely on one or several "server"/"broker" to be responsible for request handling all over the city. As such "servers" are unlikely available on each and every edge site, it introduces additional path stretch for data packets requiring delivery to other edge sites first. Such path stretch introduces additional (network and processing) delay which could be crucial for short live request flow. On the contrary, the network node at the edge is naturally sitting on the data path without introducing any additional cost to direct the (explicit/implicit) request somewhere else. Also routing system has been proven doing good in such distributed manner.


There is a dyncast (dynamic anycast) work ongoing. It is not exactly same as what Linda proposed here, but some relations can be seen, like trying to use anycast methodology to access an edge computing, especially computational intensive, service. The current discussions are about compellingness of the use cases, the deficiency of existing solutions, and proposed architecture, not gone very far into what specific routing protocols to use yet. A side meeting will be held on Wed 10am CET. You may check https://github.com/dyncast/ietf110<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdyncast%2Fietf110&data=04%7C01%7Clinda.dunbar%40futurewei.com%7C090a578c2711455d431808d8e2c1ad0b%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637508668779221152%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BARHREw51PPjAyjut9I9Z5Ym%2FV%2FDYwPl2xO2luZxIag%3D&reserved=0> for more info.

Cheers,
Yizhou

From: Lsr [mailto:[e-mail]<mailto:[e-mail]>] On Behalf Of Christian Hopps
Sent: Tuesday, March 9, 2021 9:00 AM
To: Linda Dunbar <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>; Christian Hopps <[e-mail]<mailto:[e-mail]>>
Subject: Re: [Lsr] Why not leverage Network conditions to optimize balancing among multiple App Layer Load Balancers? as proposed by draft-dunbar-lsr-5g-edge-compute-ospf-ext



On Mar 8, 2021, at 7:40 PM, Linda Dunbar <[e-mail]<mailto:[e-mail]>> wrote:

Christian,

You said at LSR session today that there might be concern of network optimizing ANYCAST traffic to better balance among multiple App Layer Load Balancers.
First of all, only the Applications that need to leverage the network condition to balance among their multiple Load Balancers will get the benefit of path selection that are based on the combination of routing distance and other dynamic running status. The networks (e.g. 5G EC Local Data Networks)  only optimize the ANYCAST traffic for the registered addresses.
The network is already responsible for selecting the shortest path to one Application Load Balancer. draft-dunbar-lsr-5g-edge-compute-ospf-ext proposes to add additional weight in path selection.

ANYCAST makes it possible to dynamically load balance across server locations based on network conditions. With multiple servers having the same ANYCAST address, it eliminates the single point of failure and bottleneck at the application layer load balancer that has the shortest routing distance. Another benefit of using ANYCAST address is removing the dependency on how UEs get the IP addresses for their Applications. Some UEs (or clients) might use stale cached IP addresses for extended period.

Network service providers can even offer this as a value added service, making network information more useful to deliver services to applications.
Isn't it a win-win approach for both network service providers and the applications owners?

As WG member,

It's not a win when their network fails.

At a high level I think the idea of a smart network is interesting. I don't have good initial feelings though about trying to achieve that by adding application load based metrics into the routing protocol. There's all sort of layer violations going on there for one, but perhaps more importantly, our routing protocols have not been tried and tested over the decades with this use in mind.

One could imagine designing a higher layer distributed load balancing application/protocol that utilized routing information though, something like that would align more closely with the layering we've been designing to all these years. It probably would not rely on anycast exclusively, but instead use anycast to talk to a server that implemented this LB protocol (something anycast is good at) which would provide a unicast address for the requested application, with the ability to adjust (reacquire a new unicast address, whatever) as conditions (either at the routing or application layer) change through notifications or polling. Just brainstorming here, but there are lots of ways one could imagine this working.

Thanks,
Chris.


Linda Dunbar

_______________________________________________
Lsr mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/lsr<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Flsr&data=04%7C01%7Clinda.dunbar%40futurewei.com%7C090a578c2711455d431808d8e2c1ad0b%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637508668779231156%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=eTZ9G6OwmE%2FHAJyIInOzMSSmQTmVcGFmXbrNm5YCMX4%3D&reserved=0>
--

[http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email]<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.verizon.com%2F&data=04%7C01%7Clinda.dunbar%40futurewei.com%7C090a578c2711455d431808d8e2c1ad0b%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637508668779231156%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=eJ5%2ByOifw0UB7MvrjhqcBLWhvhO6n9QR3%2ByuuWKs2tk%3D&reserved=0>

Gyan Mishra

Network Solutions Architect

M 301 502-1347
13101 Columbia Pike
Silver Spring, MD