Re: [Lsr] Seeking feedback to the revised draft-dunbar-lsr-5g-edge-compute

[Robert Raszuk <[e-mail]>]

> and I agree that using the IGP/Flex Algo as you are proposing is a viable
solution.

Except that IGP usually does not run between application server and load
balancer ...


On Thu, Jan 13, 2022 at 5:37 PM Les Ginsberg (ginsberg) <[e-mail]>
wrote:

> Linda –
>
>
>
> Are you saying that the scenario you are trying to address is to have a
> given “transaction” go to the currently closest/most lightly loaded
> Application Server?
>
> And there is no intent to support for long lived connections?
>
>
>
> If so, then this isn’t really a load balancing issue and I agree that
> using the IGP/Flex Algo as you are proposing is a viable solution.
>
> The concern then becomes the rate of updates to the new Prefix Metric. If
> this changes too rapidly this will heavily consume network resources by
> triggering flooding, SPF, forwarding plane updates at a high rate.
>
> Can you put some language in the draft that indicates the expected rate of
> updates to the metric and some guidelines on limiting the frequency?
>
>
>
> Thanx.
>
>
>
>    Les
>
>
>
>
>
> *From:* Linda Dunbar <[e-mail]>
> *Sent:* Thursday, January 13, 2022 7:58 AM
> *To:* Robert Raszuk <[e-mail]>
> *Cc:* Gyan Mishra <[e-mail]>; Les Ginsberg (ginsberg) <
> [e-mail]>; [e-mail]
> *Subject:* RE: [Lsr] Seeking feedback to the revised
> draft-dunbar-lsr-5g-edge-compute
>
>
>
> Robert,
>
>
>
> For your scenario of TCP flows lasting more than 8~18 hours,  multiple
> server instances SHOULDN’T be assigned with the SAME IP address (ANYCAST
> address).  Each of those instances should have one distinct IP address.
>
>
>
> The draft is for different scenario where application are instantiated at
> multiple locations behind multiple App Layer Load Balancers. They have
> relative short flows that can go to any App Layer Load Balancers. Multiple
> Load Balancers  for those applications are assigned with the same IP
> address. In Kubernetes, multiple load balancers for one type of
> microservices are assigned with one single Virtual IP address, so that the
> network can forward as one single destination.
>
>
>
> Linda
>
> *From:* Robert Raszuk <[e-mail]>
> *Sent:* Thursday, January 13, 2022 9:37 AM
> *To:* Linda Dunbar <[e-mail]>
> *Cc:* Gyan Mishra <[e-mail]>; Les Ginsberg (ginsberg) <
> [e-mail]>; [e-mail]
> *Subject:* Re: [Lsr] Seeking feedback to the revised
> draft-dunbar-lsr-5g-edge-compute
>
>
>
>
>
> > Flows among micro-services are very short.
>
>
>
> While that can be true - there is nowhere in the document a restriction or
> even a warning that this solution is aiming for short lived flows only and
> that users should be well aware about drastic nature of proposed
> mechanism to their established flows.
>
>
>
> In one of the companies I worked for average  TCP flow duration was
> anywhere from 8-18 hours and it was a very drastic event for the user to
> loose it in the middle of the day.  Various means where taken and applied
> to protect such sessions from any form of disconnects.
>
>
>
> I think we are shooting here with the wrong weapon to the target.
>
>
>
> Thx,
>
> R.
>
>
>
> On Thu, Jan 13, 2022 at 4:23 PM Linda Dunbar <[e-mail]>
> wrote:
>
> Robert,
>
>
>
> Your link to Traefik  adds more reasons why “Site index and preference”
> should be distributed by IGP:
>
>    - Site index and preferences for a cluster of micro-service instances
>    are rarely dynamically changed. Most of those values are configured.
>    Therefore, the oscillation is minimal.
>    - Flows among micro-services are very short. Put less requirements to
>    flow affinity.
>    -
>
>
>
> Linda
>
>
>
>
>
> *From:* Robert Raszuk <[e-mail]>
> *Sent:* Thursday, January 13, 2022 5:00 AM
> *To:* Gyan Mishra <[e-mail]>
> *Cc:* Les Ginsberg (ginsberg) <[e-mail]>; Linda Dunbar <
> [e-mail]>; [e-mail]
> *Subject:* Re: [Lsr] Seeking feedback to the revised
> draft-dunbar-lsr-5g-edge-compute
>
>
>
>
>
> And just to provide a sound alternative to the objective of this work.
>
>
>
> Please consider using Traefik - https://traefik.io/
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftraefik.io%2F&data=04%7C01%7Clinda.dunbar%40futurewei.com%7C4361a0b36c614fbf8e7908d9d6aa8b4b%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637776850245075888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=P%2BItvofz3%2Bgg0KEMdfxe9MluMPkQ2v1jL8a1Q50rddo%3D&reserved=0>
>
>
>
> Thx,
>
> R.
>
>
>
>
>
> On Thu, Jan 13, 2022 at 11:56 AM Robert Raszuk <[e-mail]> wrote:
>
> Gyan,
>
>
>
> I see what the draft is trying to do now. /* I did not even consider this
> for the reason described below. */
>
>
>
> But what you wrote requires little correction:
>
>
>
> "So now the server you are on gets overloaded and a site cost gets
> advertised in the IGP at which point the connection receives a TCP reset"
>
>
>
> if you *s/connection/all connections/* then you quickly realize that what
> is proposed here is a disaster.
>
>
>
> Breaking all existing flows going to one LB to suddenly timeout and all go
> to the other LB(s) is never a technique any one would seriously deploy in a
> production network.
>
>
>
> Leave alone that doing that has potential to immediately overload the
> other LB(s)/server(s) too.
>
>
>
> For me the conclusion is that IGP transport level is not the proper layer
> to address the requirement.
>
>
>
> Cheers,
>
> Robert.
>
>
>
>
>
> On Thu, Jan 13, 2022 at 7:05 AM Gyan Mishra <[e-mail]> wrote:
>
>
>
> Hi Les
>
>
>
> Agreed.
>
>
>
> My thoughts are that the context of the draft is based on an Anycast VIP
> address of a server which is proximity based load balancing and not
> necessarily ECMP/UCMP and only if the proximity is the same for multiple
> paths to the Anycast VIP would there be a ECMP/UCMP possibility.
>
>
>
> Let’s say if it’s proximity based and one path is preferred, the flow will
> take that path.  So now the server you are on gets overloaded and a site
> cost gets advertised in the IGP at which point the connection receives a
> TCP reset and flow re-establishes on the alternate path based on the site
> cost and remains there until the server goes down or  gets overloaded or a
> better path comes along.
>
>
>
> For ECMP case, ECMP has flow affinity so the flow will stay on the same
> path long lived until the connection terminates.
>
>
>
> So now in ECMP case the flow hashed to a path and maintains its affinity
> to that path.  Now all of sudden the server gets overloaded and we get a
> better site cost advertised.  So now the session terminates on current path
> and establishes again on the Anycast VIP new path based on the site cost
> advertised.
>
>
>
> The failover I believe results in the user refreshing their browser which
> is better than hanging.
>
>
>
> As the VIP prefix is the only one that experiences reconvergence on new
> path based on site cost if there is any instability with the servers that
> will be reflected to the IGP Anycast prefix as well.
>
>
>
> Is that a good or bad thing.  I think if you had to pick your poison as
> here the issue Linda is trying to solve is a server issue but leveraging
> the IGP to force re-convergence when the server is in a half baked state
> meaning it’s busy and connections are being dropped or very slow QOE for
> end user.  If you did nothing and let it ride the the user would be stuck
> on a bad connection.
>
>
>
> So this solution dynamically fixed the issue.
>
>
>
> As far as oscillation that is not a big deal as you are in a much worse
> off state connected to a dying server on its last leg as far as memory and
> CPU.
>
>
>
> This solution I can see can apply to any client / server connection and
> not just 5G and can be used by enterprises as well as SP for their
> customers to have an drastically improved QOE.
>
>
>
> I saw some feedback on the TLV and I think once that is all worked out I
> am in favor of advancing this draft.
>
>
>
> Kind Regards
>
>
>
> Gyan
>
>
>
>
>
> On Wed, Jan 12, 2022 at 10:16 PM Les Ginsberg (ginsberg) <
> [e-mail]> wrote:
>
> Gyan –
>
>
>
> The difference between ECMP and UCMP is not significant in this discussion.
>
> I don’t want to speak for Robert, but for me his point was that IGPs can
> do “multipath” well – but this does not translate into managing flows.
>
> Please see my other responses on this thread.
>
>
>
> Thanx.
>
>
>
>     Les
>
>
>
>
>
> *From:* Gyan Mishra <[e-mail]>
> *Sent:* Wednesday, January 12, 2022 5:26 PM
> *To:* Robert Raszuk <[e-mail]>
> *Cc:* Les Ginsberg (ginsberg) <[e-mail]>; Linda Dunbar <
> [e-mail]>; [e-mail]
> *Subject:* Re: [Lsr] Seeking feedback to the revised
> draft-dunbar-lsr-5g-edge-compute
>
>
>
>
>
> Robert
>
>
>
> Here are a few examples of UCMP drafts below used in core and data center
> use cases.
>
>
>
> https://datatracker.ietf.org/doc/html/draft-ietf-bess-evpn-unequal-lb-15
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-bess-evpn-unequal-lb-15&data=04%7C01%7Clinda.dunbar%40futurewei.com%7C4361a0b36c614fbf8e7908d9d6aa8b4b%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637776850245075888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=G8h11WpirgxNmk2wzr6QN9DsnYBNGQ42ft7Cz9E8pAk%3D&reserved=0>
>
>
>
> https://datatracker.ietf.org/doc/html/draft-mohanty-bess-weighted-hrw-02
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-mohanty-bess-weighted-hrw-02&data=04%7C01%7Clinda.dunbar%40futurewei.com%7C4361a0b36c614fbf8e7908d9d6aa8b4b%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637776850245075888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=zcwF%2F%2FKh77N6jyXDOXuftPALvaUb%2B2Kvj2G2tedfvL0%3D&reserved=0>
>
>
>
> https://datatracker.ietf.org/doc/html/draft-ietf-idr-link-bandwidth
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-idr-link-bandwidth&data=04%7C01%7Clinda.dunbar%40futurewei.com%7C4361a0b36c614fbf8e7908d9d6aa8b4b%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637776850245075888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=nJcXT7NffiSt7CJh%2F2bnqaa7ClnxMSCf%2BVproPb34s0%3D&reserved=0>
>
>
>
> https://datatracker.ietf.org/doc/html/draft-mohanty-bess-ebgp-dmz
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-mohanty-bess-ebgp-dmz&data=04%7C01%7Clinda.dunbar%40futurewei.com%7C4361a0b36c614fbf8e7908d9d6aa8b4b%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637776850245075888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=AbVKlR%2FrX1vhMzdzHL7J8VgiU2oxaqxSu9oMx9onJRo%3D&reserved=0>
>
>
>
>
>
>
>
> There are many use cases in routing for unequal cost load balancing
> capabilities.
>
>
>
> Kind Regards
>
>
>
> Gyan
>
>
>
> On Wed, Jan 12, 2022 at 2:23 PM Robert Raszuk <[e-mail]> wrote:
>
> Linda,
>
>
>
> > IGP has been used for the Multi-path computation for a long time
>
>
>
> IGP can and does ECMP well. Moreover, injecting metric of anycast server
> destination plays no role in it as all paths would inherit that external to
> the IGP cost.
>
>
>
> Unequal cost load balancing or intelligent traffic spread has always been
> done at the application layer *for example MPLS*
>
>
>
> Thx a lot,
>
> R.
>
>
>
> On Wed, Jan 12, 2022 at 8:18 PM Linda Dunbar <[e-mail]>
> wrote:
>
> Robert,
>
>
>
> Please see inline in green:
>
>
>
> *From:* Robert Raszuk <[e-mail]>
> *Sent:* Wednesday, January 12, 2022 1:00 PM
> *To:* Linda Dunbar <[e-mail]>
> *Cc:* Les Ginsberg (ginsberg) <[e-mail]>; [e-mail]
> *Subject:* Re: [Lsr] Seeking feedback to the revised
> draft-dunbar-lsr-5g-edge-compute
>
>
>
> Hi Linda,
>
>
>
> *[LES:] It is my opinion that what you propose will not achieve your goals
> – in part because IGPs only influence forwarding on a per packet basis –
> not a per flow/connection basis.*
>
> *[Linda] Most vendors do support flow based ECMP, with Shortest Path
> computed by attributes advertised by IGP.*
>
>
>
> I am with Les here. ECMP has nothing to do with his point.
>
>
>
> [Linda] Les said that “IGP only influence forwarding on a per packet
> basis”.  I am saying that vendors supporting “forwarding per flow” with
> equal cost computed by IGP implies  that forwarding on modern routers are
> no longer purely per packet basis.
>
>
>
>
>
> Draft says:
>
>
>
> *When those multiple server instances share one IP address (ANYCAST), the
> transient network and load conditions can be incorporated in selecting an
> optimal path among server instances for UEs.*
>
>
>
> So if we apply any new metric to indicate load of a single anycast address
> how is this going to help anything ?
>
>
>
> [Linda] The “Load” or “Aggregated Site Cost” is to differentiate multiple
> paths with the same routing distance.
>
>
>
>
>
> You would need a mechanism where the network is smart and say per src-dst
> tuple or more spreads the traffic. IGP does not play that game today I am
> afraid.
>
> [Linda] There is one SRC and multiple paths to one DST. IGP has been used
> for the Multi-path computation for a long time.
>
>
>
> Thank you, Linda
>
>
>
> Thx a lot,
> R.
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> Lsr mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/lsr
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Flsr&data=04%7C01%7Clinda.dunbar%40futurewei.com%7C4361a0b36c614fbf8e7908d9d6aa8b4b%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637776850245075888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=4VvMlYIqiEyjlRQHQkRLXocNchpxnDGqBSfKG96GCaY%3D&reserved=0>
>
> --
>
> [image: Image removed by sender.]
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.verizon.com%2F&data=04%7C01%7Clinda.dunbar%40futurewei.com%7C4361a0b36c614fbf8e7908d9d6aa8b4b%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637776850245075888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RBLTHUJUf6MujlMd%2FIpMJH36fjYXm3diFx6nS9I28E0%3D&reserved=0>
>
> *Gyan Mishra*
>
> *Network Solutions Architect *
>
> *Email [e-mail] <[e-mail]>*
>
> *M 301 502-1347*
>
>
>
> --
>
> [image: Image removed by sender.]
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.verizon.com%2F&data=04%7C01%7Clinda.dunbar%40futurewei.com%7C4361a0b36c614fbf8e7908d9d6aa8b4b%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637776850245075888%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RBLTHUJUf6MujlMd%2FIpMJH36fjYXm3diFx6nS9I28E0%3D&reserved=0>
>
> *Gyan Mishra*
>
> *Network Solutions Architect *
>
> *Email [e-mail] <[e-mail]>*
>
> *M 301 502-1347*
>
>
>
>