Re: [alto] [Dyncast] Fw: Re: CAN BoF issues #1 #5 #6

[Dirk Trossen <[e-mail]>]

Hi Peng & Linda,

To fold the separate discussions and points made (by Luis and others) about on/off-path solutions, may I suggest the following wording:
"CAN is a network layer solution, performing on-path service instance selection decisions with the possibility to adapt to different ingress routers caused by UEs roaming among different cell sites (gNBs & UPFs).  ALTO solves the problem of service instance selection as an off-path solution, which can be seen as an alternative way of addressing the problem space of CAN at the Application Layer. So in that respect, even targeting a common problem, both provide different approaches, then imposing different needs but also taking different assumptions on how applications and networks interact.

Key here is the signaling latency and signaling load that the service instance selection will incur, both in on- as well as off-path solution. This is turn may impact the frequency with which applications will query ALTO server(s), especially when UEs are roaming among different cell sites (gNBs) triggering different network paths. As a result, off-path systems, e.g., ALTO, replies for applications/services before traffic delivery might not be optimal or valid after the handover. So, more details are needed of ALTO including some extension to support multi-deployment, quick interaction, integrate more performance metric information."

Best,

Dirk

From: Dyncast [mailto:[e-mail]] On Behalf Of Linda Dunbar
Sent: 24 May 2022 19:24
To: [e-mail]; luismiguel.contrerasmurillo <[e-mail]>; alto <[e-mail]>
Cc: dyncast <[e-mail]>
Subject: Re: [Dyncast] [alto] Fw: Re: CAN BoF issues #1 #5 #6

Peng,

Your new version looks very good.

Linda

From: [e-mail]<mailto:[e-mail]> <[e-mail]<mailto:[e-mail]>>
Sent: Monday, May 23, 2022 11:02 PM
To: Linda Dunbar <[e-mail]<mailto:[e-mail]>>; luismiguel.contrerasmurillo <[e-mail]<mailto:[e-mail]>>; alto <[e-mail]<mailto:[e-mail]>>
Cc: dyncast <[e-mail]<mailto:[e-mail]>>
Subject: Re: Re: [Dyncast] [alto] Fw: Re: CAN BoF issues #1 #5 #6

Hi Linda,

Thanks. So the current answer can be described as follows to see if there are any other comments:

CAN is a network layer solution, trying to solve the problem from on-path forwarding-based decisions and can adapt to different ingress routers caused by UEs roaming among different cell sites (gNBs & UPFs).  ALTO tries to solve the problem by exposing information that applications/services can consume before traffic delivery, which can be seen as an alternative way of addressing the problem space of CAN from the Application Layer. So in that respect, even targeting a common problem, both provide different approaches, then imposing different needs but also taking different assumptions on how applications and networks interact.

Compared to the on-path routing solution, since not all applications will query ALTO server(s), especially when UEs roaming among different cell sites (gNBs) triggering different network paths, the ALTO reply for applications/services before traffic delivery might not be optimal or valid after the handover. So, more details are needed of ALTO including some extension to support multi-deployment, quick interaction, integrate more performance metric information.

Regards,
Peng
________________________________
[e-mail]<mailto:[e-mail]>

From: Linda Dunbar<mailto:[e-mail]>
Date: 2022-05-24 05:09
To: [e-mail]<mailto:[e-mail]>; luismiguel.contrerasmurillo<mailto:[e-mail]>; alto<mailto:[e-mail]>
CC: dyncast<mailto:[e-mail]>
Subject: Re: [Dyncast] [alto] Fw: Re: CAN BoF issues #1 #5 #6
Peng,

The following sentence seems not a complete sentence:
"While ALTO tries to solve the problem by exposing information that can be consumed by applications/services before traffic delivery, which can be seen as an alternative way of addressing the problem space of CAN from Application Layer."

How about the following?
"ALTO tries to solve the problem by exposing information that applications/services can consume before traffic delivery, which can be seen as an alternative way of addressing the problem space of CAN from the Application Layer"

Linda
From: [e-mail]<mailto:[e-mail]> <[e-mail]<mailto:[e-mail]>>
Sent: Monday, May 23, 2022 4:33 AM
To: Linda Dunbar <[e-mail]<mailto:[e-mail]>>; luismiguel.contrerasmurillo <[e-mail]<mailto:[e-mail]>>; alto <[e-mail]<mailto:[e-mail]>>
Cc: dyncast <[e-mail]<mailto:[e-mail]>>
Subject: Re: Re: [Dyncast] [alto] Fw: Re: CAN BoF issues #1 #5 #6

Thanks, some revisions based on Linda's reply:

CAN is a network layer solution, trying to solve the problem from on-path forwarding-based decisions and can adapt to different ingress routers caused by UEs roaming among different cell sites (gNBs & UPFs).   So in that respect, even targeting a common problem, both provide different approaches, then imposing different needs but also taking different assumptions on how applications and networks interact.

Compared to the on-path routing solution, since not all applications will query ALTO server(s), especially when UEs roaming among different cell sites (gNBs) triggering different network paths, the ALTO reply for applications/services before traffic delivery might not be optimal or valid after the handover. So, more details are needed of ALTO including some extension to support multi-deployment, quick interaction, integrate more performance metric information.

Regards,
Peng
________________________________
[e-mail]<mailto:[e-mail]>

From: Linda Dunbar<mailto:[e-mail]>
Date: 2022-05-19 01:43
To: LUIS MIGUEL CONTRERAS MURILLO<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; alto<mailto:[e-mail]>
CC: dyncast<mailto:[e-mail]>
Subject: Re: [Dyncast] [alto] Fw: Re: CAN BoF issues #1 #5 #6
Luis has good points.
Maybe the Relationship to ALTO (Issue #5) should be explained this way?

ALTO can be seen as an alternative way of addressing the problem space of computing-aware networking from Application Layer. Since not all applications will query ALTO server(s), especially when UEs roaming among different cell sites (gNBs) triggering different network paths, the ALTO reply for applications/services before traffic delivery might not be optimal or valid after the handover.
[PL] CAN may not support all applications. If there is a specific way for ALTO to solve this problem, and also can be proved to be effective, we can see it as a potential solution.
CAN is a network layer solution, trying to solve the problem from on-path forwarding-based decisions and can adapt to different ingress routers caused by UEs roaming among different cell sites (gNBs & UPFs).  While as ALTO tries to solve the problem by exposing information that can be consumed by applications/services before traffic delivery. So in that respect, even targeting a common problem, both provide different approaches, then imposing different needs but also taking different assumptions on how applications and networks interact.

My two cents,

Linda

From: Dyncast <[e-mail]<mailto:[e-mail]>> On Behalf Of LUIS MIGUEL CONTRERAS MURILLO
Sent: Wednesday, May 18, 2022 9:52 AM
To: [e-mail]<mailto:[e-mail]>; alto <[e-mail]<mailto:[e-mail]>>
Cc: dyncast <[e-mail]<mailto:[e-mail]>>
Subject: Re: [Dyncast] [alto] Fw: Re: CAN BoF issues #1 #5 #6

Hi Peng,

In my view, ALTO can be seen as an alternative (maybe complementary) way of addressing the problem space of computing-aware networking. The CAN proposition at RTG tries to solve the problem from on-path forwarding-based decisions, while ALTO try to solve the problem by exposing information that can be consumed by applications/services before traffic delivery. So in that respect, even targeting a common problem, both provide different approaches, then imposing different needs but also taking different assumptions on how applications and networks interact.

For more detailed comments, please see my answers in-line

Bets regards

Luis

De: alto <[e-mail]<mailto:[e-mail]>> En nombre de [e-mail]<mailto:[e-mail]>
Enviado el: miércoles, 18 de mayo de 2022 15:46
Para: alto <[e-mail]<mailto:[e-mail]>>
CC: dyncast <[e-mail]<mailto:[e-mail]>>
Asunto: [alto] Fw: Re: [Dyncast] CAN BoF issues #1 #5 #6

Hi ALTO WG,

There was a Computing-Aware Networking(CAN) BoF of RTG area in IETF 113, which is to steer the traffic among multiple edge sites considering both network and computing resource statues. The progress was also presented briefly in ALTO WG meeting.

In the BoF, some people cared about the relationship between CAN and ALTO. We collected this issue and got the response from the proponents, also would like to post the clarification to see if there are more comments from the WG. Thanks!

#5 What is the relation between CAN and ALTO? (issue #5)<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCAN-IETF%2FCAN-BoF-ietf113%2Fissues%2F5&data=05%7C01%7Clinda.dunbar%40futurewei.com%7Cee1f9621403840b1560708da3d397e19%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637889614417259588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yGIAUoB6mC65Zn9l42jch4EQPeZQxHl0PJtuvULPPp8%3D&reserved=0>

ALTO architecture has a central ALTO server pulling network status periodically to help managing deployment of the application and computing resource. But it is difficult for ALTO server to promptly assist many ingress nodes in choosing the optimal path based on the dynamic traffic conditions and computing resources at multiple locations because: 1) single point of bottleneck for all ingress routers to query application status;
[Luis>>] I think that this is rather a matter of scalable design, than an actual limitation in the sense that different instances of ALTO server could be put if actually needed.
2) time taken for ingress routers to get the responses from the ALTO server upon flows arrival;
[Luis>>] Again I don't see this as an actual issue in the sense that applications could interrogate ALTO server before deciding what is the best path to follow. We can expect very quick interaction with ALTO to assist on the decision, that will be later on applied to all the packets in the flow (until further optimization could be required by the application). ALTO will have timely information from the network, so always offering fresh info for assisting applications.
3) ALTO server may not know the instantaneous congestion status of the network, all link bandwidths, all information about the actual routing and whether the candidate endpoint itself is overloaded according to RFC7971
[Luis>>] In this respect ALTO can integrate performance metric information as described in draft-ietf-alto-performance-metrics
CAN is to identify various measurements for service instances including the hosting environment, get them normalized together with network metrics for ingress nodes to choose the service instances.

Regards,
Peng
________________________________
[e-mail]<mailto:[e-mail]>

From: Linda Dunbar<mailto:[e-mail]>
Date: 2022-05-18 05:46
To: [e-mail]<mailto:[e-mail]>; dyncast<mailto:[e-mail]>
CC: rtgwg<mailto:[e-mail]>
Subject: Re: [Dyncast] CAN BoF issues #1 #5 #6
Peng,

The resolution for Issue 2 "Relation to ALTO" can add more on why ALTO "can't really help to the service request". How about the following?

Relation to ALTO (issue #5<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCAN-IETF%2FCAN-BoF-ietf113%2Fissues%2F5&data=05%7C01%7Clinda.dunbar%40futurewei.com%7Cee1f9621403840b1560708da3d397e19%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637889614417259588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yGIAUoB6mC65Zn9l42jch4EQPeZQxHl0PJtuvULPPp8%3D&reserved=0>)

ALTO architecture has a central ALTO server pulling network status periodically to help managing deployment of the application and computing resource. But it is difficult for ALTO server to promptly assist many ingress nodes in choosing the optimal path based on the dynamic traffic conditions and computing resources at multiple locations because: 1) single point of bottleneck for all ingress routers to query application status; 2) time taken for ingress routers to get the responses from the ALTO server upon flows arrival; 3) ALTO server may not know the instantaneous congestion status of the network, all link bandwidths, all information about the actual routing and whether the candidate endpoint itself is overloaded according to RFC7971
CAN is to identify various measurements for service instances including the hosting environment, get them normalized together with network metrics for ingress nodes to choose the service instances.  Almost like the reverse of the ALTO.

My two cents,
Linda

From: Dyncast <[e-mail]<mailto:[e-mail]>> On Behalf Of [e-mail]<mailto:[e-mail]>
Sent: Monday, May 16, 2022 6:24 AM
To: dyncast <[e-mail]<mailto:[e-mail]>>
Cc: rtgwg <[e-mail]<mailto:[e-mail]>>
Subject: [Dyncast] CAN BoF issues #1 #5 #6

Dear All,

Based on the categories of the CAN BoF issues, here are the responses to the following issues #1 #5 #6, which clarifies the relationship to ITU-CNC, 3GPP-UPF and ALTO. Any comments are welcome.

We will post the responses to more issues involved in BoF for more comments (https://github.com/CAN-IETF/CAN-BoF-ietf113/issues<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCAN-IETF%2FCAN-BoF-ietf113%2Fissues&data=05%7C01%7Clinda.dunbar%40futurewei.com%7Cee1f9621403840b1560708da3d397e19%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637889614417259588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Oeql75KtRCl%2Fogqgs79%2F1BdMiWxvUf6aJkeDGQufEUc%3D&reserved=0>).  You can also add your comments to any of them. Thanks!

1. What is ITU-CNC and the relationship with CAN #1<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCAN-IETF%2FCAN-BoF-ietf113%2Fissues%2F1&data=05%7C01%7Clinda.dunbar%40futurewei.com%7Cee1f9621403840b1560708da3d397e19%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637889614417259588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=bLP8gTqxvf6iLxzYyi%2BWYSb0nTfFfTPXo44iwf7RffE%3D&reserved=0>

CNC focus on the vision, scenarios, requirements, architecture and network function enhancements for future mobile core network and the telecom fixed, mobile, satellite converged network, but not for internet or routing area. CAN Aims at computing and network resource optimization by steering traffic to appropriate computing resources considering not only routing metric but also computing resource metric and service affiliation.

2. Relation to ALTO #5<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCAN-IETF%2FCAN-BoF-ietf113%2Fissues%2F5&data=05%7C01%7Clinda.dunbar%40futurewei.com%7Cee1f9621403840b1560708da3d397e19%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637889614417259588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yGIAUoB6mC65Zn9l42jch4EQPeZQxHl0PJtuvULPPp8%3D&reserved=0>

ALTO has the potential opportunity to help to the deployment of the application and computing resource but can't really help to the service request because the ALTO service may not know the instantaneous congestion status of the network, all link bandwidths, all information about the actual routing and whether the candidate endpoint itself is overloaded according to RFC7971. Moreover, Alto is an indirection-based method, contrasting with the on-path solution advocated by CAN.

3. Relation to 3GPP UPF #6<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCAN-IETF%2FCAN-BoF-ietf113%2Fissues%2F6&data=05%7C01%7Clinda.dunbar%40futurewei.com%7Cee1f9621403840b1560708da3d397e19%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637889614417259588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=4FVz8Hahd6jxiarwteIIDL7SEPEBYL%2F62fH3Ih1lyMA%3D&reserved=0>

The CAN dyncast work is to depend on the network device to steering traffic other than the UPF. Virtualized UPFs in 5G have a similar issue: multiple UPFs instances can serve a group of gNB nodes. Selecting the UPF instance not only needs UPF load condition but also need network conditions.

Regards,
Peng

________________________________
[e-mail]<mailto:[e-mail]>

From: Linda Dunbar<mailto:[e-mail]>
Date: 2022-05-11 06:11
To: [e-mail]<mailto:[e-mail]>
Subject: [Dyncast] Categories of the CAN BoF issues
CAN BoF proponents:

Many thanks for creating the CAN BoF issues tracking  in the Github: https://github.com/CAN-IETF/CAN-BoF-ietf113/issues/created_by/CAN-IETF?page=1&q=is%3Aopen+is%3Aissue+author%3ACAN-IETF<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FCAN-IETF%2FCAN-BoF-ietf113%2Fissues%2Fcreated_by%2FCAN-IETF%3Fpage%3D1%26q%3Dis%253Aopen%2Bis%253Aissue%2Bauthor%253ACAN-IETF&data=05%7C01%7Clinda.dunbar%40futurewei.com%7Cee1f9621403840b1560708da3d397e19%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C637889614417259588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=NTRY6PyFNMUmbb8fdn%2B6dLpGDkcLMtP6c9xbi9C8HmI%3D&reserved=0>

I went through the issues captured in the Github and characterized them into groups. Some issues can be lumped together for the discussion. There are quite a few issues related to the requirements, which need to be clarified.

Best Regards, Linda


Issues associated with Applications vs. Underlay networks:

*         Consider not to load underlay network with application details. #35

*         We have multiple upper layer application. Do we have additional needs for routing(e.g. WG?) or we are using those applications and won't need such new WG? #30

*         It needs application information too, so it can't just make a decision at the network layer. #23

*         This is not striked as a routing problem; it's all service discovery that can be done in higher layers. #21

*         3GPP and URSP solve this based on UPF selection. It uses both endpoint + application. #20

*         One overlay plane per application. Resources/metric specific to the plane. #19

*         How does the application layer or the transport layer learn the network status to steering traffic? #16

Need more clear requirements for CAN (to be addressed by draft-liu-dyncast-ps-usecases):

*         Need to understand if three are requirement to avoid extra messages or 1ms of latency #36

*         Regarding the flow affinity, is it from network perspective or from application/computation perspective? #33

*         How to effectively compute paths? Shall we put CPUs into account? #32

*         What happens when the user moves? If so we also need to move application context. #25

*         It can only move the services around as fast as it can update the routing plane. which comes back to the point about service discovery (waiting for convergence/distribution as opposed to just updating the SD server) #24

*         Whether the interests of the organization deploying the application and the organization providing the network connectivity are aligned. Google doesn't worry about this because they are both. #17

o    The question is more what the scope and semantic of information is that will need to cross organizational boundaries. This needs further study, in particular when assuming stakeholder division between service and network provider.

*         It seems impossible to satisfy that requirement simultaneously with the latency requirement. #15

*         It wasn't clear that how hard of a requirement session persistence is. #13

o    A session usually creates ephemeral state. If execution changes from one (e.g., virtualized) service instance to another, state/context needs transfer to another. Such required transfer of state/context makes it desirable to have session persistence (or instance affinity) as the default, removing the need for explicit context transfer, while also supporting an explicit state/context transfer (e.g., when metrics change significantly).

*         Should it select UPF based on the application? Steering is done per user? or per application? #9

*         This seems to assume conventional non-distributed applications just running at the edge. what about modern frameworks like Sapphire? and Ray? #7

o    It would be good to understand the multi-site requirements of such framework, which I have understood to mainly run in single DCs.

*         Relation to 3GPP UPF #6

*         Relation to ALTO #5

*         Do the mobility issues and associated protocols are also in scope? There are scenarios where routing alone would not be sufficient. #4

*         What is the position in the edge location regarding to UPF? #3

*         Is there some sort of authorization model so that an edge can indicate whether or not it will provide compute services? #2

*         What is CNC and the relationship with CAN #1


Measurement of the Computing Resources (to be addressed by draft-du-computing-resource-representation):

*         It is hard to use existing work to measure the computation, but we can optimize the latency through the performance monitoring. We have performance/measurement matrix over there. #34

*         Clarifications on the computing resource, its requirements and characteristics would be helpful. #27

*         Each application may have a different definition of "resources" these then have to be boiled down into a single topology Network Aware Computing (NAC! :) does scale #14

*         Is computing resource measurable? #10

o    It is, and how to use the measurement would be solution related. See IFIP Networking 2022 paper on how to simply expose "computing capability" and achieve better steering with such simple measure.

*         Why compute resource is different with other resources? #8

*
Load Balance based solutions:

*         The point is that we need a standardized LB protocol #18

*         The LB as part of the application itself is superior (part of the distributed application itself is to obtain and keep updating the "best" unicast location to use). #22

*         If there is anything missing from current lbs that would prevent their use as-is? other than there is for market reasons no interop standard between different lbs? #12

*         For the load balance, should it learn the network's status? #11

*
Dyncast based Solution issues:

*         For Dyncast, when the time is short, is it possible for the router to decide the routing? It is too fast. #31

*         Is dyncast proposed to encapsulate? #29

*         Will CAN dyncast impact each and every router? How to avoid loops? #28

*         What's the assumed scale of a D-router? 10 ^ 6 sessions? 100^ 8? What's the assumed update rate? !Gb? 1Tb? #26





________________________________

Este mensaje y sus adjuntos se dirigen exclusivamente a su destinatario, puede contener información privilegiada o confidencial y es para uso exclusivo de la persona o entidad de destino. Si no es usted. el destinatario indicado, queda notificado de que la lectura, utilización, divulgación y/o copia sin autorización puede estar prohibida en virtud de la legislación vigente. Si ha recibido este mensaje por error, le rogamos que nos lo comunique inmediatamente por esta misma vía y proceda a su destrucción.

The information contained in this transmission is confidential and privileged information intended only for the use of the individual or entity named above. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, do not read it. Please immediately reply to the sender that you have received this communication in error and then delete it.

Esta mensagem e seus anexos se dirigem exclusivamente ao seu destinatário, pode conter informação privilegiada ou confidencial e é para uso exclusivo da pessoa ou entidade de destino. Se não é vossa senhoria o destinatário indicado, fica notificado de que a leitura, utilização, divulgação e/ou cópia sem autorização pode estar proibida em virtude da legislação vigente. Se recebeu esta mensagem por erro, rogamos-lhe que nos o comunique imediatamente por esta mesma via e proceda a sua destruição