Re: [Idr] some comments about the draft-ietf-idr-5g-edge-service-metadata-02.txt

["[e-mail]" <[e-mail]>]

Hi, Linda and Vincent

    Please see inline.

Best Regards
Zongpeng Du



[e-mail] & [e-mail]
 
From: Linda Dunbar
Date: 2023-06-10 04:27
To: Shihang(Vincent); Zongpeng; kmajumdar; Wanghaibo (Rainsword); gyan.s.mishra
CC: idr@ietf. org
Subject: RE: [Idr] some comments about the draft-ietf-idr-5g-edge-service-metadata-02.txt
Vincent, 
 
Answers to your comments are inserted below, 
 
Linda
 
From: Shihang(Vincent) <[e-mail]> 
Sent: Thursday, June 8, 2023 10:25 PM
To: Linda Dunbar <[e-mail]>; Zongpeng <[e-mail]>; kmajumdar <[e-mail]>; Wanghaibo (Rainsword) <[e-mail]>; gyan.s.mishra <[e-mail]>
Cc: idr@ietf. org <[e-mail]>
Subject: RE: [Idr] some comments about the draft-ietf-idr-5g-edge-service-metadata-02.txt
 
Hi Linda, Zongpeng,
 
Do we need a type ID to indicate the combination of metrics given the metric TLV already contains the type? Suppose we have N type of metric, then we need N bits to express all the combination of those metrics. 
 
[Linda] Any given metric is encoded in a sub-TLV inside the Service-Metadata-Attribute-Type. The sub-TLV has its own type to indicate its specific format. Two different metrics can have completely different encoding and formats. 
[Zongpeng] Perhaps Vincent is commenting the format ID and ALgo ID, which is proposed in my email. However, I do not mean we must mark every TLV contained, but a type can be understood by the egress and ingress that this type will contain some specific TLVs.
 
I have following comments:
The Capacity Index should be renamed to Site Degradation Index. 
[Linda] Good suggestion. Will change in the next revision. 
 
The site preference may need to be renamed as well. Use ‘Preference’? We do not care it is a preference for a site, or a instance or anything, it is a Preference for the resource that an ingress router should select.
[Linda] The term “site” is intended to refer a group of services attached to an edge that initiates the Service Metrics Metadata. To the ingress router, the “site” is basically the “Nexthop”, which can be the gateway router for a DC, an edge router for a zone within a data center, or a ToR switch. Maybe “Edge Preference” or “Next Hop Preference” is more accurate? 
 [Zongpeng] About the site, what I prefer is the Egress instead of the site. A site ID is strange for me, because the ingress is selecting an Egress, and a dedicated gateway router needs not be exposed.
 
We need two new metrics to describe the service computing power:
a)      Capacity: describes the number of computing units that are available for the instance/site. The value/content is defined and reported by service.
 
[Linda] Is “Remaining available capacity” a better metric? 
 
b)     Load: How many percentage of the computing power is used. It can be measured by network or reported by service. 
[Linda] maybe including Storage, Database capacity, East-west bandwidth among servers within a data center? 
 [Zongpeng] About the Capacity and Load, I agree that we should have a capacity value, which is relatively static, and a Load value, which is relatively dynamic.
 BTW, the computing unit is proposed by Dirk. It is used for LB. Perhaps we can also have another value about the capability, for example, if a server can finish the job fast, it can have a better score.


Best,
Hang
From: Idr <[e-mail]> On Behalf Of Linda Dunbar
Sent: Friday, June 9, 2023 8:07 AM
To: Zongpeng <[e-mail]>; kmajumdar <[e-mail]>; Wanghaibo (Rainsword) <[e-mail]>; gyan.s.mishra <[e-mail]>
Cc: idr@ietf. org <[e-mail]>
Subject: Re: [Idr] some comments about the draft-ietf-idr-5g-edge-service-metadata-02.txt
 
ZongPeng, 
 
The Service Metadata Path Attribute is now defined as to conform with RFC4271. 
 
 
      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |   Attr. Flags |Svc-Metadata-T |        Length (2 Octets)      |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               |
     |         Value (multiple Metadata sub-TLVs)                    |
     |                                                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
              Figure 1: Edge Service Metadata Path Attribute
 
Attr. Flags are defined as:
The high-order bit (bit 0): set to 1.
The second high-order bit (bit 1): set to 0 to indicate that the service-metadata is not transitive. Only intended for the receiving router. 
The third high-order bit (bit 2): same as specified by RFC4721. 
The fourth high-order bit (bit 3): set to 1 to indicate there are two octets for the Length field.  
 
The lower order flags haven’t been specified yet. Can we consider using the lower order flags to represent the categories of the service metadata carried by the TLV? 
 
Linda
 
From: Zongpeng <[e-mail]> 
Sent: Thursday, June 8, 2023 6:46 AM
To: Linda Dunbar <[e-mail]>; kmajumdar <[e-mail]>; rainsword.wang <[e-mail]>; gyan.s.mishra <[e-mail]>
Cc: idr@ietf. org <[e-mail]>; c.l <[e-mail]>
Subject: Re:RE: [Idr] some comments about the draft-ietf-idr-5g-edge-service-metadata-02.txt
 
Hi, Linda
 
         Thanks for the reply.   Please see in line
 
Best Regards
Zongpeng Du
 



 
 
------------------ Original ------------------
From: "Linda Dunbar" <[e-mail]>;
Date: Thu, Jun 8, 2023 08:44 AM
To: "Zongpeng"<[e-mail]>;"kmajumdar"<[e-mail]>;"rainsword.wang"<[e-mail]>;"gyan.s.mishra"<[e-mail]>;
Cc: "idr@ietf. org"<[e-mail]>;"c.l"<[e-mail]>;
Subject: RE: [Idr] some comments about the draft-ietf-idr-5g-edge-service-metadata-02.txt
 
Zong Peng, 
 
Thanks for the suggestions. Comments and replies are inserted below
 
From: [e-mail] <[e-mail]> 
Sent: Wednesday, June 7, 2023 10:50 AM
To: Linda Dunbar <[e-mail]>; kmajumdar <[e-mail]>; rainsword.wang <[e-mail]>; gyan.s.mishra <[e-mail]>
Cc: idr@ietf. org <[e-mail]>; c.l <[e-mail]>
Subject: [Idr] some comments about the draft-ietf-idr-5g-edge-service-metadata-02.txt
 
Hi, Linda, and the authors:
 
    Some comments about draft-ietf-idr-5g-edge-service-metadata-02.txt for your considerations.
 
    About the metric distribution, I think the BGP method is one of the important ways. And perhaps in CATS, we need to start from the simple cases. However, future extensions should be supported.
 IMHO, the way in this document should be supported in future versions of the CATS metric draft, for the scenario that the service instance does not support the direct notification of the computing information.

    For example, we can extend the format with more information to show it is extensible.
      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |   Service-Metadata Type       |        Length (2 Octets)      |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+   
|     Flags     |   FormatID    |   AlgoID      |   Reserved    |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               |
     |         Value (multiple Metadata sub-TLVs)                    |
     |                                                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
              Figure 1: Edge Service Metadata Path Attribute
o  FormatID (1 octets): indicate a specific format, i.e., which set of TLVs would be included.
     o  AlgoID (1 octets): the suggested algorithm to compare the cost to reach the service in different servers.
[Linda] Does “FormatID” and “AlgoID” apply to all the sub-TLVs carried in the “Service-Metadata” Path attribute?  What if the Service metadata carried have different “Formats”? 
•        [zongpeng] What I mean here is a format type here. Perhaps we can change it to
•        o  FormatID (1 octets): indicate a specific format, i.e., which set of TLVs would be included.
•                
•        For example, the formatID is set to 1, which indicates that we would include the TLVs defined in this document:
•        - The site Capacity Index,
•        - The Site Preference Index,
•        - The Load Measurement Index for the attached edge services.
•        If the formatID is set to 2, we would include other TLV set.
 •     For example, Joel has suggested that we can only contain one TLV, i.e. the latency of processing the job. Thus, when the formatID=2, and the processing latency TLV will be included, and optionally a capability TLV can be included. 
•          Of course, we can have other ways to show the design is evolvable, for a new scenario in future . 
 
    Thus, we can define different FormatIDs for different scenarios. 
 
 Another suggestion about the draft is about the migration of the computing information. You can consider whether it is valuable and acceptable.
 In the current CATS framework draft, as shown below, the granularity of the computing information can be per-service instance Or aggregated per-site.
 [Linda] Absolutely should allow metrics Aggregated per-site, per-address-family, or per instance. They are reflected by different sub-TLVs carried within the Metadata Path Attribute. 
•            [zongpeng] I am not quite sure about how the address-family works here. Others are OK. Perhaps we can have a Flag bit to indicate whether the information is aggregated.
   Note that, depending on the design considerations and service   requirements, per-service instance computing-related metrics or   aggregated per-site computing related metrics (and a combination   thereof) can be used by a C-PS.  Using aggregated per-site computing   related metrics appears as a privileged option scalability-wise, but   relies on Egress CATS-Routers that connect to various service   instances to select the proper service instance.
 [Linda] Agree
 However, IMHO, we can also merge the computing information in the per Egress Router granularity. The advantage is that we can merge again the computing information if more than one site connect to the Egress. However, more TLVs are needed here.
[Linda] We should allow both of the following scenarios 
network & DC are under one administrative control. Application running status & environment can be measured and analyzed by DC & application controller (however still need some algorithms to aggregate them together for meaningful indication)
network operator doesn’t have access to the application running status. 
 
Example of Aggregation Descriptions
 
     It is said that a cluster of servers serving the same Application behind a Layer 7 Load balancer can be considered as one merged Application server.
     In this case, the compute metrics from each server within the site will be merged at the site level.
    Optionally, if the Egress can do another round of load balancing among the local sites that connects directly to it, the Egress can merge again the compute metrics from the local sites. The advantage is that we can reduce the metric information that needs to be transferred in the network. But new TLVs need to be designed.
[Linda] Agree.  How about adding the following section?
•            [zongpeng] It is OK for me.
5.2 Service Delay Index 
For a deployment environment where the network and the Edge Data Centers are under one administrative control, e.g., a Mobile Service Provider, the application/service running status & environment metrics collected by DC & application controller are much more meaningful than the IP layer load measurement collected by the Edge routers as described in Section 4.4.  
However, it is not easy to predict which site has “the fastest processing time” or the shortest delay before the service request arrives because:
-      The given service instance shares the same physical infrastructure with many other applications & service instances. Service requests by other applications, UEs, or applications running behavior can impact the processing time for the given service instance.
-      The given service instance can be served by a cluster of servers behind a Load Balancer. To the network, the service is identified by one service ID. 
-      The service complexity is different. One service may call many micro services, need to access multiple backend databases, need to go through sophisticated security scrubbing functions, etc. Another service can be responded with a few simple steps. Without application internal logic, it is difficult to estimate processing time  
 
Here are some metrics, which can be easily collected and measured. They are meaningful in predicting service processing time for a new service request:
o  CPU utilization for the server where the instance is instantiated
o  The network utilization for the links to the server where the instance is instantiated 
o  The number of databases that the service instance will access
o  The memory utilization of the databases
While out of scope, we assume there is an algorithm that can derive the Service Delay Index from all the measurements above. 
 
Thank you
Linda
 
 
Best Regards
Zongpeng Du
 


[e-mail] & [e-mail] 
 
 
[Idr] I-D Action: draft-ietf-idr-5g-edge-service-metadata-02.txt
[e-mail] Thu, 04 May 2023 23:17 UTCShow header
A New Internet-Draft is available from the on-line Internet-Draftsdirectories. This Internet-Draft is a work item of the Inter-Domain Routing(IDR) WG of the IETF.   Title           : BGP Extension for 5G Edge Service Metadata   Authors         : Linda Dunbar                     Kausik Majumdar                     Haibo Wang                     Gyan Mishra   Filename        : draft-ietf-idr-5g-edge-service-metadata-02.txt   Pages           : 19   Date            : 2023-05-04Abstract:   This draft describes a new Metadata Path Attribute and some   sub-TLVs for egress routers to advertise the Edge Service   Metadata of the directly attached edge services (ES). The Edge   Service Metadata can be used by the ingress routers in the 5G   Local Data Network to make path selections not only based on   the routing cost but also the running environment of the edge   services. The goal is to improve latency and performance for   5G edge services.   The extension enables an edge service at one specific location   to be more preferred than the others with the same IP address   (ANYCAST) to receive data flow from a specific source, like   specific User Equipment (UE).The IETF datatracker status page for this Internet-Draft is:https://datatracker.ietf.org/doc/draft-ietf-idr-5g-edge-service-metadata/There is also an htmlized version available at:https://datatracker.ietf.org/doc/html/draft-ietf-idr-5g-edge-service-metadata-02A diff from the previous version is available at:https://author-tools.ietf.org/iddiff?url2=draft-ietf-idr-5g-edge-service-metadata-02