[alto] Fw: [Cats] Thoughts about CATS metrics

[Jordi Ros Giralt <[e-mail]>]

ALTOers,

Forwarding a conversation on metrics in the CATS WG. Feel free to add as needed.

Thanks,
Jordi
________________________________
From: Jordi Ros Giralt <[e-mail]>
Sent: Tuesday, May 16, 2023 23:10
To: [e-mail] <[e-mail]>; [e-mail] <[e-mail]>
Subject: Re: [Cats] Thoughts about CATS metrics

Many thanks Adrian for sharing a summary of the discussion about metrics.

I just wanted to add a comment that has been discussed in the ALTO WG, related to the principle of "avoiding reinventing the wheel" whenever possible. In this regard, since ALTO has also been working on defining key metrics, it would be good to coordinate this effort across the groups. It's likely that we can share common ground.

E.g., the I-D "ALTO Performance Cost Metrics" (https://datatracker.ietf.org/doc/draft-ietf-alto-performance-metrics/) discusses 'delay' too.

Thank you for initiating this conversation.

Jordi





________________________________
From: Cats <[e-mail]> on behalf of Adrian Farrel <[e-mail]>
Sent: Thursday, May 4, 2023 14:30
To: [e-mail] <[e-mail]>
Subject: [Cats] Thoughts about CATS metrics


WARNING: This email originated from outside of Qualcomm. Please be wary of any links or attachments, and do not enable macros.

Hi WG,



A small group had a one hour call last week to discuss metrics for CATS.



This email is to let you know a summary of what we concluded and out proposed next steps. It brings the discussion onto the public mailing list (where it belongs). In keeping with all IETF work, these off-list discussions do not constrain the working group in any way, but hopefully they will stimulate progress.



Please continue the discussion. You’re all encouraged to review and comment on draft-du-cats-computing-modeling-description.



Best Adrian



===



On the call:

  *   Peng Liu and Adrian Farrel as CATS chairs and facilitating the call.
  *   Zongpeng Du as an author of draft-du-cats-computing-modeling-description
  *   Kehan Yao, Joel Halpern, Tony Li, and John Drake



Summary:

  *   It is important the metric scheme used is flexible and extensible to support future requirements for metrics or metric-combination schemes we haven't thought of yet.
  *   For simplicity of specification and implementation, the initial metrics specification should cover only those metrics we think we need to solve immediate problems.
  *   On the call, the only requirement that we identified that would be used to select a server/instance in the immediate use cases is "delay": that includes network propagation time and processing time. But, as above, it must be possible to add new metrics in the future, and email discussions have suggested that we might want a “composite metric”, bandwidth, or server capacity.
  *   The precise meaning of delay for compute and other possible metrics needs to be discussed and standardised if it is to be useful, because the ingress edge and all implementations of the same service need to have a common understanding.



Next Steps:

  *   We think that draft-du-cats-computing-modeling-description contains some good material about the larger problem of characterizing services and servers for many needs, and material that is directly relevant to what metrics CATS should advertise / propagate / use.
  *   Some of us plan to send the authors of draft-du-cats-computing-modeling-description some suggestions for how to add to draft, and how to improve the clarity of the draft.