[Cats] Comments on draft-ietf-cats-metric-definition: in-band signalling of the normalization profile, and metric collection time

["Dikshit, Saumya" <[e-mail]>]

Hello Authors of https://datatracker.ietf.org/doc/draft-ietf-cats-metric-definition/

I read -10 version of the draft and two things I want to call out . Kindly have a look:

1.  A departure from the Section 4.3 defaults is not visible in-band
--------------------------------------------------

Section 4.3 is honest about the underlying issue, and I think the framing is right:
   "While normalization functions can convert raw metrics into
   unitless scores to enable comparison, these scores may not be
   directly comparable across different implementations."

and then fixes it with a default policy:
   "Unless otherwise specified by the implementation in accompanying
   documentation, scores in the range 0-3 indicate low capability
   (not recommended for steering), 4-7 indicate medium capability
   (steering optional), and 8-10 indicate high capability (priority
   for steering).  This mapping is normative for all CATS Level 1 and
   Level 2 metrics defined in this document."

My concern is the escape hatch.  Both it and the equivalent one for the normalization function :
   "Other functions (e.g., sigmoid) are permitted but their
   parameters must be documented."
are satisfied by out-of-band documentation.

So a C-PS that receives a Level 2 score of 7 from two implementations has no way,  from the message alone, to tell whether both used the default mapping or whether one of them documented something else in its release notes.

  *
The Section 4.1 field list (Metric_Type, Level, Format, Length, Unit, Source, Statistics, Value) doesn't carry that.
  *
 ‘Source' with value 'normalization' tells me normalization happened, not which mapping was applied.

That matters more than it first looks, because Section 3.4 already says:
   "Different implementations may apply different functions to
   characterize the overall performance of the underlying computing
   and communication resources."

Two suggestions, if agreeable and FWIW can be further discussed :

  1.
If we can add a field to Section 4.1 - something like Normalization_Profile’ , whose default value means “the Section 4.3 defaults apply", and which otherwise identifies the profile in use.
     *
A receiver can then at least detect that two sources are not on the same scale, even if it can't reconcile them.

  1.
 If the WG prefers not to add a field, it can be plainly put in Section 4.3 that cross-implementation comparability of Level 2 scores is not guaranteed by this document and is a deployment concern.  T

I'd lean to #1, because the cost is one field and it makes the mismatch detectable


2.  Metrics carry a measurement window but not a collection time
--------------------------------------------------
Section 4.1 has no field for when the measurement was taken. Measurement_Window appears in the per-metric registry entries and gives the length of the window, which is a different thing.

This collides with O-REQ-4 in draft-ietf-cats-oam-fw-01:

   "O-REQ-4: Computing Metric Freshness Handling.  Computing metrics
   SHOULD be accompanied by timestamps indicating the time of
   collection.  This enables the C-SMA or C-PS to optionally
   configure a maximum acceptable staleness threshold for each metric
   type to determine subsequent handling."

As things stand, an implementation can be compliant with -10 and still not give the C-PS what O-REQ-4 asks for.

  *
Adding an optional collection-time field to Section 4.1 would close that and
  *
would also give the C-PS something to use when a computing metric goes quiet .
  *
As we know, today "no update" and "unchanged value" look the same.

Note that, this is separate from the freshness handling in SEC-4, which is about replay protection rather than about the age of a value being an input to a steering the decision.
Both are needed and they're not the same requirement.

I've been putting together a short use-cases companion to the OAM framework draft (draft-dikshit-cats-oam-usecases),
and the two gaps above are what made two of its use cases little tricky  to pen down concretely :

  *
 the closed-loop SLA one needs metric age, and
  *
the multi-vendor one

needs some way to tell whether two scores are on the same scale.
.
Happy to write text for either if that's useful.

Thanks,
Saumya