Re: [ippm] WGLC for draft-ietf-ippm-pam

[Greg Mirsky <[e-mail]>]

Hi Luis,
thank you for supporting the work on PAM and for your thoughtful comments
and suggestions. Please find responses below inline under the GIM>> tag.
Please review the attached diff that highlights updates (including those to
address comments received from Thomas)

Regards,
Greg

On Fri, Aug 11, 2023 at 3:43 AM Luis M. Contreras <[e-mail]>
wrote:

> Hi all,
>
> After reviewing the document, I do support the progress of this draft as
> it provides a useful approach for service characterization when multiple
> SLOs are in place.
>
> Some few comments / questions:
>
> - Abstract refers to Network Slice service, shouldn’t be better to name it
> as IETF Network Slice service for consistency with TEAS work?
>
> - Also in Abstract: "PAM can be used by providers and/or users of a
> Network Slice service". It would be good to align terminology with the one
> in IETF Network Slice framework (e.g., customer instead of user).
>
GIM>> Thank you for pointing that out. Perhaps the following update be
acceptable addressing both comments above:

OLD TEXT:
   For example, PAM can be
   used by providers and/or users of a Network Slice service to assess
   whether the service is provided in compliance with its defined SLOs.
NEW TEXT:
   For example, PAM can be
   used by providers and/or customers of an network slice service, e.g.,
   IETF Network Slice Service, to assess whether the service is provided
   in compliance with its defined SLOs.

> - When introducing violated intervals, in section 3.1 it is referred that
> the calculation is done between two nodes. I assume those nodes should not
> be necessary adjacent nodes, but maybe it could be worth clarifying that.
>
GIM>> I agree, the determination of an interval (violated, severely
violated, or free of violation) follows the determination of the applicable
SLO. Would the following update be acceptable?

OLD TEXT:

   Mechanisms of setting levels of threshold of an SLO are outside the

   scope of this document.

NEW TEXT:

   The monitoring performance parameters to determine the quality of an

   interval is performed between the elements of the network that are

   referred to for the SLO corresponding to the performance parameter.

   Mechanisms of setting levels of a threshold of an SLO are outside the

   scope of this document.

> Furthermore, thinking on IETF Network Slice services, there could be
> additional considerations: (1) the calculation can involve more than two
> nodes (e.g., in point to multipoint or any to any connectivity constructs),
>
GIM>> To the best of my understanding, SLA and SLO, even for p2mp or mp2mp
services, are defined between two UNIs. WDYT?

> and (2) the definition probably need to consider the fact of being
> measured between SDPs (i.e., client ports of nodes, even considering the
> situation where those client ports could also be part of the same node).
> With these considerations, the mentioned calculation is equally
> applicable?
>
GIM>> I think that the list of where a measurement point (MP) may be
located can be extended by adding, for example, a probe. Often (I don't
know if "always") the location of MPs is specified in the SLA or a
normative document that defines the relevant Service OAM. And yes, the
monitoring of PAM and calculation of PAM-related metrics would be
applicable if an MP is placed at an SDP. Would you suggest a text to add?

> - How to account for events that could impact more than one SLO at the
> same time? For instance, a mis-ordered packet could impact at the same time
> the SLOs of packet ordering and jitter (for instance). It would be
> interesting to have some guidance on how to consider those situations,
> i.e., if the SLO violation should be accounted per SLO or summarized for
> instance accounting simply the one of highest severity. Another possibility
> could be to include an additional indicator reflecting these situations.
>
GIM>> That is an interesting scenario. I think that we have covered that by
pointing out that the violation is determined by the degradation of at
least one of the set of monitored parameters:

   *  VI is a time interval during which at least one of the performance

      parameters degraded below its pre-defined optimal level threshold.

   *  SVI is a time interval during which at least one the performance

      parameters degraded below its pre-defined critical threshold.
Thus, as I think of that scenario, a failure may constitute itself as a VI
or SVI of one or more SLOs.

> - Being PAM targeting multi-SLO services, would it not be worthy to define
> the same kind of precision metrics per SLO as well? As today, the
> precision metrics show an aggregated view of the potential issues found,
> but it would be beneficial to provide more detailed insights by showing the
> same kind of metrics per SLO so that the provider or consumer can have more
> insightful information about what issue is affecting the service.
>
GIM>> If we imagine a service that is defined by a single SLO, then, in my
opinion, the PAM is applicable as we have defined it. Do you think that we
need to add some text illustrating how PAM metrics can be applied to each
SLO that composes SLA and then to the combination of those SLOs as a
multi-SLO governed service?

>
>
> Thanks,
>
> best regards
>
> Luis
>
>
>
>
>
>
>
>
>
>
>
> El mar, 8 ago 2023 a las 17:30, Tommy Pauly (<tpauly=
> [e-mail]>) escribió:
>
>> Hello IPPM,
>>
>> This email starts a Working Group Last Call for "Precision Availability
>> Metrics for Services Governed by Service Level Objectives
>> (SLOs)”, draft-ietf-ippm-pam.
>>
>> https://www.ietf.org/archive/id/draft-ietf-ippm-pam-05.html
>> https://datatracker.ietf.org/doc/draft-ietf-ippm-pam/
>>
>> Please review the document and send your comments in response to this
>> email, along with whether you think the document is ready to progress.
>>
>> Please send your reviews and feedback by *Tuesday, August 22*.
>>
>> Best,
>> Tommy & Marcus
>> _______________________________________________
>> ippm mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/ippm
>>
>
>
> --
> ___________________________________________
> Luis M. Contreras
> [e-mail]
> [e-mail]
> Global CTIO unit / Telefonica
> _______________________________________________
> ippm mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/ippm
>