Re: [ippm] Call for Adoption of draft-mhmcsfh-ippm-pam

[Tommy Pauly <[e-mail]>]

Hi Greg, Benoit,

We haven’t closed this call yet, since we’d like to see if we can resolve these issues and come to consensus before adoption.

Benoit, can you continue this discussion with Greg on clarifying things?

Thanks,
Tommy

> On Sep 14, 2022, at 1:50 AM, Greg Mirsky <[e-mail]> wrote:
> 
> Hi Benoit,
> thank you for your comments and questions. Please find my notes in-lined below under the GIM>> tag. I am looking forward to continuing our discussion.
> 
> Regards,
> Greg
> 
> On Mon, Sep 5, 2022 at 9:21 AM Benoit Claise <benoit.claise=[e-mail] <mailto:[e-mail]>> wrote:
>> Dear all,
>> 
>> I don't dispute the importance of this work. However, the scope of this work is not clear yet IMO.
>> 
>>  - SLO, sure,  but what's not clear to me is: SLO per customer, per service, per class of service, per flow, per application
>> I found "Precision Availability Metrics (PAM), aimed at capturing end-to-end service levels for a flow, specifically the degree to which flows comply with the SLOs that are in effect". 
>> So OK, we speak about flow. So what is your flow definition?
> GIM>> The scope of monitoring is the same as the scope of SLA that is composed of the set of SLOs. 
>> - Btw, based on the previous quoted sentence, I don't understand this PAM name. No mention of SLA, no mention of flow, no notion of service.
>>  Basically, you report a service level indicator (SLI). You confused me with PAM
> GIM>> The intention is to report not raw SLI, i.e., measurable metric, but rather how the SLI is conforming to its SLO.
>  
>> - How are you going to report this flow definition, along with the SLI? IPFIX key fields? With a YANG model?
>> This section 6 content is key to understand how to use those SLIs in an operational environment
>>    The following is a list of items for which further discussion is
>>    needed as to whether they should be included in the scope of this
>>    specification:
>> 
>>    *  A YANG data model.
>> 
>>    *  A set of IPFIX Information Elements.
>> 
>>    *  Statistical metrics: e.g., histograms/buckets.
> GIM>> We welcome collaboration on all or any of these problems.
>  
>> - I am not a big fan to specify some level of thresholding in specifications.
>>    *  VI is a time interval during which at least one of the performance
>>       parameters degraded below its pre-defined optimal level threshold.
>> 
>>    *  SVI is a time interval during which at least one the performance
>>       parameters degraded below its pre-defined critical threshold.
>> 
>> Based on my experience, most of the time, we don't get the threshold values/names right, and we don't get the number of them right.
>>     ex: violated, severely violated ... why not extremely violated, catastrophically violated?
> GIM>> Agree that it might take several iterations to set thresholds right. Would note that draft-ietf-teas-ietf-network-slices <https://datatracker.ietf.org/doc/draft-ietf-teas-ietf-network-slices/> gives and example of SLO in Section 4.1 using target/bound values, i.e., thresholds, as following:
>    *  A Service Level Objective (SLO) is a target value or range for the
>       measurements returned by observation of an SLI.  For example, an
>       SLO may be expressed as "SLI <= target", or "lower bound <= SLI <=
>       upper bound".  A customer can determine whether the provider is
>       meeting the SLOs by performing measurements on the traffic.
>  
>> Trying to express, from the measurement aspects, whether the observations are SEVERELY impacting (that's the way I read SVI) is not the right approach IMO.
>> This is maybe you open issues in section 6
>>     * Policies regarding the definition of "violated" and "severely violated" time interval.
> GIM>> Yes, that is our intention to further work on improving these definitions. 
>> 
>> Bottom line: 
>> Granted, IPPM is about performance metrics but specifying metrics without specifying how they will be used in an operational environment is not the right way IMO.
>> I believe the scope of this document is NOT clear enough to be adopted. In other words, I don't know what I'm signing for...
>> 
>> Regards, Benoit
>> 
>> On 9/1/2022 7:25 PM, Tommy Pauly wrote:
>>> Hello IPPM,
>>> 
>>> As discussed at IETF 114, we’re starting an adoption call for Precision Availability Metrics for SLO-Governed End-to-End Services, draft-mhmcsfh-ippm-pam.
>>> 
>>> https://datatracker.ietf.org/doc/draft-mhmcsfh-ippm-pam/
>>> 
>>> The current version is here:
>>> 
>>> https://www.ietf.org/archive/id/draft-mhmcsfh-ippm-pam-02.html
>>> 
>>> Please reply to this email by Thursday, September 15, to indicate whether you support adoption of this draft.
>>> 
>>> Best,
>>> Tommy & Marcus
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> ippm mailing list
>>> [e-mail] <mailto:[e-mail]>
>>> https://www.ietf.org/mailman/listinfo/ippm
>> 
>> _______________________________________________
>> ippm mailing list
>> [e-mail] <mailto:[e-mail]>
>> https://www.ietf.org/mailman/listinfo/ippm
> _______________________________________________
> ippm mailing list
> [e-mail] <mailto:[e-mail]>
> https://www.ietf.org/mailman/listinfo/ippm