Re: [Teas] What SLOs and SLEs should be in the Slicing Service Model?

[Gyan Mishra <[e-mail]>]

John / All

Per this thread where we are discussing what SLO/SLE taxonomy should be in
the service model does bring up some questions related to the network slice
draft.

In the thread as Adrian started off the thread stating that Protection
qualifies as an SLE  but Reliability is definitely an SLO and there may be
a gray area of whether or not it's one or the other.

I wonder if there could be characteristics that could be both SLO & SLE?

The term measurable  & unmeasurable realization feature really helps define
what characteristics define an SLO or SLE.

Does not have to be an exhaustive list but I think more than just the
common SLO's & SLE's.  I think the gray area ones we should try to spell
out in the draft.

Below are good points that Med brought up and I think are applicable to
section 4 of the draft. Especially that over time the what maybe an SLE
today maybe an SLO tomorrow and vice versa.

Flip side of that is maybe to only spell out the concrete ones that will
never change sides.

**Mentioned by Med at the beginning of this thread***

* Things would be much simpler if we just focus on service requirements
without making an assumption how such requirement is expressed and whether
it is quantified/quantitative/qualitative/measurable/etc. Whether/how a
specific service requirement is covered by service assurance/fulfillment
can be part of the slice service definition itself.

* What we may tag as an SLE today because of the technology limitations,
may not stay as such forever. It may be true that an "expectation" may not
be easily assessed using current techniques (and thus be tagged as SLE),
but this does not prevent that innovative means would be defined in the
future (which means that it is an SLO, not an SLE anymore).

* We are artificially adding extra complexity for the modelling part as
service requirement will need to be classified based as SLO or SLEs.

We can branch this off on a separate thread if everyone thinks there is any
merit to follow up on this topic.

Kind Regards

Gyan

On Fri, Mar 25, 2022 at 5:55 PM John E Drake <[e-mail]> wrote:

> Did we not make it clear that SLOs and SLEs are used to create an SLA?
>
> Yours Irrespectively,
>
> John
>
>
> Juniper Business Use Only
>
> > -----Original Message-----
> > From: Teas <[e-mail]> On Behalf Of Joel M. Halpern
> > Sent: Friday, March 25, 2022 3:52 PM
> > To: Gyan Mishra <[e-mail]>
> > Cc: [e-mail]; TEAS WG <[e-mail]>
> > Subject: Re: [Teas] What SLOs and SLEs should be in the Slicing Service
> Model?
> >
> > [External Email. Be cautious of content]
> >
> >
> > Sounds like we need to clarify the verbiage.  SLEs are, as I understand
> it, indeed
> > non-measurables.  But they are not the legal translation of the SLOs.
> They are
> > expression of customer expectations that are not directly observable or
> > measurable.  The example I have the easiest time understanding is that
> the
> > customer expects the operator to encrypt the traffic across the service.
> >
> > Yours,
> > Joel
> >
> > On 3/25/2022 3:07 PM, Gyan Mishra wrote:
> > > Hi Med & Joel
> > >
> > > I reread section 4.1 of the slice draft and to me it seems that SLO is
> > > from provider POV measurable indicators and SLE is the is unmeasurable
> > > expectations which is makes up the customer fulfillment of obligations
> > > by the provider which would be like taking the tangible measurement
> > > characteristics from the SLO and translation into legal obligation
> > > fulfillment verbiage that goes into the service agreement.  So as the
> > > SLE is not tangible but just a translation of SLO metrics into legal
> > > verbiage my thoughts are that the framework as well as Yang model need
> > > only focus on the SLO tangible metrics and leave the intangible for
> > > the lawyers writing the customer agreement.
> > >
> > >
> > > 4.1
> > > <
> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/html/draft-
> > ietf-teas-ietf-network-slices-09*section-4.1__;Iw!!NEt6yMaO-
> > gk!VrEVB46KF20816iW_4FlbM4JGWGSLyFu-vk4PGWX_-
> > LiOR92MnzCJzyL9ESsViw$ >.
> > > Objectives for IETF Network Slices
> > >
> > >     An IETF Network Slice service is defined in terms of quantifiable
> > >     characteristics known as Service Level Objectives (SLOs) and
> > >     unquantifiable characteristics known as Service Level Expectations
> > >     (SLEs).  SLOs are expressed in terms Service Level Indicators
> (SLIs),
> > >     and together with the SLEs form the contractual agreement between
> > >     service customer and service provider known as a Service Level
> > >     Agreement (SLA).
> > >
> > >     The terms are defined as follows:
> > >
> > >     *  A Service Level Indicator (SLI) is a quantifiable measure of an
> > >        aspect of the performance of a network.  For example, it may be
> a
> > >        measure of throughput in bits per second, or it may be a measure
> > >        of latency in milliseconds.
> > >
> > >     *  A Service Level Objective (SLO) is a target value or range for
> the
> > >        measurements returned by observation of an SLI.  For example, an
> > >        SLO may be expressed as "SLI <= target", or "lower bound <= SLI
> <=
> > >        upper bound".  A customer can determine whether the provider is
> > >        meeting the SLOs by performing measurements on the traffic.
> > >
> > >     *  A Service Level Expectation (SLE) is an expression of an
> > >        unmeasurable service-related request that a customer of an IETF
> > >        Network Slice makes of the provider.  An SLE is distinct from an
> > >        SLO because the customer may have little or no way of
> determining
> > >        whether the SLE is being met, but they still contract with the
> > >        provider for a service that meets the expectation.
> > >
> > >
> > >
> > > On Thu, Mar 24, 2022 at 10:36 AM <[e-mail]
> > > <mailto:[e-mail]>> wrote:
> > >
> > >     Re-,
> > >
> > >     They shouldn't!
> > >
> > >     This is why I'm suggesting to not have that tagging frozen in the
> > >     model. We can simply have a provision for a set of parameters.
> These
> > >     parameters will be included to characterize the service with a
> > >     service assurance component that will call out the identity of the
> > >     subset of parameters that will be used as committed ones. That
> > >     component will also include other data to ensure the same based
> used
> > >     for assessment, etc.
> > >
> > >     Cheers,
> > >     Med
> > >
> > >      > -----Message d'origine-----
> > >      > De : Joel M. Halpern <[e-mail]
> > >     <mailto:[e-mail]>>
> > >      > Envoyé : jeudi 24 mars 2022 15:29
> > >      > À : BOUCADAIR Mohamed INNOV/NET
> > <[e-mail]
> > >     <mailto:[e-mail]>>
> > >      > Cc : 'TEAS WG' <[e-mail] <mailto:[e-mail]>>;
> > >     [e-mail] <mailto:[e-mail]>
> > >      > Objet : Re: [Teas] What SLOs and SLEs should be in the Slicing
> > >     Service
> > >      > Model?
> > >      >
> > >      > If the others are best effort, why would they be included in
> the SLO?
> > >      > (We do not assume that every IETF network slice service will
> > >     specify all
> > >      > possible parameters.)
> > >      >
> > >      > Yours,
> > >      > Joel
> > >      >
> > >      > On 3/24/2022 10:20 AM, [e-mail]
> > >     <mailto:[e-mail]> wrote:
> > >      > > Re-,
> > >      > >
> > >      > > Some services may be sensitive to delay for example but the
> > >     slice service
> > >      > request may include (for whatever reason) not only the delay,
> but
> > >     also
> > >      > other attributes that are currently listed as SLOs in the
> draft. The
> > >      > provider is requested to only commit on the delay and best
> effort
> > >     for the
> > >      > other attributes.
> > >      > >
> > >      > > Cheers,
> > >      > > Med
> > >      > >
> > >      > >> -----Message d'origine-----
> > >      > >> De : Joel M. Halpern <[e-mail]
> > >     <mailto:[e-mail]>> Envoyé : jeudi 24 mars
> > >      > >> 2022 15:07 À : BOUCADAIR Mohamed INNOV/NET
> > >      > >> <[e-mail]
> > >     <mailto:[e-mail]>> Cc : 'TEAS WG'
> > <[e-mail]
> > >     <mailto:[e-mail]>>;
> > >      > >> [e-mail] <mailto:[e-mail]> Objet : Re:
> > >     [Teas] What SLOs and SLEs should be
> > >      > >> in the Slicing Service Model?
> > >      > >>
> > >      > >> Interesting.  If they are indeed not coupled, then you are
> clearly
> > >      > >> correct about representation.
> > >      > >>
> > >      > >> Can you give an example so I can understand when they would
> not be
> > >      > coupled.
> > >      > >> I had leapt to the (quite possibly incorrect) conclusion that
> > >     the two
> > >      > >> sets of properties went together.
> > >      > >>
> > >      > >> Thank you,
> > >      > >> Joel
> > >      > >>
> > >      > >> On 3/24/2022 10:03 AM, [e-mail]
> > >     <mailto:[e-mail]> wrote:
> > >      > >>> Hi Joel,
> > >      > >>>
> > >      > >>> It is.
> > >      > >>>
> > >      > >>> As mentioned below, this should be covered as part of
> service
> > >      > >> assurance/fulfillment/reporting parameters. Which parameters
> > >     to put
> > >      > >> there is deployment-specific. Not all parameters tagged as
> SLO
> > >     in the
> > >      > >> framework will end up as part of the
> > >     assurance/fulfillment/reporting.
> > >      > >>>
> > >      > >>> Cheers,
> > >      > >>> Med
> > >      > >>>
> > >      > >>>> -----Message d'origine-----
> > >      > >>>> De : Joel M. Halpern <[e-mail]
> > >     <mailto:[e-mail]>> Envoyé : jeudi 24 mars
> > >      > >>>> 2022 14:52 À : BOUCADAIR Mohamed INNOV/NET
> > >      > >>>> <[e-mail]
> > >     <mailto:[e-mail]>>; [e-mail]
> > >     <mailto:[e-mail]>; 'TEAS WG'
> > >      > >>>> <[e-mail] <mailto:[e-mail]>> Objet : Re: [Teas]
> > >     What SLOs and SLEs should be in
> > >      > >>>> the Slicing Service Model?
> > >      > >>>>
> > >      > >>>> Isn't it important to distinguish between "these are
> things I
> > >      > >>>> expect you to do, measure, and report" and "these are
> things I
> > >      > >>>> would like you to do even though they are not measurable or
> > >      > reportable"?
> > >      > >>>>
> > >      > >>>> Yours,
> > >      > >>>> Joel
> > >      > >>>>
> > >      > >>>> On 3/23/2022 10:08 AM, [e-mail]
> > >     <mailto:[e-mail]> wrote:
> > >      > >>>>> Hi all,
> > >      > >>>>>
> > >      > >>>>> This message actually triggers a companion comment I have
> > >     on the
> > >      > >>>>> SLO/SLE
> > >      > >>>> taxonomy. From the service modeling standpoint, I suggest
> > >     that we
> > >      > >>>> don't inherit that taxonomy for various reasons:
> > >      > >>>>>
> > >      > >>>>> * Things would be much simpler if we just focus on service
> > >      > >>>>> requirements
> > >      > >>>> without making an assumption how such requirement is
> > >     expressed and
> > >      > >>>> whether it is
> > >     quantified/quantitative/qualitative/measurable/etc.
> > >      > >>>> Whether/how a specific service requirement is covered by
> service
> > >      > >>>> assurance/fulfillment can be part of the slice service
> > >     definition
> > >      > >> itself.
> > >      > >>>>>
> > >      > >>>>> * What we may tag as an SLE today because of the
> technology
> > >      > >>>>> limitations,
> > >      > >>>> may not stay as such forever. It may be true that an
> > >     "expectation"
> > >      > >>>> may not be easily assessed using current techniques (and
> thus be
> > >      > >>>> tagged as SLE), but this does not prevent that innovative
> means
> > >      > >>>> would be defined in the future (which means that it is an
> > >     SLO, not
> > >      > >>>> an SLE
> > >      > >> anymore).
> > >      > >>>>>
> > >      > >>>>> * We are artificially adding extra complexity for the
> modelling
> > >      > >>>>> part as
> > >      > >>>> service requirement will need to be classified based as SLO
> > >     or SLEs.
> > >      > >>>>>
> > >      > >>>>> * I remember that Kiran agreed at least to not import that
> > >      > >>>>> taxonomy into
> > >      > >>>> the data model when we were discussing the call for
> adoption
> > >     of the
> > >      > >>>> slice
> > >      > >>>> definition:
> > >      > >>>>>
> > >      > >>>>> ==(the full message from Kiran can be found in the
> archives)===
> > >      > >>>>> "However, it should not imply that NBI models are required
> > >     to have
> > >      > >>>> SLE/SLO indicators and I totally agreed with your comments
> on
> > >      > >>>> draft-wd- teas-ietf-network-slice-nbi-yang-03"
> > >      > >>>>> ==
> > >      > >>>>>
> > >      > >>>>> Cheers,
> > >      > >>>>> Med
> > >      > >>>>>
> > >      > >>>>>> -----Message d'origine-----
> > >      > >>>>>> De : Teas <[e-mail]
> > >     <mailto:[e-mail]>> De la part de Adrian Farrel
> > >      > >>>>>> Envoyé
> > >      > >>>>>> : mercredi 23 mars 2022 14:47 À : 'TEAS WG' <
> [e-mail]
> > >     <mailto:[e-mail]>> Objet :
> > >      > >>>>>> [Teas] What SLOs and SLEs should be in the Slicing
> Service
> > >     Model?
> > >      > >>>>>>
> > >      > >>>>>> Hi,
> > >      > >>>>>>
> > >      > >>>>>> Sorry for my audio being a mess in TEAS today.
> > >      > >>>>>>
> > >      > >>>>>> I believe I heard the discussion between Kireeti and Reza
> > >      > >>>>>> correctly, and there was some follow-up in the chat.
> > >      > >>>>>>
> > >      > >>>>>> I agree that "Protection" is a realisation feature and
> so it
> > >      > >>>>>> qualifies as an SLE, if at all.
> > >      > >>>>>> But "Reliability" is clearly an SLO. Usually expressed as
> > >     maximum
> > >      > >>>>>> down- time per unit of time, or maximum lost traffic.
> > >      > >>>>>>
> > >      > >>>>>> There was one thing that I *think* I heard. This was Reza
> > >     saying
> > >      > >>>>>> that the service YANG model was only including the SLOs
> > >     and SLEs
> > >      > >>>>>> noted in the framework. Maybe I misheard "only" because I
> > >     think
> > >      > >>>>>> that might be a mistake.
> > >      > >>>>>> The YANG model should certainly be interested in what the
> > >      > >>>>>> framework says, but it is not a requirement that all
> SLOs and
> > >      > >>>>>> SLEs in the framework be in the model if the authors
> find that
> > >      > >>>>>> there is no interest in implementing them, and there
> should
> > >      > >>>>>> certainly be no limitation about including additional
> SLOs
> > >     or SLEs
> > >      > in the YANG model.
> > >      > >>>>>> Indeed, the SLOs and SLEs listed in the framework are
> > >     presented
> > >      > >>>>>> as
> > >      > >>>> examples.
> > >      > >>>>>>
> > >      > >>>>>> Cheers,
> > >      > >>>>>> Adrian
> > >      > >>>>>>
> > >      > >>>>>> _______________________________________________
> > >      > >>>>>> Teas mailing list
> > >      > >>>>>> [e-mail] <mailto:[e-mail]>
> > >      > >>>>>>
> >
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/teas__;!!N
> > Et6yMaO-gk!VrEVB46KF20816iW_4FlbM4JGWGSLyFu-vk4PGWX_-
> > LiOR92MnzCJzyLzCREI4M$
> > >
> > <
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/teas__;
> !!
> > NEt6yMaO-gk!VrEVB46KF20816iW_4FlbM4JGWGSLyFu-vk4PGWX_-
> > LiOR92MnzCJzyLzCREI4M$ >
> > >      > >>>>>
> > >      > >>>>>
> > >
> > _________________________________________________________________
> > _
> > >      > >>>>> __ __
> > ___________________________________________________
> > >      > >>>>>
> > >      > >>>>> Ce message et ses pieces jointes peuvent contenir des
> > >     informations
> > >      > >>>>> confidentielles ou privilegiees et ne doivent donc pas
> etre
> > >      > >>>>> diffuses, exploites ou copies sans autorisation. Si vous
> > >     avez recu
> > >      > >>>>> ce message par erreur, veuillez le signaler a l'expediteur
> > >     et le
> > >      > >>>>> detruire ainsi que
> > >      > >>>> les pieces jointes. Les messages electroniques etant
> > >     susceptibles
> > >      > >>>> d'alteration, Orange decline toute responsabilite si ce
> > >     message a
> > >      > >>>> ete altere, deforme ou falsifie. Merci.
> > >      > >>>>>
> > >      > >>>>> This message and its attachments may contain confidential
> or
> > >      > >>>>> privileged information that may be protected by law; they
> > >     should
> > >      > >>>>> not be
> > >      > >>>> distributed, used or copied without authorisation.
> > >      > >>>>> If you have received this email in error, please notify
> the
> > >     sender
> > >      > >>>>> and
> > >      > >>>> delete this message and its attachments.
> > >      > >>>>> As emails may be altered, Orange is not liable for
> messages
> > >     that
> > >      > >>>>> have
> > >      > >>>> been modified, changed or falsified.
> > >      > >>>>> Thank you.
> > >      > >>>>>
> > >      > >>>>> _______________________________________________
> > >      > >>>>> Teas mailing list
> > >      > >>>>> [e-mail] <mailto:[e-mail]>
> > >      > >>>>>
> >
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/teas__;!!N
> > Et6yMaO-gk!VrEVB46KF20816iW_4FlbM4JGWGSLyFu-vk4PGWX_-
> > LiOR92MnzCJzyLzCREI4M$
> > >
> > <
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/teas__;
> !!
> > NEt6yMaO-gk!VrEVB46KF20816iW_4FlbM4JGWGSLyFu-vk4PGWX_-
> > LiOR92MnzCJzyLzCREI4M$ >
> > >      > >>>
> > >      > >>>
> > >
> > _________________________________________________________________
> > ___
> > >      > >>> __ ___________________________________________________
> > >      > >>>
> > >      > >>> Ce message et ses pieces jointes peuvent contenir des
> > >     informations
> > >      > >>> confidentielles ou privilegiees et ne doivent donc pas etre
> > >      > >>> diffuses, exploites ou copies sans autorisation. Si vous
> avez
> > >     recu
> > >      > >>> ce message par erreur, veuillez le signaler a l'expediteur
> et le
> > >      > >>> detruire ainsi que
> > >      > >> les pieces jointes. Les messages electroniques etant
> susceptibles
> > >      > >> d'alteration, Orange decline toute responsabilite si ce
> > >     message a ete
> > >      > >> altere, deforme ou falsifie. Merci.
> > >      > >>>
> > >      > >>> This message and its attachments may contain confidential or
> > >      > >>> privileged information that may be protected by law; they
> > >     should not
> > >      > >>> be
> > >      > >> distributed, used or copied without authorisation.
> > >      > >>> If you have received this email in error, please notify the
> > >     sender
> > >      > >>> and
> > >      > >> delete this message and its attachments.
> > >      > >>> As emails may be altered, Orange is not liable for messages
> that
> > >      > >>> have
> > >      > >> been modified, changed or falsified.
> > >      > >>> Thank you.
> > >      > >>>
> > >      > >
> > >      > >
> > >
> > _________________________________________________________________
> > _____
> > >      > > ___________________________________________________
> > >      > >
> > >      > > Ce message et ses pieces jointes peuvent contenir des
> informations
> > >      > > confidentielles ou privilegiees et ne doivent donc pas etre
> > >     diffuses,
> > >      > > exploites ou copies sans autorisation. Si vous avez recu ce
> message
> > >      > > par erreur, veuillez le signaler a l'expediteur et le detruire
> > >     ainsi que
> > >      > les pieces jointes. Les messages electroniques etant
> susceptibles
> > >      > d'alteration, Orange decline toute responsabilite si ce message
> a ete
> > >      > altere, deforme ou falsifie. Merci.
> > >      > >
> > >      > > This message and its attachments may contain confidential or
> > >      > > privileged information that may be protected by law; they
> > >     should not be
> > >      > distributed, used or copied without authorisation.
> > >      > > If you have received this email in error, please notify the
> > >     sender and
> > >      > delete this message and its attachments.
> > >      > > As emails may be altered, Orange is not liable for messages
> > >     that have
> > >      > been modified, changed or falsified.
> > >      > > Thank you.
> > >      > >
> > >
> > >
> > >
> > _________________________________________________________________
> > _____
> > > ___________________________________________________
> > >
> > >     Ce message et ses pieces jointes peuvent contenir des informations
> > >     confidentielles ou privilegiees et ne doivent donc
> > >     pas etre diffuses, exploites ou copies sans autorisation. Si vous
> > >     avez recu ce message par erreur, veuillez le signaler
> > >     a l'expediteur et le detruire ainsi que les pieces jointes. Les
> > >     messages electroniques etant susceptibles d'alteration,
> > >     Orange decline toute responsabilite si ce message a ete altere,
> > >     deforme ou falsifie. Merci.
> > >
> > >     This message and its attachments may contain confidential or
> > >     privileged information that may be protected by law;
> > >     they should not be distributed, used or copied without
> authorisation.
> > >     If you have received this email in error, please notify the sender
> > >     and delete this message and its attachments.
> > >     As emails may be altered, Orange is not liable for messages that
> > >     have been modified, changed or falsified.
> > >     Thank you.
> > >
> > >     _______________________________________________
> > >     Teas mailing list
> > >     [e-mail] <mailto:[e-mail]>
> > >
> >
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/teas__;!!N
> > Et6yMaO-gk!VrEVB46KF20816iW_4FlbM4JGWGSLyFu-vk4PGWX_-
> > LiOR92MnzCJzyLzCREI4M$
> > >
> > > <https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/tea
> > > s__;!!NEt6yMaO-gk!VrEVB46KF20816iW_4FlbM4JGWGSLyFu-vk4PGWX_-
> > LiOR92MnzC
> > > JzyLzCREI4M$ >
> > >
> > > --
> > >
> > > <https://urldefense.com/v3/__http://www.verizon.com/__;!!NEt6yMaO-gk!V
> > > rEVB46KF20816iW_4FlbM4JGWGSLyFu-vk4PGWX_-LiOR92MnzCJzyLqOCoT3c$
> > >
> > >
> > > *Gyan Mishra*
> > >
> > > /Network Solutions A//rchitect /
> > >
> > > /Email [e-mail] <mailto:[e-mail]>//
> > > /
> > >
> > > /M 301 502-1347
> > >
> > > /
> > >
> > >
> >
> > _______________________________________________
> > Teas mailing list
> > [e-mail]
> >
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/teas__;!!N
> > Et6yMaO-gk!VrEVB46KF20816iW_4FlbM4JGWGSLyFu-vk4PGWX_-
> > LiOR92MnzCJzyLzCREI4M$
>


-- 

<http://www.verizon.com/>

*Gyan Mishra*

*Network Solutions A**rchitect *

*Email [e-mail] <[e-mail]>*



*M 301 502-1347*