Re: [Teas] What SLOs and SLEs should be in the Slicing Service Model?

["Joel M. Halpern" <[e-mail]>]

Sounds like we need to clarify the verbiage.  SLEs are, as I understand 
it, indeed non-measurables.  But they are not the legal translation of 
the SLOs.  They are expression of customer expectations that are not 
directly observable or measurable.  The example I have the easiest time 
understanding is that the customer expects the operator to encrypt the 
traffic across the service.

Yours,
Joel

On 3/25/2022 3:07 PM, Gyan Mishra wrote:
> Hi Med & Joel
> 
> I reread section 4.1 of the slice draft and to me it seems that SLO is 
> from provider POV measurable indicators and SLE is the is unmeasurable 
> expectations which is makes up the customer fulfillment of obligations 
> by the provider which would be like taking the tangible measurement 
> characteristics from the SLO and translation into legal obligation 
> fulfillment verbiage that goes into the service agreement.  So as the 
> SLE is not tangible but just a translation of SLO metrics into legal 
> verbiage my thoughts are that the framework as well as Yang model need 
> only focus on the SLO tangible metrics and leave the intangible for the 
> lawyers writing the customer agreement.
> 
> 
> 4.1 
> <https://datatracker.ietf.org/doc/html/draft-ietf-teas-ietf-network-slices-09#section-4.1>. 
> Objectives for IETF Network Slices
> 
>     An IETF Network Slice service is defined in terms of quantifiable
>     characteristics known as Service Level Objectives (SLOs) and
>     unquantifiable characteristics known as Service Level Expectations
>     (SLEs).  SLOs are expressed in terms Service Level Indicators (SLIs),
>     and together with the SLEs form the contractual agreement between
>     service customer and service provider known as a Service Level
>     Agreement (SLA).
> 
>     The terms are defined as follows:
> 
>     *  A Service Level Indicator (SLI) is a quantifiable measure of an
>        aspect of the performance of a network.  For example, it may be a
>        measure of throughput in bits per second, or it may be a measure
>        of latency in milliseconds.
> 
>     *  A Service Level Objective (SLO) is a target value or range for the
>        measurements returned by observation of an SLI.  For example, an
>        SLO may be expressed as "SLI <= target", or "lower bound <= SLI <=
>        upper bound".  A customer can determine whether the provider is
>        meeting the SLOs by performing measurements on the traffic.
> 
>     *  A Service Level Expectation (SLE) is an expression of an
>        unmeasurable service-related request that a customer of an IETF
>        Network Slice makes of the provider.  An SLE is distinct from an
>        SLO because the customer may have little or no way of determining
>        whether the SLE is being met, but they still contract with the
>        provider for a service that meets the expectation.
> 
> 
> 
> On Thu, Mar 24, 2022 at 10:36 AM <[e-mail] 
> <mailto:[e-mail]>> wrote:
> 
>     Re-,
> 
>     They shouldn't!
> 
>     This is why I'm suggesting to not have that tagging frozen in the
>     model. We can simply have a provision for a set of parameters. These
>     parameters will be included to characterize the service with a
>     service assurance component that will call out the identity of the
>     subset of parameters that will be used as committed ones. That
>     component will also include other data to ensure the same based used
>     for assessment, etc.
> 
>     Cheers,
>     Med
> 
>      > -----Message d'origine-----
>      > De : Joel M. Halpern <[e-mail]
>     <mailto:[e-mail]>>
>      > Envoyé : jeudi 24 mars 2022 15:29
>      > À : BOUCADAIR Mohamed INNOV/NET <[e-mail]
>     <mailto:[e-mail]>>
>      > Cc : 'TEAS WG' <[e-mail] <mailto:[e-mail]>>;
>     [e-mail] <mailto:[e-mail]>
>      > Objet : Re: [Teas] What SLOs and SLEs should be in the Slicing
>     Service
>      > Model?
>      >
>      > If the others are best effort, why would they be included in the SLO?
>      > (We do not assume that every IETF network slice service will
>     specify all
>      > possible parameters.)
>      >
>      > Yours,
>      > Joel
>      >
>      > On 3/24/2022 10:20 AM, [e-mail]
>     <mailto:[e-mail]> wrote:
>      > > Re-,
>      > >
>      > > Some services may be sensitive to delay for example but the
>     slice service
>      > request may include (for whatever reason) not only the delay, but
>     also
>      > other attributes that are currently listed as SLOs in the draft. The
>      > provider is requested to only commit on the delay and best effort
>     for the
>      > other attributes.
>      > >
>      > > Cheers,
>      > > Med
>      > >
>      > >> -----Message d'origine-----
>      > >> De : Joel M. Halpern <[e-mail]
>     <mailto:[e-mail]>> Envoyé : jeudi 24 mars
>      > >> 2022 15:07 À : BOUCADAIR Mohamed INNOV/NET
>      > >> <[e-mail]
>     <mailto:[e-mail]>> Cc : 'TEAS WG' <[e-mail]
>     <mailto:[e-mail]>>;
>      > >> [e-mail] <mailto:[e-mail]> Objet : Re:
>     [Teas] What SLOs and SLEs should be
>      > >> in the Slicing Service Model?
>      > >>
>      > >> Interesting.  If they are indeed not coupled, then you are clearly
>      > >> correct about representation.
>      > >>
>      > >> Can you give an example so I can understand when they would not be
>      > coupled.
>      > >> I had leapt to the (quite possibly incorrect) conclusion that
>     the two
>      > >> sets of properties went together.
>      > >>
>      > >> Thank you,
>      > >> Joel
>      > >>
>      > >> On 3/24/2022 10:03 AM, [e-mail]
>     <mailto:[e-mail]> wrote:
>      > >>> Hi Joel,
>      > >>>
>      > >>> It is.
>      > >>>
>      > >>> As mentioned below, this should be covered as part of service
>      > >> assurance/fulfillment/reporting parameters. Which parameters
>     to put
>      > >> there is deployment-specific. Not all parameters tagged as SLO
>     in the
>      > >> framework will end up as part of the
>     assurance/fulfillment/reporting.
>      > >>>
>      > >>> Cheers,
>      > >>> Med
>      > >>>
>      > >>>> -----Message d'origine-----
>      > >>>> De : Joel M. Halpern <[e-mail]
>     <mailto:[e-mail]>> Envoyé : jeudi 24 mars
>      > >>>> 2022 14:52 À : BOUCADAIR Mohamed INNOV/NET
>      > >>>> <[e-mail]
>     <mailto:[e-mail]>>; [e-mail]
>     <mailto:[e-mail]>; 'TEAS WG'
>      > >>>> <[e-mail] <mailto:[e-mail]>> Objet : Re: [Teas]
>     What SLOs and SLEs should be in
>      > >>>> the Slicing Service Model?
>      > >>>>
>      > >>>> Isn't it important to distinguish between "these are things I
>      > >>>> expect you to do, measure, and report" and "these are things I
>      > >>>> would like you to do even though they are not measurable or
>      > reportable"?
>      > >>>>
>      > >>>> Yours,
>      > >>>> Joel
>      > >>>>
>      > >>>> On 3/23/2022 10:08 AM, [e-mail]
>     <mailto:[e-mail]> wrote:
>      > >>>>> Hi all,
>      > >>>>>
>      > >>>>> This message actually triggers a companion comment I have
>     on the
>      > >>>>> SLO/SLE
>      > >>>> taxonomy. From the service modeling standpoint, I suggest
>     that we
>      > >>>> don't inherit that taxonomy for various reasons:
>      > >>>>>
>      > >>>>> * Things would be much simpler if we just focus on service
>      > >>>>> requirements
>      > >>>> without making an assumption how such requirement is
>     expressed and
>      > >>>> whether it is
>     quantified/quantitative/qualitative/measurable/etc.
>      > >>>> Whether/how a specific service requirement is covered by service
>      > >>>> assurance/fulfillment can be part of the slice service
>     definition
>      > >> itself.
>      > >>>>>
>      > >>>>> * What we may tag as an SLE today because of the technology
>      > >>>>> limitations,
>      > >>>> may not stay as such forever. It may be true that an
>     "expectation"
>      > >>>> may not be easily assessed using current techniques (and thus be
>      > >>>> tagged as SLE), but this does not prevent that innovative means
>      > >>>> would be defined in the future (which means that it is an
>     SLO, not
>      > >>>> an SLE
>      > >> anymore).
>      > >>>>>
>      > >>>>> * We are artificially adding extra complexity for the modelling
>      > >>>>> part as
>      > >>>> service requirement will need to be classified based as SLO
>     or SLEs.
>      > >>>>>
>      > >>>>> * I remember that Kiran agreed at least to not import that
>      > >>>>> taxonomy into
>      > >>>> the data model when we were discussing the call for adoption
>     of the
>      > >>>> slice
>      > >>>> definition:
>      > >>>>>
>      > >>>>> ==(the full message from Kiran can be found in the archives)===
>      > >>>>> "However, it should not imply that NBI models are required
>     to have
>      > >>>> SLE/SLO indicators and I totally agreed with your comments on
>      > >>>> draft-wd- teas-ietf-network-slice-nbi-yang-03"
>      > >>>>> ==
>      > >>>>>
>      > >>>>> Cheers,
>      > >>>>> Med
>      > >>>>>
>      > >>>>>> -----Message d'origine-----
>      > >>>>>> De : Teas <[e-mail]
>     <mailto:[e-mail]>> De la part de Adrian Farrel
>      > >>>>>> Envoyé
>      > >>>>>> : mercredi 23 mars 2022 14:47 À : 'TEAS WG' <[e-mail]
>     <mailto:[e-mail]>> Objet :
>      > >>>>>> [Teas] What SLOs and SLEs should be in the Slicing Service
>     Model?
>      > >>>>>>
>      > >>>>>> Hi,
>      > >>>>>>
>      > >>>>>> Sorry for my audio being a mess in TEAS today.
>      > >>>>>>
>      > >>>>>> I believe I heard the discussion between Kireeti and Reza
>      > >>>>>> correctly, and there was some follow-up in the chat.
>      > >>>>>>
>      > >>>>>> I agree that "Protection" is a realisation feature and so it
>      > >>>>>> qualifies as an SLE, if at all.
>      > >>>>>> But "Reliability" is clearly an SLO. Usually expressed as
>     maximum
>      > >>>>>> down- time per unit of time, or maximum lost traffic.
>      > >>>>>>
>      > >>>>>> There was one thing that I *think* I heard. This was Reza
>     saying
>      > >>>>>> that the service YANG model was only including the SLOs
>     and SLEs
>      > >>>>>> noted in the framework. Maybe I misheard "only" because I
>     think
>      > >>>>>> that might be a mistake.
>      > >>>>>> The YANG model should certainly be interested in what the
>      > >>>>>> framework says, but it is not a requirement that all SLOs and
>      > >>>>>> SLEs in the framework be in the model if the authors find that
>      > >>>>>> there is no interest in implementing them, and there should
>      > >>>>>> certainly be no limitation about including additional SLOs
>     or SLEs
>      > in the YANG model.
>      > >>>>>> Indeed, the SLOs and SLEs listed in the framework are
>     presented
>      > >>>>>> as
>      > >>>> examples.
>      > >>>>>>
>      > >>>>>> Cheers,
>      > >>>>>> Adrian
>      > >>>>>>
>      > >>>>>> _______________________________________________
>      > >>>>>> Teas mailing list
>      > >>>>>> [e-mail] <mailto:[e-mail]>
>      > >>>>>> https://www.ietf.org/mailman/listinfo/teas
>     <https://www.ietf.org/mailman/listinfo/teas>
>      > >>>>>
>      > >>>>>
>     __________________________________________________________________
>      > >>>>> __ __ ___________________________________________________
>      > >>>>>
>      > >>>>> Ce message et ses pieces jointes peuvent contenir des
>     informations
>      > >>>>> confidentielles ou privilegiees et ne doivent donc pas etre
>      > >>>>> diffuses, exploites ou copies sans autorisation. Si vous
>     avez recu
>      > >>>>> ce message par erreur, veuillez le signaler a l'expediteur
>     et le
>      > >>>>> detruire ainsi que
>      > >>>> les pieces jointes. Les messages electroniques etant
>     susceptibles
>      > >>>> d'alteration, Orange decline toute responsabilite si ce
>     message a
>      > >>>> ete altere, deforme ou falsifie. Merci.
>      > >>>>>
>      > >>>>> This message and its attachments may contain confidential or
>      > >>>>> privileged information that may be protected by law; they
>     should
>      > >>>>> not be
>      > >>>> distributed, used or copied without authorisation.
>      > >>>>> If you have received this email in error, please notify the
>     sender
>      > >>>>> and
>      > >>>> delete this message and its attachments.
>      > >>>>> As emails may be altered, Orange is not liable for messages
>     that
>      > >>>>> have
>      > >>>> been modified, changed or falsified.
>      > >>>>> Thank you.
>      > >>>>>
>      > >>>>> _______________________________________________
>      > >>>>> Teas mailing list
>      > >>>>> [e-mail] <mailto:[e-mail]>
>      > >>>>> https://www.ietf.org/mailman/listinfo/teas
>     <https://www.ietf.org/mailman/listinfo/teas>
>      > >>>
>      > >>>
>     ____________________________________________________________________
>      > >>> __ ___________________________________________________
>      > >>>
>      > >>> Ce message et ses pieces jointes peuvent contenir des
>     informations
>      > >>> confidentielles ou privilegiees et ne doivent donc pas etre
>      > >>> diffuses, exploites ou copies sans autorisation. Si vous avez
>     recu
>      > >>> ce message par erreur, veuillez le signaler a l'expediteur et le
>      > >>> detruire ainsi que
>      > >> les pieces jointes. Les messages electroniques etant susceptibles
>      > >> d'alteration, Orange decline toute responsabilite si ce
>     message a ete
>      > >> altere, deforme ou falsifie. Merci.
>      > >>>
>      > >>> This message and its attachments may contain confidential or
>      > >>> privileged information that may be protected by law; they
>     should not
>      > >>> be
>      > >> distributed, used or copied without authorisation.
>      > >>> If you have received this email in error, please notify the
>     sender
>      > >>> and
>      > >> delete this message and its attachments.
>      > >>> As emails may be altered, Orange is not liable for messages that
>      > >>> have
>      > >> been modified, changed or falsified.
>      > >>> Thank you.
>      > >>>
>      > >
>      > >
>     ______________________________________________________________________
>      > > ___________________________________________________
>      > >
>      > > Ce message et ses pieces jointes peuvent contenir des informations
>      > > confidentielles ou privilegiees et ne doivent donc pas etre
>     diffuses,
>      > > exploites ou copies sans autorisation. Si vous avez recu ce message
>      > > par erreur, veuillez le signaler a l'expediteur et le detruire
>     ainsi que
>      > les pieces jointes. Les messages electroniques etant susceptibles
>      > d'alteration, Orange decline toute responsabilite si ce message a ete
>      > altere, deforme ou falsifie. Merci.
>      > >
>      > > This message and its attachments may contain confidential or
>      > > privileged information that may be protected by law; they
>     should not be
>      > distributed, used or copied without authorisation.
>      > > If you have received this email in error, please notify the
>     sender and
>      > delete this message and its attachments.
>      > > As emails may be altered, Orange is not liable for messages
>     that have
>      > been modified, changed or falsified.
>      > > Thank you.
>      > >
> 
>     _________________________________________________________________________________________________________________________
> 
>     Ce message et ses pieces jointes peuvent contenir des informations
>     confidentielles ou privilegiees et ne doivent donc
>     pas etre diffuses, exploites ou copies sans autorisation. Si vous
>     avez recu ce message par erreur, veuillez le signaler
>     a l'expediteur et le detruire ainsi que les pieces jointes. Les
>     messages electroniques etant susceptibles d'alteration,
>     Orange decline toute responsabilite si ce message a ete altere,
>     deforme ou falsifie. Merci.
> 
>     This message and its attachments may contain confidential or
>     privileged information that may be protected by law;
>     they should not be distributed, used or copied without authorisation.
>     If you have received this email in error, please notify the sender
>     and delete this message and its attachments.
>     As emails may be altered, Orange is not liable for messages that
>     have been modified, changed or falsified.
>     Thank you.
> 
>     _______________________________________________
>     Teas mailing list
>     [e-mail] <mailto:[e-mail]>
>     https://www.ietf.org/mailman/listinfo/teas
>     <https://www.ietf.org/mailman/listinfo/teas>
> 
> -- 
> 
> <http://www.verizon.com/>
> 
> *Gyan Mishra*
> 
> /Network Solutions A//rchitect /
> 
> /Email [e-mail] <mailto:[e-mail]>//
> /
> 
> /M 301 502-1347
> 
> /
> 
>