Re: [Teas-ns-dt] definitions section 4 improvement discussion

[Greg Mirsky <[e-mail]>]

Hi Jie,
I would note that different calculatable performance metrics differently
reflect the presence of extreme measurement results. For example, a single
high value will have a larger impact on the mean than on the median. AFAIK,
for that reason providing percentile (95, 99, and 99.9) values might be
informative, operationally useful, and less dependent on the measurement
time period.

Regards,
Greg

On Thu, May 7, 2020 at 8:33 AM Dongjie (Jimmy) <[e-mail]> wrote:

> Hi Luis and Joel,
>
>
>
> I also agree that in addition to using parameters such as packet loss,
> bandwidth, latency, delay variation to describe the service requirement,
> for some services there is also requirement to have dedicated network
> resources allocated, so as to avoid or reduce unpredicted interference from
> other services or customers in the network. This can be seen as a generic
> requirement, and is not bound to any specific implementation.
>
>
>
> Note that we may claim that the requirement of particular bandwidth,
> latency, delay variation can be met in a statistic multiplexing network
> with no resource reservation, while actually they are met at average over a
> relatively long time scale. This may not be good enough for services which
> were carried using a dedicated network (no interference from other
> customers) and plan to migrate to shared infrastructure, or which require
> those typical parameters be met at a shorter time scale.
>
>
>
> Best regards,
>
> Jie
>
>
>
> *From:* Teas-ns-dt [mailto:[e-mail]] *On Behalf Of *Luis
> M. Contreras
> *Sent:* Wednesday, May 6, 2020 7:33 PM
> *To:* Joel M. Halpern <[e-mail]>
> *Cc:* LUIS MIGUEL CONTRERAS MURILLO <
> [e-mail]>gt;; Kiran Makhijani <
> [e-mail]>gt;; Eric Gray <eric.gray=[e-mail]>rg>;
> [e-mail]
> *Subject:* Re: [Teas-ns-dt] definitions section 4 improvement discussion
>
>
>
> Hi Joel,
>
>
>
> yes, the customer is required to always express the service in terms of
> SLO parameters as you point out, with or without the extra request of being
> provided using resources in an isolated / dedicated / reserved manner.
>
>
>
> SLOs should be always there.
>
>
>
> Best regards
>
>
>
> Luis
>
>
>
>
>
> El mar., 5 may. 2020 a las 23:43, Joel M. Halpern (<[e-mail]>)
> escribió:
>
> Luis, if I am reading your note properly, you refer to a customer asking
> for a TDM behavior.
>
> As far as I know, we are talking about a packet service.
> Also presumably, what the custoemr actually wants can be described in
> terms of the SLO parameters we typically use (loss, delay, delay
> variation, bandwidth).
> If so, it is actually better for the customer and the providing operator
> if the request is made in terms of those parameters, rather than an
> under-specified term like TDM packet service.
>
> Then the operator is free to use any technique under the cover to meet
> the requirement.  What combination of dedicated resources,
> under-utilization of shared resources, prioritization, and other
> techniques, are used by the operator is up to him.  As long as the SLOs
> are met.
>
> Yours,
> Joel
>
> On 5/5/2020 5:19 PM, Luis M. Contreras wrote:
> > Hi Eric,
> >
> > thanks for removing the Notice, included by default on my emails
> > (internal policies), I will use my gmail address to avoid that.
> Apologies.
> >
> > I think I'm not so far from your view, except in some specific points. I
> > quote here some parts of your email, apologies in advance if I cut too
> > many words altering the real message
> >
> >  >> we cannot (and should not try to) "focus" on "programmable
> > constructs" or "concentrate" on "programmable capabilities."
> >
> > I think we can assume programmable solutions since we are dealing with a
> > controller. If programmable is not the proper word (*) , maybe we can
> > say controllable nodes/equipment via a centralized element, such in the
> > case of ACTN (RFC8453) or ABNO (RFC7491).
> > (*) I'm not native English speaker so probably I cannot express
> > precisely the concept, but I hope it can be properly understood.
> >
> >  >> TSC can them implement the requested transport slice service in any
> > way that is determined to be best by underlay capabilities and provider
> > policies, that is otherwise completely independent of the fact that the
> > transport slice service will look like a (packetized) TDM service.
> >
> > Without entering on technology options/details I only foresee two ways
> > of ensuring a behavior like a TDM service for the customer requesting it
> > when mixed in the same network with other non-TDM like service requests:
> > (1) overprovisioning the network in such a way that the TDM-like
> > customers do not experience any issue because a permanent excess and
> > availability of resources (bandwidth, queues, express links for reducing
> > latency, etc)
> > (2) reservation of resources being dedicated only for those TDM-like
> > customers, coexisting gracefully with the other kind of customers not
> > needing the TDM-like behavior.
> > I think option (2) is the best option.
> >
> >  >> that some customers are reluctant to actually work out exactly what
> > they really need and this is the underlying argument for why they want
> > the TSC NBI to “accept” isolation as an objective
> >
> > Even in the case of requesting isolation, the customer will be required
> > always to express her/his needs (or SLOs). Isolation is an additional
> > atribute. In other words, requesting isolation withour detailing SLOs
> > such as throughput, latency, etc is nor practical
> >
> > Best regards
> >
> > Luis
> >
> >
> > El mar., 5 may. 2020 a las 20:55, Eric Gray
> > (<eric.gray=[e-mail]
> > <mailto:[e-mail]>>) escribió:
> >
> >     Luis,____
> >
> >     __ __
> >
> >     I removed multiple copies of the inappropriate "Notice" included in
> >     your earlier E-Mail, below.____
> >
> >     __ __
> >
> >     We had multiple discussions about the fact that we should not say
> >     anything about what goes on inside (or under) a Transport Slice
> >     Controller, or through the transport slice controller.____
> >
> >     __ __
> >
> >     If you think back, you should recall that we are only modeling
> >     anything beneath the TSC NBI logically and we are not asserting that
> >     any of the logical constructs associated with this logical model can
> >     be presumed to exist.____
> >
> >     __ __
> >
> >     For specific examples, there may be no TSC SBI and there may be no
> >     actual network controller.____
> >
> >     __ __
> >
> >      From our perspective, it is useful to think of the TSC as a black
> >     box, controlled via the NBI, and having physical connectivity to
> >     service users only as will have been determined in deployment.____
> >
> >     __ __
> >
> >     Consequently, we cannot (and should not try to) "focus" on
> >     "programmable constructs" or "concentrate" on "programmable
> >     capabilities."____
> >
> >     __ __
> >
> >     That will be the task of a TSC implementation.  We are not defining
> >     TSC implementation, at this time (or quite possibly ever). ____
> >
> >     __ __
> >
> >     If you want to create a TSC implementation, here is not the place to
> >     do so.____
> >
> >     __ __
> >
> >     This  is, in my opinion, exactly what folks are missing here: i.e. -
> >     since we are trying to define how we can instruct a TSC to provide a
> >     service, with no more information about how the TSC is allowed to do
> >     that than that minimum unavoidable information, then we cannot
> >     describe requested services in terms that cannot be seen directly by
> >     the user.____
> >
> >     __ __
> >
> >     The minute we start to talk about what a TSC must do in order to
> >     meet a set of service criteria we might expect assuming some
> >     specific way of providing the service requested - without spelling
> >     those service criteria (parameters or objectives) out explicitly -
> >     we are talking about "how the TSC provides" the service, rather than
> >     "what the service is."____
> >
> >     __ __
> >
> >     So if a customer wants the equivalent of a TDM service, with a
> >     bandwidth */X/* – for example, they need to translate this into a
> >     list of objectives that they would expect from such a TDM service
> >     (e.g. =latency, delay variation, etc.).____
> >
> >     __ __
> >
> >     Once a customer actually goes through this process, they may (and
> >     likely will) realize that there are some characteristics of a TDM
> >     service that they may not need for their particular application – at
> >     least not to the extent that an actual TDM service could be able to
> >     provide it.____
> >
> >     __ __
> >
> >     If that is the case, then they may either skip those objectives, or
> >     accept a lower level of assurance (in either case, capitalizing on
> >     an opportunity to reduce service costs).____
> >
> >     __ __
> >
> >     Even if they decide that they need all of the expected
> >     characteristics of a corresponding TDM transport slice service, they
> >     can request such a service (in packetized form) from a TSC and the
> >     TSC can them implement the requested transport slice service in any
> >     way that is determined to be best by underlay capabilities and
> >     provider policies, that is otherwise completely independent of the
> >     fact that the transport slice service will look like a (packetized)
> >     TDM service.____
> >
> >     __ __
> >
> >     This allows for efficiencies and cost savings to the transport slice
> >     service provider – especially if the transport slice service
> >     provider is also providing other services, with different objectives
> >     and expectations.____
> >
> >     __ __
> >
> >     And it makes the process of trying to define an abstract interface
> >     between the transport slice user and the transport slice provider a
> >     good deal simpler.____
> >
> >     __ __
> >
> >     So, win-win.____
> >
> >     __ __
> >
> >     What I am afraid of is that some customers are reluctant to actually
> >     work out exactly what they really need and this is the underlying
> >     argument for why they want the TSC NBI to “accept” isolation as an
> >     objective, and perform the necessary translation for them.____
> >
> >     __ __
> >
> >     But it must be abundantly clear by now that – if the customer
> >     doesn’t tell us exactly what they mean by isolation – then any
> >     translation we define will likely fail to match their (undefined)
> >     expectations.____
> >
> >     __ __
> >
> >     And this is largely a “customer-management” issue – hence something
> >     that does not belong in the scope of our work here.____
> >
> >     __ __
> >
> >     --____
> >
> >     Eric____
> >
> >     __ __
> >
> >     -----Original Message-----
> >     From: Teas-ns-dt <[e-mail]
> >     <mailto:[e-mail]>> On Behalf Of LUIS MIGUEL
> >     CONTRERAS MURILLO
> >     Sent: Tuesday, May 5, 2020 5:08 AM
> >     To: Joel Halpern Direct <[e-mail]
> >     <mailto:[e-mail] <jmh.direct@joelhalpern..com>>>;
> Kiran Makhijani
> >     <[e-mail] <mailto:[e-mail]>>;
> >     [e-mail] <mailto:[e-mail]>
> >     Subject: Re: [Teas-ns-dt] definitions section 4 improvement
> discussion
> >
> >     __ __
> >
> >     Hi Joel,____
> >
> >     __ __
> >
> >     Some comments to your points:____
> >
> >     __ __
> >
> >     <jmh>____
> >
> >     I had assumed that things with high reliability needs would specify
> >     that they need 99.99% (or 5-9s, or even 6-9s) for the relevant SLO
> >     parameters.  And pay the price the operator needs to get to have
> >     that be offerable.  I would not expect the customer to express this
> >     kind of high reliability requirement as an isolation requirement, as
> >     isolation does not actually equate to reliability.  (I can have an
> >     isolated fiber.  If it is cut, I am out of luck.  And I got no
> >     benefit from having the isolated fiber.) </jmh>____
> >
> >     __ __
> >
> >     Isolation implies dedication of a number of resources. With those
> >     dedicated resources two situations could happen: (1) everything
> >     works fine, so no impacts on SLOs; (2) something works wrong, thus
> >     the operator can allocate different but equivalent resources in such
> >     a way that SLOs are not impacted.____
> >
> >     This works for sure at certain levels. If we are talking of
> >     dedicated infrastructure (as in your example of the fiber) this
> >     could not be possible. However there are connectivity constructs
> >     that make it possible (the referred lambda, for instance, which
> >     could be provided through a different path).____
> >
> >     I would say that the explicit implication of isolation is dedication
> >     of resources, while the implicit one is some service guarantees
> >     (apart from others such as maybe security, predictability,
> >     robustness, etc).____
> >
> >     __ __
> >
> >     <jmh>____
> >
> >     At the same time, if we really wanted to get into isolation, we
> >     would have to get into the difference among dedicate conduit,
> >     dedicate fiber,____
> >
> >     dedicate lambda, dedicate time slice, ...).   None of those are
> >     degrees.____
> >
> >        They are different ways of delivering a service.  With different
> >     effects.  Note that this is quite separate from a requirement that
> >     two different access points from a customer must have no failure
> >     point in common.  That is a complex thing to express in an SLO, and
> >     hard to____
> >
> >     monitor.   But it is Very different from what has been described
> as____
> >
> >     isolation.____
> >
> >     </jmh>____
> >
> >     __ __
> >
> >     What we could expect from a transport slice controller (TSC) is to
> >     work on programmable assets (via network controllers probably). Then
> >     I guess the focus should be on the programmable constructs providing
> >     isolation, not in the physical infrastructure which will not be
> >     addressable by the TSC. So I would concentrate on programmable
> >     capabilities, leaving the rest apart.____
> >
> >     Maybe, as discussed, isolation cannot be expressed through a SLO,
> >     but it will be an attribute or characteristic of the service.____
> >
> >     Regarding the description in the draft, yes, should be changed to
> >     avoid confusion.____
> >
> >     __ __
> >
> >     <jmh>____
> >
> >     The biggest risk is that if we geet too specific about how these
> >     things will be measured, we can end up with volumes of text..  I
> >     agree with the description Luis used.  But if we start trying to
> >     define (as contracts have to) the exact start and end of measurement
> >     intervals, the exact clock skew allowed by the measurement tools,
> >     and the exact technique to be used to measure these things, we will
> >     create a never-ending nightmare for ourselves.____
> >
> >     </jmh>____
> >
> >     Probably it would be enough referring that some time window could be
> >     considered, describing in rough way, not exact details. Or just
> >     referring some of the existing RFCs dealing with the topic as
> >     examples/references for defining SLOs.____
> >
> >     __ __
> >
> >     Thanks____
> >
> >     __ __
> >
> >     Best regards____
> >
> >     __ __
> >
> >     Luis____
> >
> >     __ __
> >
> >     -----Mensaje original-----____
> >
> >     De: Joel Halpern Direct <[e-mail]
> >     <mailto:[e-mail]>> Enviado el: lunes, 4 de mayo
> >     de 2020 22:06____
> >
> >     Para: LUIS MIGUEL CONTRERAS MURILLO
> >     <luismiguel.contrerasmurillo@telefonica..com
> <luismiguel.contrerasmurillo@telefonica..com%0b>>     <mailto:
> luismiguel.contrerasmurillo@telefonica..
> <luismiguel.contrerasmurillo@telefonica.>.com>>.com>>; Kiran
> >     Makhijani <kiranm@futurewei..com <mailto:[e-mail]>>;
> >     [e-mail] <mailto:[e-mail]>____
> >
> >     Asunto: Re: definitions section 4 improvement discussion____
> >
> >     __ __
> >
> >     Trimmed to one part I want to comment on.  Reply bracketted by
> >     <jmh></jmh> for quoting clarity.____
> >
> >     __ __
> >
> >     Yours,____
> >
> >     Joel____
> >
> >     __ __
> >
> >     On 5/4/2020 3:56 PM, LUIS MIGUEL CONTRERAS MURILLO wrote:____
> >
> >      > Hi Kiran,____
> >
> >      >__ __
> >
> >      > Some comments from my side in-line (adding also Joel who is not
> >     in the ____
> >
> >      > TEAS NS DT list - I think- but was part of the discussion
> today)____
> >
> >      >__ __
> >
> >      > Best regards____
> >
> >      >__ __
> >
> >      > Luis____
> >
> >      >__ __
> >
> >      > *De:*Teas-ns-dt <[e-mail]
> >     <mailto:[e-mail]>> *En nombre de *Kiran ____
> >
> >      > Makhijani *Enviado el:* lunes, 4 de mayo de 2020 18:07____
> >
> >      > *Para:* [e-mail] <mailto:[e-mail]>____
> >
> >      > *Asunto:* [Teas-ns-dt] definitions section 4 improvement
> >     discussion____
> >
> >     ....____
> >
> >      >  2. Dealing with failures:____
> >
> >      >__ __
> >
> >      > I am little worried about how we are mixing up the avoidable, ____
> >
> >      > recoverable and non-recoverable failures. I did not agree that
> ____
> >
> >      > fan-failure faults are same as congestion. In case of congestion
> >     - it ____
> >
> >      > must be avoided by minimizing or penalizing other less-critical
> ____
> >
> >      > traffic - because a transport slice asked for this. In case of
> ____
> >
> >      > recoverable HA or failover should kick in. finally
> >     non-recoverable are ____
> >
> >      > multiple faults:- both active and failover links saw problems and
> >     it ____
> >
> >      > is impossible to send any traffic. In transport slices, we can
> >     specify ____
> >
> >      > enough to prevent avoidable and recoverable failures. For
> >     example, ____
> >
> >      > consider a critical-service transport slice and what must be done
> >     to ____
> >
> >      > provide disruption free and always-on, and reliable service?____
> >
> >      >__ __
> >
> >      > */[Luis>>] I also see them as different kind of events. In order
> >     to ____
> >
> >      > deal with infrastructure failures, operators typically design the
> >     ____
> >
> >      > network architecture to avoid the kind of problems as the example
> >     of ____
> >
> >      > the fan failure. IN case of happening something like that (which
> ____
> >
> >      > implies that the node goes down), an alternative path in the
> >     network ____
> >
> >      > will take care of the traffic, avoiding impact on customers.
> >     Networks ____
> >
> >      > can be designed to deal with single or multiple failures. More
> ____
> >
> >      > protection more cost./*____
> >
> >      >__ __
> >
> >      > */However a network design like before could have events of
> >     congestion ____
> >
> >      > (flash crowd events, exceptional situations -e.g. COVID-19-,
> >     failures ____
> >
> >      > in parts of the network, etc). In this situations isolation, ____
> >
> >      > understood as the way of dedicating resources to customers, could
> >     ____
> >
> >      > guarantee that the resources allocated to a given customer are
> ____
> >
> >      > respected and not used for others. Examples of situations like
> >     thos ____
> >
> >      > could be the same lambda example as before, some calendar slots
> >     in ____
> >
> >      > FlexEthernet, some queues allocated for a specific customer (at
> >     least in ingress), etc.____
> >
> >      > Statistical multiplexing mechanisms cannot guarantee that, is
> >     clear, ____
> >
> >      > bit there are other mechanisms as the ones mentioned before,
> >     which are ____
> >
> >      > also part of the overall transport network./*____
> >
> >      >__ __
> >
> >      > */This does not mean that the cost of such dedication of
> >     resources ____
> >
> >      > will be equivalent to non-dedicated resources, for sure not. But
> >     there ____
> >
> >      > could be situation where such kind of guarantees are needed:
> >     emergency ____
> >
> >      > services, banks, stock markets, etc. Such kind of specialized
> >     services ____
> >
> >      > will be the ones demanding such kind of special solutions with
> >     the ____
> >
> >      > expectation of being more economic than building and running
> >     their own ____
> >
> >      > network (as happened in the computing world by the way)./*____
> >
> >      >__ __
> >
> >      > */So maybe this will not be the common norm, but there are a
> >     number of ____
> >
> >      > use cases that will have such needs, and because of that we need
> >     to ____
> >
> >      > define a solution being comprehensive, not partial./*____
> >
> >     __ __
> >
> >     I had assumed that things with high reliability needs would specify
> >     that they need 99.99% (or 5-9s, or even 6-9s) for the relevant SLO
> >     parameters.  And pay the price the operator needs to get to have
> >     that be offerable.  I would not expect the customer to express this
> >     kind of high reliability requirement as an isolation requirement, as
> >     isolation does not actually equate to reliability.  (I can have an
> >     isolated fiber.  If it is cut, I am out of luck.  And I got no
> >     benefit from having the isolated fiber.)____
> >
> >     __ __
> >
> >     At the same time, if we really wanted to get into isolation, we
> >     would have to get into the difference among dedicate conduit,
> >     dedicate fiber,____
> >
> >     dedicate lambda, dedicate time slice, ...).   None of those are
> >     degrees.____
> >
> >        They are different ways of delivering a service.  With different
> >     effects.  Note that this is quite separate from a requirement that
> >     two different access points from a customer must have no failure
> >     point in common.  That is a complex thing to express in an SLO, and
> >     hard to____
> >
> >     monitor.   But it is Very different from what has been described
> as____
> >
> >     isolation.____
> >
> >     __ __
> >
> >      >__ __
> >
> >      >  3. Monitoring in the context of measurable SLOs:____
> >
> >      >__ __
> >
> >      > Jie raised an interesting comment on directly measurable
> >     attributes.____
> >
> >      > Many SLOs will require different means of monitoring.  We have
> >     not ____
> >
> >      > discussed this as a characteristic of a transport slice. Should
> ____
> >
> >      > customer specify any monitoring as part of the transport
> slice?____
> >
> >      > Perhaps not, but then there is a mention of "monitoring
> >     thresholds" in ____
> >
> >      > framework document in 3.3. We should have corresponding text in
> >     the definition.____
> >
> >      >__ __
> >
> >      > */[Luis>>] In the examples I have mentioned in the point number
> >     1, ____
> >
> >      > apart from defining the SLO and the threshold, also the way of
> ____
> >
> >      > measuring it and the time window are specified. So, I think we
> >     can go ____
> >
> >      > in that direction and I think there are a number of RFCs we could
> >     ____
> >
> >      > leverage on in this respect./*____
> >
> >     __ __
> >
> >     The biggest risk is that if we geet too specific about how these
> >     things will be measured, we can end up with volumes of text..  I
> >     agree with the description Luis used.  But if we start trying to
> >     define (as contracts have to) the exact start and end of measurement
> >     intervals, the exact clock skew allowed by the measurement tools,
> >     and the exact technique to be used to measure these things, we will
> >     create a never-ending nightmare for ourselves.____
> >
> >     __ __
> >
> >      >__ __
> >
> >      > Regards____
> >
> >      >__ __
> >
> >      > Kiran____
> >
> >      >__ __
> >
> >      >__ __
> >
> >      >
> >
>  ----------------------------------------------------------------------____
> >
> >     __ __
> >
> >     --- [SNIP] ---____
> >
> >     __ __
> >
> >     ____________________________________
> >
> >     __ __
> >
> >     --- [SNIP] ---____
> >
> >     __ __
> >
> >     --____
> >
> >     Teas-ns-dt mailing list____
> >
> >     [e-mail] <mailto:[e-mail]>____
> >
> >     https://www.ietf.org/mailman/listinfo/teas-ns-dt____
> >
> >     --
> >     Teas-ns-dt mailing list
> >     [e-mail] <mailto:[e-mail]>
> >     https://www.ietf.org/mailman/listinfo/teas-ns-dt
> >
> >
> >
> > --
> > ___________________________________________
> > Luis M. Contreras
> > [e-mail] <mailto:[e-mail]>
> > [e-mail]
> > <mailto:[e-mail]>
> > Global CTIO unit / Telefonica
>
>
>
>
> --
>
> ___________________________________________
>
> Luis M. Contreras
>
> [e-mail]
>
> [e-mail]
>
> Global CTIO unit / Telefonica
> --
> Teas-ns-dt mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/teas-ns-dt
>