Re: [alto] ALTO Extension: Defining a Cost Metrics document?

["Y. Richard Yang" <[e-mail]>]

On Oct 29, 2013 6:01 AM, "Qin Wu" <[e-mail]> wrote:
>
> Hi, Richard:
>
>
>
> From: [e-mail] [mailto:[e-mail]] On Behalf Of
Y. Richard Yang
> Sent: Monday, October 28, 2013 4:47 AM
> To: Qin Wu
> Cc: [e-mail]; IETF ALTO
> Subject: RE: [alto] ALTO Extension: Defining a Cost Metrics document?
>
>
>
> Hi Qin,
>
> On Oct 25, 2013 11:14 PM, "Qin Wu" <[e-mail]> wrote:
> >
> > Hi,Richard:
> >
> > Thank for bring these examples.
> >
> > I see these examples from two perspectives:
> >
> > 1.       What cost metrics we should define?
> >
> > 2.       When, where and how to calculate these metrics (i.e.,
methodology ).
> >
> > I think we can not define these cost metrics from scratch, we can not
define all the metrics we think useful. Instead, we should define them
mostly based on standardized work,
> >
> > AT&T have a good practice for measuring these metrics. But I believe
AT&T also doing a good job in standardizing these metrics and associated
methodology,e.g.,
> >
> > in IETF IPPM WG.
> >
> > Therefore I think we should try to reuse the existing methodology
associated metrics defined somewhere.
>
> It may not be necessary to limit ALTO to metrics only defined by other
WGs. In other words, limiting ALTO to the representation layer. But as a
first step, this can be fine. I am not very familiar with IPPM. Here is a
question. How does IPPM handle parameterized metrics, e.g., 15-min vs 5-min
measurement interval of latency? I see two problems:
>
> - How does a network expose the parameters available (e.g., a network
says I can support 5 or 15 min intervals)? ALTO IRD allows exposures of
such capabilities.
>
> [Qin]: IPPM defines methodology for each metric. In the methodology, it
include parameters like source address, dst address, time to measure this
metric.
>
> However IPPM don’t give default value for interval. IPPM didn’t define
how to report those metrics.  Routing protocol can be extended to report
those metrics. However Routing protocol didn’t report measurement interval
directly in the routing message. Instead, such value of interval can be
pre-configured.

So we have a problem here? Hence, we need to include more fields in
CostType: CostMetric, CostMode, CostParams? Then IRD needs to expose the
parameters?

> -How does a client specify the parameter chosen or desired?
>
> [Qin]: Is a client referred to “ALTO server” that gathers metrics from
Routing protocol, or ALTO client that request information from ALTO server?
>
> If it is the former, I think ALTO server has its own local policy to
decide how to gather those metrics and in which frequency to gather those
metrics.

I meant the client. But thank you for pointing out that the ALTO Server can
be the client of the Routing system. So we go back to manual configuration?

> If it is the latter, I think ALTO client only care about which endpoint
he can connect and which path he can traverse. He does not need to care
about
>
> Whether this metric is latest measured or measured one hour ago. He can
just assume the metric he get is the latest update.

Not sure about this. The latest assumption assumes real-time operation.
Also, the measurement interval can have large impact on interpreting the
results. For example, jitter is essentially variance and can go through
substantial smoothing (law of large numbers) if the interval is long.
Hence, the client can benefit knowing it.
>
> Does IPPM have the mechanisms that ALTO can use?
>
> [Qin]: IPPM does define Delay Metric, Packet loss metric, Jitter Metric,
bandwidth metrics, see RFC5136, RFC2681, RFC2679,RFC3393,RFC2680.
>
> We can consider to use them as a reference when we follow RFC6390
performance template.

Good to know.

Thanks!

Richard
>
>
>
>
> >
> >
> >
> > Also I believe we are not intending to design ALTO as the whole
measurement system or measurement platform.
> >
> > What ALTO server should do is to gather/aggregate/abstract/filter
useful metrics in a standard way and  provide them
> >
> > to the alto client or use them as filtering to choose appropriate
endpoint to connect.
>
> Agree. The measurement infrastructure might collect data at 1 min
interval, and ALTO exposes only hourly data to certain clients? It is
policy controlled.
>
> Richard
> >
> >
> >
> > Regards!
> >
> > -Qin
> >
> > From: [e-mail] [mailto:[e-mail]] On Behalf Of
Y. Richard Yang
> > Sent: Monday, October 21, 2013 9:25 AM
> > To: Qin Wu
> > Cc: IETF ALTO; [e-mail]
> > Subject: Re: [alto] ALTO Extension: Defining a Cost Metrics document?
> >
> >
> >
> > Hi all,
> >
> >
> >
> > Just a thought, some networks already provide public performance
metrics, and ALTO should be able to provide such info in a standard way and
cover these metrics, if we agree:
> >
> >
> >
> > -AT&T
> >
> >  - http://ipnetwork.bgtmo.ip.att.net/pws/averages.html
> >
> >  - http://ipnetwork.bgtmo.ip.att.net/pws/network_delay.html
> >
> >    shows latency, loss, jitter, reliability, modem success rate
> >
> >
> >
> >  - In particular, the link provides a methodology page (
http://ipnetwork.bgtmo.ip.att.net/pws/glossary.html), which points to a
major challenge in defining the metrics: metrics have parameters (e.g., the
AT&T link specifies 15-min interval for latency), and I assume that ALTO
cannot work with a single interval, but then how do we handle parameters?
> >
> >
> >
> > - CenturyLink (formerly Qwest):
> >
> >   - https://kai04.centurylink.com/PtapRpts/Public/BackboneReport.aspx
> >
> >     shows jitter, latency, pkt delivery rate, availability
> >
> >
> >
> > ...
> >
> >
> >
> > And we could think that ALTO could be extended to be used as a standard
way to check on the service outage of an endpoint (
http://customer.comcast.com/help-and-support/cable-tv/outages-in-your-area/),
which may imply performance metrics as well...
> >
> >
> >
> > Thanks.
> >
> >
> >
> > Richard
> >
> >
> >
> > On Mon, Oct 14, 2013 at 2:30 AM, Qin Wu <[e-mail]> wrote:
> >
> > From: [e-mail] [mailto:[e-mail]] On Behalf Of
Y. Richard Yang
> > Sent: Monday, October 14, 2013 7:20 AM
> > To: IETF ALTO
> > Cc: [e-mail]; Qin Wu
> > Subject: [alto] ALTO Extension: Defining a Cost Metrics document?
> >
> >
> >
> > Dear all,
> >
> >
> >
> > I am reading up on the documents that define cost metrics.
> >
> >
> >
> > The motivation is that the base ALTO protocol (
http://www.ietf.org/id/draft-ietf-alto-protocol-20.txt) has defined only
one Cost Metric: 'routingcost':
> >
> >
> >
> > - Defined the semantics at Sec. 6.1.1.1 of , and then listed it at
Table 3.
> >
> >
> >
> > - Used "hopcount" in examples of Sec. 9.2.3 and 9.2.4, but the
semantics of not formally defined.
> >
> >
> >
> > Given the aforementioned state of the base protocol, I see good value
in that the WG produces a WG document that defines a relatively complete
set of Cost Metrics.
> >
> >
> >
> > I particular, I read the following:
> >
> >
> >
> > - http://tools.ietf.org/html/draft-lee-alto-app-net-info-exchange-02
> >
> >   (Sec. 3.4 introduced three metrics: hopcount, latency, pktcost, and
cost)
> >
> >
> >
> > - http://tools.ietf.org/html/draft-wu-alto-json-te-01
> >
> >   Defined a set of metrics: in Sec. 4. This work, as stated in the
document, is motivated by
> >
> >
> >
> > - http://www.ietf.org/id/draft-ietf-ospf-te-metric-extensions-04.txt
> >
> >
> >
> > During the review of ALTO base protocol, we are suggested to document
performance metrics (cost metrics) per the guideline of
> >
> > - RFC 6390 Guidelines for Considering New Performance Metric
Development. A. Clark, B. Claise. October 2011. (Format: TXT=49930 bytes)
(Also BCP0170) (Status: BEST CURRENT PRACTICE)
> >
> >
> >
> > Here a first question, I have, is whether the authors will produce a
"simple" document, at the upcoming IETF, whose only purpose is to:
> >
> >
> >
> >   define a set of cost metrics, including the nameing, the semantics,
... following the guideline per RFC 6390, that can benefit the base
protocol.
> >
> >
> >
> > [Qin] This is exactly what I we are doing in draft-wu-alto-json-te. We
are checking if we can give a complete list of cost metrics that are built
based on
> >
> > draft-ietf-idr-ls-distribution-03,RFC5305,
draft-wu-idr-te-pm-bgp,draft-ietf-ospf-te-metric-extensions-04,
draft-ietf-isis-te-metric-extensions-01.
> >
> > We will further generalize them to firstly have some base metrics that
can applied either to the whole path or any link in the path and then have
> >
> > Derived metrics that are link specific.
> >
> >
> >
> > The update (v-02) will come in a few days.
> >
> >
> >
> >
> >
> > I feel that such a document is focused, and has good value by itself.
> >
> >
> >
> > The implications of the introducing multiple cost metrics can be
explored in another document, which I will send in another email shortly.
> >
> >
> >
> > Thanks.
> >
> >
> >
> > Richard
> >
> >
> >
> >