Re: [lmap] Which performance metrics for LMAP?

<philip.eardley@bt.com> Mon, 11 March 2013 18:03 UTC

Return-Path: <philip.eardley@bt.com>
X-Original-To: lmap@ietfa.amsl.com
Delivered-To: lmap@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 48DA511E81B7 for <lmap@ietfa.amsl.com>; Mon, 11 Mar 2013 11:03:54 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -103.068
X-Spam-Level:
X-Spam-Status: No, score=-103.068 tagged_above=-999 required=5 tests=[AWL=-0.070, BAYES_00=-2.599, HTML_MESSAGE=0.001, J_CHICKENPOX_26=0.6, RCVD_IN_DNSWL_LOW=-1, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HxJtIfC-PItp for <lmap@ietfa.amsl.com>; Mon, 11 Mar 2013 11:03:53 -0700 (PDT)
Received: from smtpe1.intersmtp.com (smtp63.intersmtp.com [62.239.224.236]) by ietfa.amsl.com (Postfix) with ESMTP id 95C1D11E819D for <lmap@ietf.org>; Mon, 11 Mar 2013 11:03:52 -0700 (PDT)
Received: from EVMHT69-UKRD.domain1.systemhost.net (10.36.3.129) by RDW083A007ED63.smtp-e3.hygiene.service (10.187.98.12) with Microsoft SMTP Server (TLS) id 8.3.279.1; Mon, 11 Mar 2013 18:03:51 +0000
Received: from EMV65-UKRD.domain1.systemhost.net ([169.254.1.43]) by EVMHT69-UKRD.domain1.systemhost.net ([10.36.3.129]) with mapi; Mon, 11 Mar 2013 18:03:51 +0000
From: philip.eardley@bt.com
To: Michael.K.Bugenhagen@centurylink.com, bclaise@cisco.com
Date: Mon, 11 Mar 2013 18:03:51 +0000
Thread-Topic: [lmap] Which performance metrics for LMAP?
Thread-Index: AQHOGybaVh0e7cNwrUuMBtZyDyvaL5iazHk4gAYACVs=
Message-ID: <9510D26531EF184D9017DF24659BB87F343774C520@EMV65-UKRD.domain1.systemhost.net>
References: <513879B5.8030609@cisco.com>, <FB3C2704-00A1-4584-9717-DB464555AEAF@centurylink.com>
In-Reply-To: <FB3C2704-00A1-4584-9717-DB464555AEAF@centurylink.com>
Accept-Language: en-US, en-GB
Content-Language: en-GB
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US, en-GB
Content-Type: multipart/alternative; boundary="_000_9510D26531EF184D9017DF24659BB87F343774C520EMV65UKRDdoma_"
MIME-Version: 1.0
Cc: lmap@ietf.org
Subject: Re: [lmap] Which performance metrics for LMAP?
X-BeenThere: lmap@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Large Scale Measurement of Access network Performance <lmap.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/lmap>, <mailto:lmap-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/lmap>
List-Post: <mailto:lmap@ietf.org>
List-Help: <mailto:lmap-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/lmap>, <mailto:lmap-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 11 Mar 2013 18:03:54 -0000

agree with this,

also, i think the work of lmap is on the Controller-MA and MA-Collector interface (MA = MEasurement Agent)
for instance, Controller tells MA "Run test X to MA-2 every hour and report results to Collector-5 once per day"
"test X" is defined in a registry. IPPM will hopefully be re-chartered to define a registry of IPPM tests.
but I can see sometimes you might want "test X" to refer to some non-IPPM test - possibly another stds organisation could define such a registry?

phil
________________________________
From: lmap-bounces@ietf.org [lmap-bounces@ietf.org] On Behalf Of Bugenhagen, Michael K [Michael.K.Bugenhagen@centurylink.com]
Sent: 07 March 2013 22:16
To: Benoit Claise
Cc: lmap@ietf.org
Subject: Re: [lmap] Which performance metrics for LMAP?

Benoit,

       I think that the requirement to"future proof" the test framework in order to ensure it will support any future test we throw at it requires that the test framework be able to support both service commissioning (Throughput) type tests, as well as other SLA like tracking tests such as Twamp and the Ethernet OAM, and even service level testing.    So essentially they should all fit inside the framework

M

Sent from my iPhone

On Mar 7, 2013, at 5:28 AM, "Benoit Claise" <bclaise@cisco.com<mailto:bclaise@cisco.com>> wrote:

Dear all,

Another clarifying question: I'm not too sure which performance metric types LMAP targets.
draft-linsner-lmap-use-cases-00 mentions:

   It is assumed that measurement
   tests run by OTT providers would only include the metrics associated
   with layer 3 and up

The different drafts use different terms: service, end user experience, quality of experience.
Does Quality of Experience imply performance metrics per application, per 5 tuple (IP addresses, ports, protocol), per DCSP?
Or do we want to stay at layer 3 performance metrics, typically IPPM type of metrics?
If the latter, then LMAP is about large scale deployment of IPPM, right?

While I fully understand the business needs for "and up" in "include the metrics associated with layer 3 and up", let's not boil the ocean.
Performance Metrics at Other Layer (PMOL<http://datatracker.ietf.org/wg/pmol/charter/>), a concluded WG, tried and it proved to be difficult.

Some more discussions, on the mailing list or during the BoF, on this topic would be appreciated.

Regards, Benoit
_______________________________________________
lmap mailing list
lmap@ietf.org<mailto:lmap@ietf.org>
https://www.ietf.org/mailman/listinfo/lmap