[lmap] Standardize behaviors for test environments - lmap framework and IPPM registries.

["Bugenhagen, Michael K" <[e-mail]>]

Al, Phil,

   In order to execute a test we need consistant MA behavior "about" or when different test environment variables are encountered.

As Al pointed out, we have some "information model" objects that call out different environmental conditions -

First -  We need to identify those with definitions vs. just objects.

Secondly - but very possibly more challenging -

	How do we take those "environmental" attributes and define the test behavior around those when the test being used is in a registry.

Example case ...

Test A = throughput
Test B = latency
Environmental x = cross traffic.

Behaviors of the test with the environmental... 
For Test A ---  terminate a test when environmental "x" is detected, then mark the test as terminated due to "environment x".
For Test B ---  record environmental "x" during test b.

What this indicates to me is that we need a very Standard Test Execution string that details out all the environmental with Codified "behavior" codes as to what the MA does during and after the test when it experiences a environmental..

	More problematically operationally reaction to things like cross traffic generally doesn't use "lab like" absolutes like "zero cross traffic" so we will also have some fuzzy logic (thresholds) in our environmental definitions.

So - Cap statement....
At the end of the day it looks like we still need a standardized (about environmental) test instruction set that is implemented the same across the board.   Both from a Attribute, and behavior aspect.

This constitutes a Gap that needs to be fixed.


Regards,
Mike

   









-----Original Message-----
From: MORTON, ALFRED C (AL) [mailto:[e-mail]] 
Sent: Wednesday, April 16, 2014 3:11 PM
To: Bugenhagen, Michael K; 'MARCELO BAGNULO BRAUN'; '[e-mail]'
Subject: RE: [lmap] lmap framework and IPPM registries.

In the original registry proposal(s) over a year ago now, we had an Environment (sub) Registry:
http://tools.ietf.org/html/draft-bagnulo-ippm-new-registry-00#section-2.3

We were convinced to take this out of the registry design, it's not really part of the metric specifications and there likely were other reasons I've forgotten, but it's out now for good.

The Info model draft covers this topic.
http://tools.ietf.org/html/draft-ietf-lmap-information-model-00#section-3.7
...
   The result data rows may optionally include an indication of the
   cross-traffic (e.g., the total number of octets of non-measurement
   traffic passing through the interfaces used by a Measurement Task
   during the measurement period).

Al

> -----Original Message-----
> From: Bugenhagen, Michael K 
> [mailto:[e-mail]]
> Sent: Wednesday, April 16, 2014 3:50 PM
> To: MORTON, ALFRED C (AL); 'MARCELO BAGNULO BRAUN'; '[e-mail]'
> Subject: RE: [lmap] lmap framework and IPPM registries.
> 
> Marcelo, Al,
> 
>     One gap I think we have here is how to identify a common set of 
> "network environment" attributes that need to be used by the tests.
> They aren't identified in the test methodology as a common abstract 
> MIB, or attribute across the board so this needs to be handled elsewhere.
> 
> My assumption what that it would be in the registry as some type of 
> "header" or description on how to execute a test, but if I'm following 
> you that assumption is wrong.
> 
>    Example -
> 
> Cross Traffic, or customer traffic...
> 
> Some tests have assumptions about the presence of this, and when it's 
> acceptable to do the test, or not based on it's existence.
> Given there is NO common MIB / attribute for it today... this becomes 
> an issue in defining the "test method"
> 
> Kind of a gap... with different ways to close it.
> 
> Regards,
> Mike
> 
> 
> 
> 
> -----Original Message-----
> From: lmap [mailto:[e-mail]] On Behalf Of MORTON, ALFRED 
> C
> (AL)
> Sent: Tuesday, April 15, 2014 3:14 PM
> To: MARCELO BAGNULO BRAUN; [e-mail]
> Subject: Re: [lmap] lmap framework and IPPM registries.
> 
> For me and those whose needs I understand, a URI for metrics seems useful.
> I would guess that IANA has already been asked for some form of ID 
> persistence, and if they agree to provide when requested, then it 
> would be a (new) requirement in the IANA section.
> 
> Also, we need both a metric and a method of measurement, these are 
> both present in our current version of the metric registry.
> 
> >From Framework 5.2.1 Instruction
> 
>    o  the Measurement Task configurations, each of which needs:
> 
>       *  the Measurement Method, specified as a URN to a registry entry.
>          The registry could be defined by the IETF
>          [I-D.manyfolks-ippm-metric-registry], locally by the operator
>          of the measurement system or perhaps by another standards
>          organisation.
> 
> So let's fix this as follows:
> 
>    o  the Measurement Task configurations, each of which needs:
> 
>       *  the Registered Metric, specified as a URI to a registry entry,
>          and which includes the specification of a Measurement Method.
>          The registry could be defined by the IETF
>          [I-D.manyfolks-ippm-metric-registry], locally by the operator
>          of the measurement system or perhaps by another standards
>          organisation.
> 
> (off-list discussion contributed to this suggestion), Al
> 
> 
> Al
> 
> > -----Original Message-----
> > From: lmap [mailto:[e-mail]] On Behalf Of MARCELO 
> > BAGNULO BRAUN
> > Sent: Monday, April 14, 2014 10:50 AM
> > To: [e-mail]
> > Subject: [lmap] lmap framework and IPPM registries.
> >
> > Hi,
> >
> > One issue i realized is that the current lmap framework states the 
> > the measurement method is specified as a URN to a registry entry, 
> > while the current IPPM registry draft do not have a URN as metric 
> > identifier but a name and a metric identifier.
> >
> > I think it is very important that these two efforts are aligned so i 
> > suggest that we discuss whether a URN is the proper way to go or the 
> > current flat identifier proposed in the ippm registry drafts.
> >
> > I am uncertain what is the benefit of an URN versus a flat identifier.
> > Regards, marcelo
> >
> >
> > --
> > ----
> > MARCELO BAGNULO BRAUN
> > WebCartero
> > Universidad Carlos III de Madrid
> >
> > _______________________________________________
> > lmap mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/lmap
> 
> _______________________________________________
> lmap mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/lmap