Re: [manet] draft-nguyen-manet-ecds-mib and Performance Metrics (UNCLASSIFIED)

["Cole, Robert G CIV USARMY CERDEC (US)" <[e-mail]>]

Classification: UNCLASSIFIED
Caveats: NONE

I agree that we should be more precise in our naming and descriptions.  This
question is showing up often in our MIB discussions.  I have made it a habit
of placing these counters within a PerformanceGroup instead of a
StatisticsGroup because I felt that Statistics implied we were collecting
statistical data (e.g., averages) instead of performance data (counts).  But
I agree that using the term Performance Metric goes too far and is not a
correct description of the data.  Going forward I'll try to be more
consistent and use the term 'Performance Information' as you all suggest and
as we did in RFC 6779.

Thanks,
Bob

Robert G. Cole
Comm:  443.395.8744
Email: [e-mail]


-----Original Message-----
From: Benoit Claise [mailto:[e-mail]] 
Sent: Friday, May 24, 2013 6:03 AM
To: [e-mail]
Cc: Nguyen, James H CIV USARMY CERDEC (US); 'Fred Baker';
[e-mail];
[e-mail]; [e-mail]; [e-mail]
Subject: Re: draft-nguyen-manet-ecds-mib and Performance Metrics
(UNCLASSIFIED)

Hi Adrian,



	Hi Benoit,

	 

	Thanks for cutting me in on this thread.

	 

	I do think this type of discussion should take place on WG mailing
lists so I am copying MANET on this reply.

	 

	It appears that the phrase "performance metric" causes alarm bells
to ring in several fire stations around the world. Although I strongly
resent words or phrases being captured for more specific and focused meaning
than they deserve, it is also clear that the pain caused by having a hundred
firemen show up on your doorstep at 3am is sufficient (unless you like
firemen at 3am ;-) to warrant taking Benoit's advice.

	 

	- Avoid the term "performance metric" unless you mean it in the

	 sense of RFC 6390 (or be prepared to have a fight)

This is simply not correct!
Based on my two messages below (for the 2 drafts):


	Since RFC 6779 uses the term performance information (in the
abstract), I
	would propose that draft-ietf-manet-olsrv2-mib also uses this term,
and not
	the "performance metric". That would avoid some confusion.
	
	...
	
	Like 
	Ulrich did for draft-ietf-manet-olsrv2-mib, it might better to use 
	"performance information" instead of "performance metrics" in your 
	draft.

These are advice, and should be considered as such.

Regards, Benoit



	

	- Use the term "performance information" or maybe "performance

	  counters"

	 

	<VBS>

	 

	Adrian

	 

	From: Benoit Claise [mailto:[e-mail]
<blockedmailto:[e-mail]> ] 
	Sent: 24 May 2013 09:33
	To: Nguyen, James H CIV USARMY CERDEC (US)
	Cc: Fred Baker; [e-mail]; me;
[e-mail]; [e-mail]; Adrian Farrel
	Subject: Re: draft-nguyen-manet-ecds-mib and Performance Metrics
(UNCLASSIFIED)

	 

	Hi James, draft-ietf-manet-smf-mib authors,
	
	Agreed, RFC 6390 doesn't apply here.
	See the email exchange regarding draft-ietf-manet-olsrv2-mib below,
which I reviewed part of the performance metric directorate.
	Like Ulrich did for draft-ietf-manet-olsrv2-mib, it might better to
use "performance information" instead of "performance metrics" in your
draft.

	Benoit,
	 
	thank you very much for this review. I agree that using the term
	"performance information" instead of "performance metrics" is a good
	idea. We will make the change.
	 
	Best regards
	Ulrich
	 
	On Mon, Apr 15, 2013 at 6:19 AM, Benoit Claise <[e-mail]>
<blockedmailto:[e-mail]>  wrote:
	Dear all,
	 
	I reviewed http://tools.ietf.org/html/draft-ietf-manet-olsrv2-mib-06
<blockedhttp://tools.ietf.org/html/draft-ietf-manet-olsrv2-mib-06> , from a
	performance metric directorate point of view.
	 
	This draft doesn't contain any reference to RFC6390, but contains
	"performance metric". Hence this review was triggered. For details
about the
	directorate, see
	http://www.ietf.org/iesg/directorate/performance-metrics.html
<blockedhttp://www.ietf.org/iesg/directorate/performance-metrics.html> 
	 
	 
	Definition of Managed Objects for the  Optimized Link State Routing
	Protocol version 2
	 
	Abstract
	 
	   This document defines the Management Information Base (MIB)
module
	   for configuring and managing the Optimized Link State Routing
	   protocol version 2 (OLSRv2).  The OLSRv2-MIB module is structured
	   into state information, performance metrics, and notifications.
This
	   additional state and performance information is useful to
	   troubleshoot problems and performance issues of the routing
protocol.
	   Different levels of compliance allow implementers to use smaller
	   subsets of all defined objects, allowing for this MIB module to
be
	   deployed on more constrained routers.
	 
	 
	Basically, all performance metrics come from this table:
	 
	   o  olsrv2InterfacePerfTable - records performance counters for
each
	      active OLSRv2 interface on this device. selected path to each
	      destination for which any such path is known.  This table has
	      AUGMENTS { nhdpInterfacePerfEntry } and as such it is indexed
via
	      nhdpIfIndex from the NHDP-MIB.
	 
	NHDP-MIB is RFC 6779:
	   NhdpInterfacePerfEntry ::=
	      SEQUENCE {
	         nhdpIfHelloMessageXmits
	            Counter32,
	         nhdpIfHelloMessageRecvd
	            Counter32,
	         nhdpIfHelloMessageXmitAccumulatedSize
	            Counter64,
	         nhdpIfHelloMessageRecvdAccumulatedSize
	            Counter64,
	         nhdpIfHelloMessageTriggeredXmits
	            Counter32,
	         nhdpIfHelloMessagePeriodicXmits
	            Counter32,
	         nhdpIfHelloMessageXmitAccumulatedSymmetricNeighborCount
	            Counter32,
	         nhdpIfHelloMessageXmitAccumulatedHeardNeighborCount
	            Counter32,
	         nhdpIfHelloMessageXmitAccumulatedLostNeighborCount
	            Counter32
	      }
	 
	 
	This draft contains similar objects in olsrv2InterfacePerfTable :
	 
	    Olsrv2InterfacePerfEntry ::=
	       SEQUENCE {
	          olsrv2IfTcMessageXmits
	             Counter32,
	          olsrv2IfTcMessageRecvd
	             Counter32,
	          olsrv2IfTcMessageXmitAccumulatedSize
	             Counter64,
	          olsrv2IfTcMessageRecvdAccumulatedSize
	             Counter64,
	          olsrv2IfTcMessageTriggeredXmits
	             Counter32,
	          olsrv2IfTcMessagePeriodicXmits
	             Counter32,
	          olsrv2IfTcMessageForwardedXmits
	             Counter32,
	          olsrv2IfTcMessageXmitAccumulatedMPRSelectorCount
	             Counter32
	       }
	 
	 
	Personally, I don't believe that those objects should be subject to
the RFC
	6390 template definition. (Performance Metric Definition Template,
section
	5.4.4, RFC 6390).
	First reason: NhdpInterfacePerfEntry, from NHDP-MIB [RFC 6779] was
not
	subject to it
	Second reason: these objects are not really performance metrics, but
mainly
	basic monitoring objects.
	 
	Since RFC 6779 uses the term performance information (in the
abstract), I
	would propose that draft-ietf-manet-olsrv2-mib also uses this term,
and not
	the "performance metric". That would avoid some confusion. However,
keeping
	the olsrv2InterfacePerfTable OID name is perfectly fine, for
consistency
	reason with RFC 6779.
	 
	Regards, Benoit
	 

	
	Regards, Benoit

		Classification: UNCLASSIFIED
		Caveats: NONE
		 
		Fred,
		 
		There are two counters that are defined for performance
metrics.  Please see
		below.  As we stated it in Section 9 "Applicability
Statement," ECDS-MIB is
		an extension of SMF-MIB.  Thus, ECDS-MIB's inherited
SMF-MIB's performance
		metrics.  
		 
		As I can see, ECDS-MIB does apply 6390's (i), (ii), (iii),
and somewhat
		(iv).  Please let me know if you have any suggestion to
improve the draft.
		 
		 
		      (i) the degree to which its absence would cause
significant loss
		      of information on the behavior or performance of the
application
		      or system being measured
		 
		      (ii) the correlation between the Performance Metric,
the QoS, and
		      the QoE delivered to the user (person or other
application)
		 
		      (iii) the degree to which the Performance Metric is
able to
		      support the identification and location of problems
affecting
		      service quality
		 
		      (iv) the requirement to develop policies (Service
Level Agreement,
		      and potentially Service Level Contract) based on the
Performance
		      Metric
		 
		 
		 
		--
		 -- E-CDS Performance Group
		 --
		 
		 ecdsPerformanceGroup OBJECT IDENTIFIER ::= { ecdsMIBObjects
3 }
		 
		 ecdsInEcdsChange OBJECT-TYPE
		         SYNTAX          Counter32
		         MAX-ACCESS      read-only
		         STATUS          current
		         DESCRIPTION
		                 "This object indicates how many times the
current
		                  node is configured to be in E-CDS."
		 ::= { ecdsPerformanceGroup 1 }
		 
		 ecdsCurrentRtrPriValueChange OBJECT-TYPE
		         SYNTAX          Counter32
		         MAX-ACCESS      read-only
		         STATUS          current
		         DESCRIPTION
		                 "This object indicates how many times the
Router
		                  Priority of the current node has been
changed."
		 ::= { ecdsPerformanceGroup 2 }
		 
		 
		 
		James Nguyen
		US Army CERDEC S&TCD
		Email: [e-mail]
		Phone:  [phone]
		 
		 
		-----Original Message-----
		From: Fred Baker [mailto:[e-mail]
<blockedmailto:[e-mail]> ] 
		Sent: Thursday, May 23, 2013 11:12 AM
		To: [e-mail]
		Cc: [e-mail]
		Subject: draft-nguyen-manet-ecds-mib and Performance Metrics
		 
		Hi:
		 
		I have a question for you. Your document mentions
performance metrics.
		Would you kindly take a look at RFC 6390 to see if any of
its considerations
		apply to it?  "No" is an acceptable response, of course; the
point is to ask
		the question.
		 
		6390 Guidelines for Considering New Performance Metric
Development. A.
		     Clark, B. Claise. October 2011. (Format: TXT=49930
bytes) (Also
		     BCP0170) (Status: BEST CURRENT PRACTICE)
		 
		 
		Classification: UNCLASSIFIED
		Caveats: NONE
		 
		 

	 



Classification: UNCLASSIFIED
Caveats: NONE