Re: [manet] draft-nguyen-manet-ecds-mib and Performance Metrics (UNCLASSIFIED)

["Adrian Farrel" <[e-mail]>]

Hi Benoit,
 
Thanks for cutting me in on this thread.
 
I do think this type of discussion should take place on WG mailing lists so I am
copying MANET on this reply.
 
It appears that the phrase "performance metric" causes alarm bells to ring in
several fire stations around the world. Although I strongly resent words or
phrases being captured for more specific and focused meaning than they deserve,
it is also clear that the pain caused by having a hundred firemen show up on
your doorstep at 3am is sufficient (unless you like firemen at 3am ;-) to
warrant taking Benoit's advice.
 
- Avoid the term "performance metric" unless you mean it in the
 sense of RFC 6390 (or be prepared to have a fight)
- Use the term "performance information" or maybe "performance
  counters"
 
<VBS>
 
Adrian
 
From: Benoit Claise [mailto:[e-mail]] 
Sent: 24 May 2013 09:33
To: Nguyen, James H CIV USARMY CERDEC (US)
Cc: Fred Baker; [e-mail]; me;
[e-mail]; [e-mail]; Adrian Farrel
Subject: Re: draft-nguyen-manet-ecds-mib and Performance Metrics (UNCLASSIFIED)
 
Hi James, draft-ietf-manet-smf-mib authors,

Agreed, RFC 6390 doesn't apply here.
See the email exchange regarding draft-ietf-manet-olsrv2-mib below, which I
reviewed part of the performance metric directorate.
Like Ulrich did for draft-ietf-manet-olsrv2-mib, it might better to use
"performance information" instead of "performance metrics" in your draft.
Benoit,
 
thank you very much for this review. I agree that using the term
"performance information" instead of "performance metrics" is a good
idea. We will make the change.
 
Best regards
Ulrich
 
On Mon, Apr 15, 2013 at 6:19 AM, Benoit Claise  <mailto:[e-mail]>
<[e-mail]> wrote:
Dear all,
 
I reviewed http://tools.ietf.org/html/draft-ietf-manet-olsrv2-mib-06, from a
performance metric directorate point of view.
 
This draft doesn't contain any reference to RFC6390, but contains
"performance metric". Hence this review was triggered. For details about the
directorate, see
http://www.ietf.org/iesg/directorate/performance-metrics.html
 
 
Definition of Managed Objects for the  Optimized Link State Routing
Protocol version 2
 
Abstract
 
   This document defines the Management Information Base (MIB) module
   for configuring and managing the Optimized Link State Routing
   protocol version 2 (OLSRv2).  The OLSRv2-MIB module is structured
   into state information, performance metrics, and notifications.  This
   additional state and performance information is useful to
   troubleshoot problems and performance issues of the routing protocol.
   Different levels of compliance allow implementers to use smaller
   subsets of all defined objects, allowing for this MIB module to be
   deployed on more constrained routers.
 
 
Basically, all performance metrics come from this table:
 
   o  olsrv2InterfacePerfTable - records performance counters for each
      active OLSRv2 interface on this device. selected path to each
      destination for which any such path is known.  This table has
      AUGMENTS { nhdpInterfacePerfEntry } and as such it is indexed via
      nhdpIfIndex from the NHDP-MIB.
 
NHDP-MIB is RFC 6779:
   NhdpInterfacePerfEntry ::=
      SEQUENCE {
         nhdpIfHelloMessageXmits
            Counter32,
         nhdpIfHelloMessageRecvd
            Counter32,
         nhdpIfHelloMessageXmitAccumulatedSize
            Counter64,
         nhdpIfHelloMessageRecvdAccumulatedSize
            Counter64,
         nhdpIfHelloMessageTriggeredXmits
            Counter32,
         nhdpIfHelloMessagePeriodicXmits
            Counter32,
         nhdpIfHelloMessageXmitAccumulatedSymmetricNeighborCount
            Counter32,
         nhdpIfHelloMessageXmitAccumulatedHeardNeighborCount
            Counter32,
         nhdpIfHelloMessageXmitAccumulatedLostNeighborCount
            Counter32
      }
 
 
This draft contains similar objects in olsrv2InterfacePerfTable :
 
    Olsrv2InterfacePerfEntry ::=
       SEQUENCE {
          olsrv2IfTcMessageXmits
             Counter32,
          olsrv2IfTcMessageRecvd
             Counter32,
          olsrv2IfTcMessageXmitAccumulatedSize
             Counter64,
          olsrv2IfTcMessageRecvdAccumulatedSize
             Counter64,
          olsrv2IfTcMessageTriggeredXmits
             Counter32,
          olsrv2IfTcMessagePeriodicXmits
             Counter32,
          olsrv2IfTcMessageForwardedXmits
             Counter32,
          olsrv2IfTcMessageXmitAccumulatedMPRSelectorCount
             Counter32
       }
 
 
Personally, I don't believe that those objects should be subject to the RFC
6390 template definition. (Performance Metric Definition Template, section
5.4.4, RFC 6390).
First reason: NhdpInterfacePerfEntry, from NHDP-MIB [RFC 6779] was not
subject to it
Second reason: these objects are not really performance metrics, but mainly
basic monitoring objects.
 
Since RFC 6779 uses the term performance information (in the abstract), I
would propose that draft-ietf-manet-olsrv2-mib also uses this term, and not
the "performance metric". That would avoid some confusion. However, keeping
the olsrv2InterfacePerfTable OID name is perfectly fine, for consistency
reason with RFC 6779.
 
Regards, Benoit
 

Regards, Benoit
Classification: UNCLASSIFIED
Caveats: NONE
 
Fred,
 
There are two counters that are defined for performance metrics.  Please see
below.  As we stated it in Section 9 "Applicability Statement," ECDS-MIB is
an extension of SMF-MIB.  Thus, ECDS-MIB's inherited SMF-MIB's performance
metrics.  
 
As I can see, ECDS-MIB does apply 6390's (i), (ii), (iii), and somewhat
(iv).  Please let me know if you have any suggestion to improve the draft.
 
 
      (i) the degree to which its absence would cause significant loss
      of information on the behavior or performance of the application
      or system being measured
 
      (ii) the correlation between the Performance Metric, the QoS, and
      the QoE delivered to the user (person or other application)
 
      (iii) the degree to which the Performance Metric is able to
      support the identification and location of problems affecting
      service quality
 
      (iv) the requirement to develop policies (Service Level Agreement,
      and potentially Service Level Contract) based on the Performance
      Metric
 
 
 
--
 -- E-CDS Performance Group
 --
 
 ecdsPerformanceGroup OBJECT IDENTIFIER ::= { ecdsMIBObjects 3 }
 
 ecdsInEcdsChange OBJECT-TYPE
         SYNTAX          Counter32
         MAX-ACCESS      read-only
         STATUS          current
         DESCRIPTION
                 "This object indicates how many times the current
                  node is configured to be in E-CDS."
 ::= { ecdsPerformanceGroup 1 }
 
 ecdsCurrentRtrPriValueChange OBJECT-TYPE
         SYNTAX          Counter32
         MAX-ACCESS      read-only
         STATUS          current
         DESCRIPTION
                 "This object indicates how many times the Router
                  Priority of the current node has been changed."
 ::= { ecdsPerformanceGroup 2 }
 
 
 
James Nguyen
US Army CERDEC S&TCD
Email: [e-mail]
Phone:  [phone]
 
 
-----Original Message-----
From: Fred Baker [mailto:[e-mail]] 
Sent: Thursday, May 23, 2013 11:12 AM
To: [e-mail]
Cc: [e-mail]
Subject: draft-nguyen-manet-ecds-mib and Performance Metrics
 
Hi:
 
I have a question for you. Your document mentions performance metrics.
Would you kindly take a look at RFC 6390 to see if any of its considerations
apply to it?  "No" is an acceptable response, of course; the point is to ask
the question.
 
6390 Guidelines for Considering New Performance Metric Development. A.
     Clark, B. Claise. October 2011. (Format: TXT=49930 bytes) (Also
     BCP0170) (Status: BEST CURRENT PRACTICE)
 
 
Classification: UNCLASSIFIED
Caveats: NONE