Re: [manet] draft-nguyen-manet-ecds-mib and Performance Metrics (UNCLASSIFIED)

[Benoit Claise <[e-mail]>]

Hi Adrian,

> Hi Benoit,
>
> Thanks for cutting me in on this thread.
>
> I do think this type of discussion should take place on WG mailing 
> lists so I am copying MANET on this reply.
>
> It appears that the phrase "performance metric" causes alarm bells to 
> ring in several fire stations around the world. Although I strongly 
> resent words or phrases being captured for more specific and focused 
> meaning than they deserve, it is also clear that the pain caused by 
> having a hundred firemen show up on your doorstep at 3am is sufficient 
> (unless you like firemen at 3am ;-) to warrant taking Benoit's advice.
>
> - Avoid the term "performance metric" unless you mean it in the
>
> sense of RFC 6390 (or be prepared to have a fight)
>
This is simply not correct!
Based on my two messages below (for the 2 drafts):

    Since RFC 6779 uses the term performance information (in the abstract), I

    would propose that draft-ietf-manet-olsrv2-mib also uses this term, and not

    the "performance metric". That would avoid some confusion.

    ...

    Like
    Ulrich did for draft-ietf-manet-olsrv2-mib, it might better to use
    "performance information" instead of "performance metrics" in your
    draft.

These are advice, and should be considered as such.

Regards, Benoit

> - Use the term "performance information" or maybe "performance
>
> counters"
>
> <VBS>
>
> Adrian
>
> *From:*Benoit Claise [mailto:[e-mail]]
> *Sent:* 24 May 2013 09:33
> *To:* Nguyen, James H CIV USARMY CERDEC (US)
> *Cc:* Fred Baker; [e-mail]; me; 
> [e-mail]; [e-mail]; Adrian Farrel
> *Subject:* Re: draft-nguyen-manet-ecds-mib and Performance Metrics 
> (UNCLASSIFIED)
>
> Hi James, draft-ietf-manet-smf-mib authors,
>
> Agreed, RFC 6390 doesn't apply here.
> See the email exchange regarding draft-ietf-manet-olsrv2-mib below, 
> which I reviewed part of the performance metric directorate.
> Like Ulrich did for draft-ietf-manet-olsrv2-mib, it might better to 
> use "performance information" instead of "performance metrics" in your 
> draft.
>
> Benoit,
>   
> thank you very much for this review. I agree that using the term
> "performance information" instead of "performance metrics" is a good
> idea. We will make the change.
>   
> Best regards
> Ulrich
>   
> On Mon, Apr 15, 2013 at 6:19 AM, Benoit Claise<[e-mail]>  <mailto:[e-mail]>  wrote:
> Dear all,
>   
> I reviewedhttp://tools.ietf.org/html/draft-ietf-manet-olsrv2-mib-06, from a
> performance metric directorate point of view.
>   
> This draft doesn't contain any reference to RFC6390, but contains
> "performance metric". Hence this review was triggered. For details about the
> directorate, see
> http://www.ietf.org/iesg/directorate/performance-metrics.html
>   
>   
> Definition of Managed Objects for the   Optimized Link State Routing
> Protocol version 2
>   
> Abstract
>   
>     This document defines the Management Information Base (MIB) module
>     for configuring and managing the Optimized Link State Routing
>     protocol version 2 (OLSRv2).   The OLSRv2-MIB module is structured
>     into state information, performance metrics, and notifications.   This
>     additional state and performance information is useful to
>      troubleshoot problems and performance issues of the routing protocol.
>     Different levels of compliance allow implementers to use smaller
>     subsets of all defined objects, allowing for this MIB module to be
>     deployed on more constrained routers.
>   
>   
> Basically, all performance metrics come from this table:
>   
>     o   olsrv2InterfacePerfTable - records performance counters for each
>        active OLSRv2 interface on this device. selected path to each
>        destination for which any such path is known.   This table has
>        AUGMENTS { nhdpInterfacePerfEntry } and as such it is indexed via
>        nhdpIfIndex from the NHDP-MIB.
>   
> NHDP-MIB is RFC 6779:
>     NhdpInterfacePerfEntry ::=
>        SEQUENCE {
>           nhdpIfHelloMessageXmits
>              Counter32,
>           nhdpIfHelloMessageRecvd
>              Counter32,
>           nhdpIfHelloMessageXmitAccumulatedSize
>              Counter64,
>           nhdpIfHelloMessageRecvdAccumulatedSize
>              Counter64,
>           nhdpIfHelloMessageTriggeredXmits
>              Counter32,
>           nhdpIfHelloMessagePeriodicXmits
>              Counter32,
>           nhdpIfHelloMessageXmitAccumulatedSymmetricNeighborCount
>              Counter32,
>           nhdpIfHelloMessageXmitAccumulatedHeardNeighborCount
>              Counter32,
>           nhdpIfHelloMessageXmitAccumulatedLostNeighborCount
>              Counter32
>        }
>   
>   
> This draft contains similar objects in olsrv2InterfacePerfTable :
>   
>      Olsrv2InterfacePerfEntry ::=
>         SEQUENCE {
>            olsrv2IfTcMessageXmits
>               Counter32,
>            olsrv2IfTcMessageRecvd
>               Counter32,
>            olsrv2IfTcMessageXmitAccumulatedSize
>               Counter64,
>            olsrv2IfTcMessageRecvdAccumulatedSize
>               Counter64,
>            olsrv2IfTcMessageTriggeredXmits
>               Counter32,
>             olsrv2IfTcMessagePeriodicXmits
>               Counter32,
>            olsrv2IfTcMessageForwardedXmits
>               Counter32,
>            olsrv2IfTcMessageXmitAccumulatedMPRSelectorCount
>               Counter32
>         }
>   
>   
> Personally, I don't believe that those objects should be subject to the RFC
> 6390 template definition. (Performance Metric Definition Template, section
> 5.4.4, RFC 6390).
> First reason: NhdpInterfacePerfEntry, from NHDP-MIB [RFC 6779] was not
> subject to it
> Second reason: these objects are not really performance metrics, but mainly
> basic monitoring objects.
>   
> Since RFC 6779 uses the term performance information (in the abstract), I
> would propose that draft-ietf-manet-olsrv2-mib also uses this term, and not
> the "performance metric". That would avoid some confusion. However, keeping
> the olsrv2InterfacePerfTable OID name is perfectly fine, for consistency
> reason with RFC 6779.
>   
> Regards, Benoit
>   
>
>
> Regards, Benoit
>
>     Classification: UNCLASSIFIED
>
>     Caveats: NONE
>
>       
>
>     Fred,
>
>       
>
>     There are two counters that are defined for performance metrics.   Please see
>
>     below.   As we stated it in Section 9 "Applicability Statement," ECDS-MIB is
>
>     an extension of SMF-MIB.   Thus, ECDS-MIB's inherited SMF-MIB's performance
>
>     metrics.   
>
>       
>
>     As I can see, ECDS-MIB does apply 6390's (i), (ii), (iii), and somewhat
>
>     (iv).   Please let me know if you have any suggestion to improve the draft.
>
>       
>
>       
>
>            (i) the degree to which its absence would cause significant loss
>
>            of information on the behavior or performance of the application
>
>            or system being measured
>
>       
>
>            (ii) the correlation between the Performance Metric, the QoS, and
>
>            the QoE delivered to the user (person or other application)
>
>       
>
>            (iii) the degree to which the Performance Metric is able to
>
>            support the identification and location of problems affecting
>
>            service quality
>
>       
>
>            (iv) the requirement to develop policies (Service Level Agreement,
>
>            and potentially Service Level Contract) based on the Performance
>
>             Metric
>
>       
>
>       
>
>       
>
>     --
>
>       -- E-CDS Performance Group
>
>       --
>
>       
>
>       ecdsPerformanceGroup OBJECT IDENTIFIER ::= { ecdsMIBObjects 3 }
>
>       
>
>       ecdsInEcdsChange OBJECT-TYPE
>
>               SYNTAX           Counter32
>
>               MAX-ACCESS       read-only
>
>               STATUS           current
>
>               DESCRIPTION
>
>                       "This object indicates how many times the current
>
>                        node is configured to be in E-CDS."
>
>       ::= { ecdsPerformanceGroup 1 }
>
>       
>
>       ecdsCurrentRtrPriValueChange OBJECT-TYPE
>
>               SYNTAX           Counter32
>
>               MAX-ACCESS       read-only
>
>               STATUS           current
>
>               DESCRIPTION
>
>                       "This object indicates how many times the Router
>
>                        Priority of the current node has been changed."
>
>       ::= { ecdsPerformanceGroup 2 }
>
>       
>
>       
>
>       
>
>     James Nguyen
>
>     US Army CERDEC S&TCD
>
>     Email:[e-mail]  <mailto:[e-mail]>
>
>     Phone:   [phone]
>
>       
>
>       
>
>     -----Original Message-----
>
>     From: Fred Baker [mailto:[e-mail]]
>
>     Sent: Thursday, May 23, 2013 11:12 AM
>
>     To:[e-mail]  <mailto:[e-mail]>
>
>     Cc:[e-mail]  <mailto:[e-mail]>
>
>     Subject: draft-nguyen-manet-ecds-mib and Performance Metrics
>
>       
>
>     Hi:
>
>       
>
>     I have a question for you. Your document mentions performance metrics.
>
>     Would you kindly take a look at RFC 6390 to see if any of its considerations
>
>     apply to it?   "No" is an acceptable response, of course; the point is to ask
>
>     the question.
>
>       
>
>     6390 Guidelines for Considering New Performance Metric Development. A.
>
>           Clark, B. Claise. October 2011. (Format: TXT=49930 bytes) (Also
>
>           BCP0170) (Status: BEST CURRENT PRACTICE)
>
>       
>
>       
>
>     Classification: UNCLASSIFIED
>
>     Caveats: NONE
>
>       
>
>       
>