RE: [OPS-AREA] RE: WG Review: Performance Metrics at Other Layers (pmol)

["Natale, Bob" <[e-mail]>]

Hi Leen,

I'd like to strongly endorse the point made in your last paragraph --
top-down metrics identification and orientation is essential for
efficient and effective solution design (focus, prioritization, etc.).

[Apologies if the attachment is overkill in this venue, but I'm using
it in several other contexts to show the criticality of metrics (i.e.,
the primary subset of a domain ontology to be addressed) and how that
might fit into a policy-driven management context.  I would appreciate
any feedback on that, critical or otherwise.]

Cheers,
BobN

-----Original Message-----
From: MAK, Leen (Leen) [mailto:[e-mail]] 
Sent: Wednesday, October 24, 2007 9:50 AM
To: [e-mail]
Cc: [e-mail]
Subject: [OPS-AREA] RE: WG Review: Performance Metrics at Other Layers
(pmol)

All,

I read in the draft charter: "The framework will also address the need
to specify the intended audience and the motivation for the performance
metrics."

The word "also" suggests that this task is understood as a kind of
secondary task, next to the task to "describe the necessary elements of
performance metrics of protocols and applications ".

I would like to suggest that the specification of intended audience and
motivation should be a primary task.
If working bottom-up from the consideration of protocols and their
properties, it is very easy and very tempting to define all sorts of
nice and fancy parameters and metrics. However, collecting those
parameters and storing them and processing them does not come for free.

In order to limit the set of metrics to what is really useful, the
approach should be the other way. The first question to be answered
should be: which metrics are needed by the network operators to
properly
maintain the network?

Regards,

Leen Mak.



>>  -----Original Message-----
>>  From: IESG Secretary [mailto:[e-mail]]
>>  Sent: Monday, October 22, 2007 8:15 PM
>>  To: [e-mail]
>>  Subject: WG Review: Performance Metrics at Other Layers (pmol)
>>  
>>  A new IETF working group has been proposed in the 
>>  Operations and Management Area.  The IESG has not made any 
>>  determination as yet.
>>  The following draft charter was submitted, and is provided 
>>  for informational purposes only.  Please send your comments 
>>  to the IESG mailing list ([e-mail]) by October 29.
>>  
>>  +++
>>  
>>  Performance Metrics at Other Layers (pmol) 
>>  ==============================================
>>  
>>  Current Status: Proposed Working Group
>>  
>>  WG Chairs:
>>  TBD
>>  
>>  Operations and Management Area:
>>  Dan Romascanu <[e-mail]>
>>  Ronald Bonica <[e-mail]>
>>  
>>  Description:
>>  
>>  The successful implementation and operation of IP based 
>>  applications often depends on some underlying performance 
>>  measurement infrastructure that helps service operators or 
>>  network managers to recognize when performance is 
>>  unsatisfactory and identify problems affecting service 
>>  quality. Standardized performance metrics add the desirable 
>>  features of consistent implementation, interpretation, no 
>>  comparison.
>>  
>>  The IETF has two Working Groups dedicated to the 
>>  development of performance metrics however each has strict 
>>  limitations in their
>>  charters:
>>  
>>  - The Benchmarking Methodology WG has addressed a range of 
>>  networking technologies and protocols in their long history 
>>  (such as IEEE 802.3, ATM, Frame Relay, and Routing 
>>  Protocols), but the charter strictly limits their 
>>  Performance characterizations to the laboratory environment.
>>  
>>  - The IP Performance Metrics WG has the mandate to develop 
>>  metrics applicable to the performance of Internet data 
>>  delivery, but it is specifically prohibited from developing 
>>  metrics that characterize traffic (such as a VoIP stream).
>>  
>>  The IETF also has current and completed activities related 
>>  to the reporting of application performance metrics (e.g. 
>>  RAQMON and RTCP XR) and is also actively involved in the 
>>  development of reliable transport protocols which would 
>>  affect the relationship between IP performance and 
>>  application performance.
>>  
>>  Thus there is a gap in the currently chartered coverage of IETF
WGs:
>>  development of performance metrics for IP-based protocols 
>>  and applications that operate over UDP, TCP, SCTP, DCCP, 
>>  Forward Error Correction (FEC) and other robust transport 
>>  protocols, and that can be used to characterize traffic on 
>>  live networks.
>>  
>>  The working group will focus on the completion of two RFCs:
>>  
>>  1. A PMOL framework and guidelines memo that will describe 
>>  the necessary elements of performance metrics of protocols 
>>  and applications transported over IETF-specified protocols 
>>  (such as the formal definition, purpose, and units of 
>>  measure) and the various types of metrics that characterize 
>>  traffic on live networks (such as metrics derived from 
>>  other metrics, possibly on lower layers). The framework 
>>  will also address the need to specify the intended audience 
>>  and the motivation for the performance metrics. There will 
>>  also be guidelines for a performance metric development 
>>  process that includes entry criteria for new proposals (how 
>>  a proposal might be evaluated for possible endorsement by a 
>>  protocol development working group), and how an successful 
>>  proposal will be developed by PMOL WG in cooperation with a 
>>  protocol development WG.
>>  
>>  2. A proof-of-concept RFC defining performance metrics for 
>>  SIP, based on draft-malas-performance-metrics. This memo 
>>  would serve as an example of the framework and the PMOL 
>>  development process in the IETF.
>>  
>>  Discussion of new work proposals is strongly discouraged 
>>  under the initial charter of the PMOL WG, except to advise 
>>  a protocol development WG when they are evaluating a new 
>>  work proposal for related performance metrics.
>>  
>>  The PMOL WG will also be guided by a document describing 
>>  how memos defining performance metrics are intended to 
>>  advance along the IETF Standards track (draft-bradner-metricstest).
>>  
>>  PMOL WG will take advantage of expertise and seek to avoid 
>>  overlap with other standards development organizations, 
>>  such as ETSI STQ, ITU-T SG 12, ATIS IIF, ATIS PRQC, and others.
>>  
>>  Milestones
>>  
>>  June 08 SIP Performance Metrics Draft to IESG Review for 
>>  consideration as Proposed Standard
>>  
>>  Sept 08 PMOL Framework and Guidelines Draft to IESG Review 
>>  for consideration as BCP
>>  
>>  
>>  _______________________________________________
>>  OPS-AREA mailing list
>>  [e-mail]
>>  https://www1.ietf.org/mailman/listinfo/ops-area
>>  
>>  ************************************************************
>>  **********
>>  To unsubscribe: 
>>  <mailto:[e-mail]>
>>  


_______________________________________________
OPS-AREA mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/ops-area

_______________________________________________
OPS-AREA mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/ops-area