[PMOL] Re: A question about [Fwd: WG Review: Performance Metrics at Other Layers (pmol)]

[Yangwoo Ko <[e-mail]>]

Since application developers have developed many tricks to smooth out 
some peculiarities of network-based protocols (e.g. jitter, delay, drops 
and so on), it is very often hard to related the performance 
measured/perceived at application layer with those of underlying layers. 
If this WG provides guidelines (or at least hints) to get over this 
vagueness, I will be very happy.

Regards

Leslie Daigle wrote:
> 
> 
> At the Application Performance Metrics BoF in
> Chicago, there was a lot of discussion about whether
> or how to best capture expertise in the area of performance
> metrics for application to IETF protocols.
> 
> There seemed to be 2 separate questions on the table:
> 
>     1/ how, and
>     2/ whether
> 
> to generalize a performance metric framework for IETF
> protocols.
> 
> 
> The proposed PMOL WG addresses the first question -- how
> to do it.
> 
> On the second point -- the question is really about whether
> the IETF community as a whole supports/values performance
> metrics and will invest the effort in using any general
> framework for existing/new protocols.
> 
> I believe the folks that joined the PMOL mailing list after
> the APM BoF are assuming that support is there.
> 
> I suggest that now is an excellent time for folks _not_
> involved in the PMOL effort, but who are working on
> protocols that could/should make use of its output, to
> voice some opinions on whether or not this approach
> is helpful/will be useful to them in future protocols.
> 
> 
> Leslie.
> 
> 
> -------- Original Message --------
> Subject: WG Review: Performance Metrics at Other Layers (pmol)
> Date: Mon, 22 Oct 2007 14:15:02 -0400
> From: IESG Secretary <[e-mail]>
> Reply-To: [e-mail]
> To: [e-mail]
> 
> A new IETF working group has been proposed in the Operations and
> Management Area.  The IESG has not made any determination as yet.
> The following draft charter was submitted, and is provided for
> informational purposes only.  Please send your comments to the IESG
> mailing list ([e-mail]) by October 29.
> 
> +++
> 
> Performance Metrics at Other Layers (pmol)
> ==============================================
> 
> Current Status: Proposed Working Group
> 
> WG Chairs:
> TBD
> 
> Operations and Management Area:
> Dan Romascanu <[e-mail]>
> Ronald Bonica <[e-mail]>
> 
> Description:
> 
> The successful implementation and operation of IP based applications
> often depends on some underlying performance measurement infrastructure
> that helps service operators or network managers to recognize when
> performance is unsatisfactory and identify problems affecting service
> quality. Standardized performance metrics add the desirable features of
> consistent implementation, interpretation, no comparison.
> 
> The IETF has two Working Groups dedicated to the development of
> performance metrics however each has strict limitations in their
> charters:
> 
> - The Benchmarking Methodology WG has addressed a range of networking
> technologies and protocols in their long history (such as IEEE 802.3,
> ATM, Frame Relay, and Routing Protocols), but the charter strictly
> limits their Performance characterizations to the laboratory
> environment.
> 
> - The IP Performance Metrics WG has the mandate to develop metrics
> applicable to the performance of Internet data delivery, but it is
> specifically prohibited from developing metrics that characterize
> traffic (such as a VoIP stream).
> 
> The IETF also has current and completed activities related to the
> reporting of application performance metrics (e.g. RAQMON and RTCP XR)
> and is also actively involved in the development of reliable transport
> protocols which would affect the relationship between IP performance and
> application performance.
> 
> Thus there is a gap in the currently chartered coverage of IETF WGs:
> development of performance metrics for IP-based protocols and
> applications that operate over UDP, TCP, SCTP, DCCP, Forward Error
> Correction (FEC) and other robust transport protocols, and that can be
> used to characterize traffic on live networks.
> 
> The working group will focus on the completion of two RFCs:
> 
> 1. A PMOL framework and guidelines memo that will describe the necessary
> elements of performance metrics of protocols and applications
> transported over IETF-specified protocols (such as the formal
> definition, purpose, and units of measure) and the various types of
> metrics that characterize traffic on live networks (such as metrics
> derived from other metrics, possibly on lower layers). The framework
> will also address the need to specify the intended audience and the
> motivation for the performance metrics. There will also be guidelines
> for a performance metric development process that includes entry
> criteria for new proposals (how a proposal might be evaluated for
> possible endorsement by a protocol development working group), and how
> an successful proposal will be developed by PMOL WG in cooperation with
> a protocol development WG.
> 
> 2. A proof-of-concept RFC defining performance metrics for SIP, based on
> draft-malas-performance-metrics. This memo would serve as an example of
> the framework and the PMOL development process in the IETF.
> 
> Discussion of new work proposals is strongly discouraged under the
> initial charter of the PMOL WG, except to advise a protocol development
> WG when they are evaluating a new work proposal for related performance
> metrics.
> 
> The PMOL WG will also be guided by a document describing how memos
> defining performance metrics are intended to advance along the IETF
> Standards track (draft-bradner-metricstest).
> 
> PMOL WG will take advantage of expertise and seek to avoid overlap with
> other standards development organizations, such as ETSI STQ, ITU-T SG
> 12, ATIS IIF, ATIS PRQC, and others.
> 
> Milestones
> 
> June 08 SIP Performance Metrics Draft to IESG Review for consideration
> as Proposed Standard
> 
> Sept 08 PMOL Framework and Guidelines Draft to IESG Review for
> consideration as BCP
> 
> _______________________________________________
> IETF-Announce mailing list
> [e-mail]
> https://www1.ietf.org/mailman/listinfo/ietf-announce
> 



_______________________________________________
PMOL mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/pmol