[OPS-AREA] FW: WG Review: Recharter of Benchmarking Methodology Working Group (bmwg)

["Romascanu, Dan (Dan)" <[e-mail]>]

 

-----Original Message-----
From: [e-mail]
[mailto:[e-mail]] On Behalf Of The IESG
Sent: Thursday, July 03, 2008 12:30 AM
To: [e-mail]
Cc: [e-mail]; [e-mail]
Subject: WG Review: Recharter of Benchmarking Methodology Working Group
(bmwg) 

A modified charter has been submitted for the Benchmarking Methodology
Working Group (bmwg) working group in the Operations and Management Area
of the IETF.  The IESG has not made any determination as yet.  The
modified charter is provided below for informational purposes only. 
Please send your comments to the IESG mailing list ([e-mail]) by
Wednesday, July 9, 2008.

Benchmarking Methodology Working Group (bmwg)
=====================================
Last Modified: 2008-06-13  

Current Status: Active Working Group  

Chair(s):
    Al Morton <[e-mail]>   

Operations and Management Area Director(s):     
    Dan Romascanu <[e-mail]>
    Ronald Bonica <[e-mail]>   

Operations and Management Area Advisor:     
    Ronald Bonica <[e-mail]>   

Mailing Lists:     
    General Discussion: [e-mail]
    In Body: subscribe your_email_address
    Archive: http://www.ietf.org/mail-archive/web/bmwg/index.html   

Description of Working Group:

The major goal of the Benchmarking Methodology Working Group is to make
a series of recommendations concerning the measurement of the
performance characteristics of various internetworking technologies;
further, these recommendations may focus on the systems or services that
are built from these technologies.

Each recommendation will describe the class of equipment, system, or
service being addressed; discuss the performance characteristics that
are pertinent to that class; clearly identify a set of metrics that aid
in the description of those characteristics; specify the methodologies
required to collect said metrics; and lastly, present the requirements
for the common, unambiguous reporting of benchmarking results.

To better distinguish the BMWG from other measurement initiatives in the
IETF, the scope of the BMWG is limited to technology characterization
using simulated stimuli in a laboratory environment.
Said differently, the BMWG does not attempt to produce benchmarks for
live, operational networks. Moreover, the benchmarks produced by this WG
shall strive to be vendor independent or otherwise have universal
applicability to a given technology class.

Because the demands of a particular technology may vary from deployment
to deployment, a specific non-goal of the Working Group is to define
acceptance criteria or performance requirements.

An ongoing task is to provide a forum for discussion regarding the
advancement of measurements designed to provide insight on the operation
inter-networking technologies.

In addition to its current work plan, the BMWG is explicitly tasked to
develop benchmarks and methodologies for the following technologies:

* MPLS Forwarding: Develop specific methods to characterize the latency
and forwarding performance of MPLS devices, extending the fundamental
recommendations of RFC 1242 and RFC 2544 to this networking technology.

* SIP Networking Devices: Develop new terminology and methods to
characterize the key performance aspects of network devices using SIP,
including the signaling plane scale and service rates while considering
load conditions on both the signaling and media planes. This work will
be harmonized with related SIP performance metric definitions prepared
by the PMOL working group.

Goals and Milestones:

Done Expand the current Ethernet switch benchmarking methodology draft
to define the metrics and methodologies particular to the general class
of connectionless, LAN switches.

Done Edit the LAN switch draft to reflect the input from BMWG. Issue a
new version of document for comment. If appropriate, ascertain consensus
on whether to recommend the draft for consideration as an RFC.

Done Take controversial components of multicast draft to mailing list
for
 
discussion. Incorporate changes to draft and reissue appropriately.

Done Submit workplan for initiating work on Benchmarking Methodology for
LAN Switching Devices.

Done Submit workplan for continuing work on the Terminology for
Cell/Call
 
Benchmarking draft.

Done Submit initial draft of Benchmarking Methodology for LAN Switches.

Done Submit Terminology for IP Multicast Benchmarking draft for AD
Review.

Done Submit Benchmarking Terminology for Firewall Performance for AD
review

Done Progress ATM benchmarking terminology draft to AD review.

Done Submit Benchmarking Methodology for LAN Switching Devices draft for
AD review.

Done Submit first draft of Firewall Benchmarking Methodology.

Done First Draft of Terminology for FIB related Router Performance
Benchmarking.

Done First Draft of Router Benchmarking Framework

Done Progress Frame Relay benchmarking terminology draft to AD review.

Done Methodology for ATM Benchmarking for AD review.

Done Terminology for ATM ABR Benchmarking for AD review.

Done Terminology for FIB related Router Performance Benchmarking to AD
review.

Done Firewall Benchmarking Methodology to AD Review

Done First Draft of Methodology for FIB related Router Performance
Benchmarking.

Done First draft Net Traffic Control Benchmarking Methodology.

Done Methodology for IP Multicast Benchmarking to AD Review.

Done Resource Reservation Benchmarking Terminology to AD Review

Done First I-D on IPsec Device Benchmarking Terminology

Done EGP Convergence Benchmarking Terminology to AD Review

Done Resource Reservation Benchmarking Methodology to AD Review

Done Net Traffic Control Benchmarking Terminology to AD Review

Done IGP/Data-Plane Terminology I-D to AD Review

Done IGP/Data-Plane Methodology and Considerations I-Ds to AD Review

Done Hash and Stuffing I-D to AD Review

Done IPv6 Benchmarking Methodology to AD Review

REVISED
Sept 2008 IPsec Device Benchmarking Terminology to IESG Review Sept 2008
IPsec Device Benchmarking Methodology to IESG Review

Dec 2008 Net Traffic Control Benchmarking Methodology to AD Review.
Dec 2008 Router Accelerated Test Terminology to IESG Review Dec 2008
Router Accelerated Test Methodology to IESG Review


Dec 2009 Router Accelerated Test Method for EBGP to IESG Review Dec 2009
Router Acc. Test Method for Operational Security to IESG Review Feb 2009
Terminology For Protection Benchmarking to AD Review Feb 2009
Methodology For Protection Benchmarking to AD Review Jul 2010 Basic BGP
Convergence Benchmarking Methodology to AD Review.

NEW
April 2009 Methodology for MPLS Forwarding to AD Review.
June 2009 Terminology for SIP Device Benchmarking to IESG Review.
June 2009 Methodology for SIP Device Benchmarking to IESG Review.
_______________________________________________
IETF-Announce mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/ietf-announce
_______________________________________________
OPS-AREA mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/ops-area