WG Action: RECHARTER: Benchmarking Methodology (bmwg)

[IESG Secretary <[e-mail]>]

The Benchmarking Methodology (bmwg) working group in the Operations and
Management Area of the IETF has been rechartered.  For additional
information, please contact the Area Directors or the working group
Chairs.

Benchmarking Methodology (bmwg)
---------------------------------------------------
Current Status: Active Working Group

Last Modified: 2008-07-21

Chairs:

Chair(s):
    Al Morton <[e-mail]>

Operations and Management Area Director(s):
    Dan Romascanu <[e-mail]>
    Ronald Bonica <[e-mail]>

Operations and Management Area Advisor:
    Ronald Bonica <[e-mail]>

Mailing Lists:
    General Discussion: [e-mail]
    To Subscribe: [e-mail]
    In Body: subscribe your_email_address
    Archive: http://www.ietf.org/mail-archive/web/bmwg/index.html

Description of Working Group:

The major goal of the Benchmarking Methodology Working Group is to make
a series of recommendations concerning the measurement of the
performance characteristics of various internetworking technologies;
further, these recommendations may focus on the systems or services that
are built from these technologies.

Each recommendation will describe the class of equipment, system, or
service being addressed; discuss the performance characteristics that
are pertinent to that class; clearly identify a set of metrics that aid
in the description of those characteristics; specify the methodologies
required to collect said metrics; and lastly, present the requirements
for the common, unambiguous reporting of benchmarking results.

To better distinguish the BMWG from other measurement initiatives in the
IETF, the scope of the BMWG is limited to technology characterization
using simulated stimuli in a laboratory environment.  Said differently,
the BMWG does not attempt to produce benchmarks for live, operational
networks. Moreover, the benchmarks produced by this WG shall strive to
be vendor independent or otherwise have universal applicability to a
given technology class.

Because the demands of a particular technology may vary from deployment
to deployment, a specific non-goal of the Working Group is to define
acceptance criteria or performance requirements.

An ongoing task is to provide a forum for discussion regarding the
advancement of measurements designed to provide insight on the operation
inter-networking technologies.

In addition to its current work plan, the BMWG is explicitly tasked to
develop benchmarks and methodologies for the following technologies:

* MPLS Forwarding: Develop specific methods to characterize the
latency and forwarding performance of MPLS devices, extending the
fundamental recommendations of RFC 1242 and RFC 2544 to this networking
technology.

* SIP Networking Devices: Develop new terminology and methods to
characterize the key performance aspects of network devices using
SIP, including the signaling plane scale and service rates while
considering load conditions on both the signaling and media planes. This
work will be harmonized with related SIP performance metric definitions
prepared by the PMOL working group.

Goals and Milestones:

Done	  	Expand the current Ethernet switch benchmarking methodology 
	 	draft to define the metrics and methodologies particular to 
		the general class of connectionless, LAN switches.
Done	  	Edit the LAN switch draft to reflect the input from BMWG. 
		Issue a new version of document for comment. If appropriate, 
		ascertain consensus on whether to recommend the draft for 
		consideration as an RFC.
Done	  	Take controversial components of multicast draft to mailing 
		list for discussion. Incorporate changes to draft and 
		reissue appropriately.
Done	  	Submit workplan for initiating work on Benchmarking 
		Methodology for LAN Switching Devices.
Done	  	Submit workplan for continuing work on the Terminology for 
		Cell/Call Benchmarking draft.
Done	  	Submit initial draft of Benchmarking Methodology for LAN 
		Switches.
Done	  	Submit Terminology for IP Multicast Benchmarking draft for 
		AD Review.
Done	  	Submit Benchmarking Terminology for Firewall Performance for 
		AD review
Done	  	Progress ATM benchmarking terminology draft to AD review.
Done	  	Submit Benchmarking Methodology for LAN Switching Devices 
		draft for AD review.
Done	  	Submit first draft of Firewall Benchmarking Methodology.
Done	  	First Draft of Terminology for FIB related Router 
		Performance Benchmarking.
Done	  	First Draft of Router Benchmarking Framework
Done	  	Progress Frame Relay benchmarking terminology draft to AD 
		review.
Done	  	Methodology for ATM Benchmarking for AD review.
Done	  	Terminology for ATM ABR Benchmarking for AD review.
Done	  	Terminology for FIB related Router Performance Benchmarking 
		to AD review.
Done	  	Firewall Benchmarking Methodology to AD Review
Done	  	First Draft of Methodology for FIB related Router 
		Performance Benchmarking.
Done	  	First draft Net Traffic Control Benchmarking Methodology.
Done	  	Methodology for IP Multicast Benchmarking to AD Review.
Done	  	Resource Reservation Benchmarking Terminology to AD Review
Done	  	First I-D on IPsec Device Benchmarking Terminology
Done	  	EGP Convergence Benchmarking Terminology to AD Review
Done	  	Resource Reservation Benchmarking Methodology to AD Review
Done	  	Net Traffic Control Benchmarking Terminology to AD Review
Done	  	IGP/Data-Plane Terminology I-D to AD Review
Done	  	IGP/Data-Plane Methodology and Considerations I-Ds to AD 
		Review
Done	  	Hash and Stuffing I-D to AD Review
Done	  	IPv6 Benchmarking Methodology to AD Review
Sep 2008	  	IPsec Device Benchmarking Terminology to IESG Review
Sep 2008	  	IPsec Device Benchmarking Methodology to IESG Review
Dec 2008	  	Net Traffic Control Benchmarking Methodology to AD 
			Review.
Dec 2008	  	Router Accelerated Test Terminology to IESG Review
Dec 2008	  	Router Accelerated Test Methodology to IESG Review
Feb 2009	  	Terminology For Protection Benchmarking to AD Review
Feb 2009	  	Methodology For Protection Benchmarking to AD Review
Apr 2009	  	Methodology for MPLS Forwarding to AD Review
Jun 2009	  	Terminology for SIP Device Benchmarking to IESG Review
Jun 2009	  	Methodology for SIP Device Benchmarking to IESG Review
Dec 2009	  	Router Accelerated Test Method for EBGP to IESG Review
Dec 2009	  	Router Accelerated Test Method for Operational 
			Security to IESG Review
Jul 2010	  	Basic BGP Convergence Benchmarking Methodology to AD 
			Review.
_______________________________________________
IETF-Announce mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/ietf-announce