RE: [bmwg] Draft Agenda for bmwg at IETF-67
"Poretsky, Scott" <sporetsky@reefpoint.com> Fri, 20 October 2006 14:18 UTC
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1GavCt-0004wL-Q3; Fri, 20 Oct 2006 10:18:43 -0400
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1GavCs-0004wF-BT for bmwg@ietf.org; Fri, 20 Oct 2006 10:18:42 -0400
Received: from client62.quarrytech.com ([4.17.144.62] helo=ZOE.RPS.local) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1GavCg-0007CC-40 for bmwg@ietf.org; Fri, 20 Oct 2006 10:18:42 -0400
Subject: RE: [bmwg] Draft Agenda for bmwg at IETF-67
MIME-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Date: Fri, 20 Oct 2006 10:18:27 -0400
Content-class: urn:content-classes:message
Message-ID: <4BAEA3008BEC574095447FF2A47AAD0829FEBC@ZOE.RPS.local>
X-MS-Has-Attach:
X-MimeOLE: Produced By Microsoft Exchange V6.5
X-MS-TNEF-Correlator:
Thread-Topic: [bmwg] Draft Agenda for bmwg at IETF-67
Thread-Index: Acb0TJlac+0iCR4lRLCTKsmvVCKpoAAAyARwAAC0oaA=
From: "Poretsky, Scott" <sporetsky@reefpoint.com>
To: Al Morton <acmorton@att.com>, bmwg@ietf.org
X-Spam-Score: 0.1 (/)
X-Scan-Signature: 46ad68ada464411807db2a0edd5648ae
Cc:
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
Errors-To: bmwg-bounces@ietf.org
SIP Performance Methodology is at http://www.ietf.org/internet-drafts/draft-poretsky-bmwg-sip-bench-meth-00.txt. Scott -----Original Message----- From: Poretsky, Scott Sent: Friday, October 20, 2006 10:14 AM To: 'Al Morton'; bmwg@ietf.org Subject: RE: [bmwg] Draft Agenda for bmwg at IETF-67 Hi BMWG-ers, Below is the Work Item Proposal submitted this past May for SIP Performance Benchmarking. This proposed work item was already discussed at IETF 66 in the SIPPING and BMWG working groups. The posted Terminology and Methodology (links below) can be reviewed for additional detail. At this IETF 67 meeting we will review incorporated changes, revised scope, and methodologies. www.ietf.org/internet-drafts/draft-poretsky-sip-bench-term-01.txt (-02 coming by Monday) meth-00 was submitted last week but is still awaiting posting. Scott ############################# BMWG Work Item Proposal TITLE: Session Initiation Protocol (SIP) Performance Benchmarking AUTHOR TEAM: Scott Poretsky, Reef Point Systems Dr. Carol Davids, Illinois Institute of Technology Dr. Vijay Gurbani, Lucent Technologies GOALS The purpose of this work item is to provide a single terminology and methodology from which SIP equipment vendors and VoIP service providers can measure performance benchmarking metrics for comparison and selection. It is intended to develop terms, benchmarks, and methodologies that can be applied to any type of IP device including SIP Servers, Session Border Controllers (SBCs), and Security Gateways (SEGs). MOTIVATION Service Providers are now planning VoIP and Multimedia network deployments using the IETF developed Session Initiation Protocol (SIP). Through SIP, service providers will be able to have rich service offerings and many even plan to turn off their Public Switch Telephone Network within five years. VoIP has led to development of new networking devices including SIP Servers, Session Border Controllers, and Security Gateways. The mix of voice and IP functions in this variety of devices has produced inconsistencies in vendor reported performance metrics and has caused confusion in the service provider community. SCOPE: This work item will provide terms, benchmarks, and methodologies for performance benchmarking the SIP control and media planes. The methodologies can be used for benchmarking SIP performance of SIP servers, SBCs, and SEGs. The media used for the benchmarking will be the IETF defined Real-Time Protocol (RTP). Test cases will allow testing of VoIP or Multimedia over IP by varying the number of media streams per SIP call from 1 to higher. There will be at least one test case to measure the impact of a SIP DOS Attack on the performance metrics. All SIP traffic will be in the clear; no encryption using TLS or IPsec will be used. Similar benchmarks and methodologies using encryption is a potential work item for the BMWG that could be considered in the future. There will be separate test cases in the methodology document for obtaining measurements for each of the Benchmarking Metrics defined in the Terminology. The benchmarking metrics to be measured for the SIP Control Plane and SIP Media Plane are as follow: Benchmark Metrics - SIP Control Plane Standing Calls, maximum (calls) Calls Per Second, maximum (CPS) Call Attempts Per Second, maximum (CAPS) Busy Hour Call Attempts, maximum (BHCA) Busy Hour Call Connects, maximum (BHCC) Call Completion Rate (%) Call Setup Delay, average (msec) Call Teardown Delay, average (msec) Benchmark Metrics- SIP Media Plane RTP Media Throughput, per Media Stream (pps) RTP Media Throughput, Aggregate (pps) RTP Packet Loss, average (pps) RTP Packet Delay, average (msec) RTP Packet Jitter, average (msec) In order to obtain measurements for the benchmarking metrics it will be necessary to configure the test setup with certain test parameters. Some of these test parameters may also be the benchmarking metric being measured for another test case. The test parameters to be configured for the SIP Control Plane and SIP Media Plane are as follow: Test Parameters- SIP Control Plane Call Duration (msec) Call Per Second (CPS) Call Attempts Per Second (CAPS) Test Parameters - SIP Media Plane RTP Media Streams per Call (streams per call) RTP Packet Size (bytes) RTP Media Offered Load, per Media Stream (pps) RTP Media Offered Load, Aggregate (pps) The basic test topology to be used for this benchmarking is as follows: Emulated Agents<--> SEG (optional) <à SIP Proxy Server (or SBC) <--> SIP Server (optional) <---> Emulated Agents PROPOSED MILESTONES 06/07 First Draft Terminology and Methodology 02/07 First WG Last Call 05/07 SIP WG and SIPPING WG Review 06/07 Final BMWG Last Call for Terminology and Methodology 08/07 Submittal for IESG review -----Original Message----- From: Al Morton [mailto:acmorton@att.com] Sent: Friday, October 20, 2006 9:34 AM To: bmwg@ietf.org Subject: [bmwg] Draft Agenda for bmwg at IETF-67 Benchmarking Methodology WG (bmwg) Tuesday, Afternoon Session I 1300-1500 Room Name: Nautilus 1 CHAIR(s): Al Morton <acmorton@att.com> AGENDA: 0. Agenda bashing (if we need to shuffle a few items) See https://datatracker.ietf.org/public/meeting_materials.cgi?meeting_num=67 for Agenda updates and Slides 1. Working Group Status (Chair) Topics/Drafts not covered by presentations below: Hash and Stuffing Draft (WGLC completed) http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-hash-stuffing-06.txt Terms and Methods for Benchmarking IPsec Devices (WG review and feedback is needed! Please read and comment...) http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-ipsec-term-08.txt http://www.ietf.cnri.reston.va.us/internet-drafts/draft-ietf-bmwg-ipsec-meth-01.txt (Recently Expired, see http://tools.ietf.org/wg/bmwg/ for these versions) Methodology for Benchmarking Network-layer Traffic Control Mechanisms http://tools.ietf.org/wg/bmwg/draft-ietf-bmwg-dsmmeth/draft-ietf-bmwg-dsmmeth-02.txt IGP Data plane convergence benchmark I-Ds - Publication Requested Standard Introduction/Security Paragraph for BMWG memos Check the BMWG mail archive for comments: http://www.ietf.org/mail-archive/working-groups/bmwg/current/ 2. Techniques for Benchmarking Core Router Accelerated Stress Testing. Discussion GOALS: Review Last Call responses (numerous from many reviewers), updates to new revisions, and open issues Draft file name, or preferably the complete URL: http://www.ietf.org/internet-drafts/draft-ietf-bmwg-acc-bench-term-10.txt http://www.ietf.org/internet-drafts/draft-ietf-bmwg-acc-bench-meth-06.txt draft-ietf-bmwg-acc-bench-meth-opsec-00 Presenter: Scott Poretsky 3. IPv6 Benchmarking Methodology Discussion GOALS: a) Updates based on BMWG and v6ops feedback - Includes recommendations for benchmarking with traffic that has the Hop-by-Hop Extension Header (as suggested by v6ops and BMWG) - Updated frame size recommendations for Ethernet to include Jumbo frames (Appendix updated to include line rates values for all frame sizes) - Proposal for IPv6 benchmarking prefix assignment by IANA - The Security section contains the new standard text discussed within BMWG (we can take this opportunity to discuss the Security write-up) b) Call for making the document a WG work item and discuss next steps Presenter: Gunter van de Velde, Diego Dugatkin, or Ahmed Hamza (one of these three gentlemen will be presenting) Draft: http://tools.ietf.org/wg/bmwg/draft-popoviciu-bmwg-ipv6benchmarking-02.txt 4. Sub-IP Protection Mechanisms Discussion GOALS: Introduce this new official work item and review changes in the -00 terminology and methodology documents different from the individual submittals. Presenter: Scott Poretsky Drafts: http://www.ietf.org/internet-drafts/draft-ietf-bmwg-protection-term-00.txt http://www.ietf.org/internet-drafts/draft-ietf-bmwg-protection-meth-00.txt 5. Milestone Status (Chair) ******* New Work Proposals ********* 6. SIP Performance Benchmarking Discussion GOALS: Review changes to Terminology, review new Methodology. Ask working group if ready to be an official work item. Presenter: Scott Poretsky Related Drafts: draft-poretsky-bmwg-sip-bench-term-01.txt draft-poretsky-bmwg-sip-bench-meth-00.txt 7. Multicast VPN Scalability Benchmarking Discussion GOALS: - Introduce the problem of MVPN scalability benchmarking and need for this draft - Introduce MVPN Metric proposal - Summarize proposed methodology - Request BMWG to take it on as WG work item Statement of proposed work: Multicast VPN (MVPN) is a service deployed by VPN service providers to enable their customers to use IP multicast applications over VPNs. With the increased popularity the scalability of deploying such a service is becoming of a great interest. This document defines standard metric and test methodology for characterizing and comparing control plane MVPN scalability of Provider Edge (PE) devices that implement MVPN service. Presenter: Silvija Dry Related Draft: http://www.ietf.org/internet-drafts/draft-sdry-bmwg-mvpnscale-00.txt 8. Extending the current methodologies to cover wireless LAN switches and wireless LAN meshes TOPIC: WLAN Switch and Mesh Benchmarking GOALS: Provide background Propose specific work items Solicit help towards generating a candidate draft DRAFTS: None PRESENTER: Jerry Perser, Tom Alexander Statement of Proposed Work ------------------------------ Enterprise wireless LANs are now comprised of highly IP-centric devices. Considerable work is being done in various IETF WGs in this area. For example, CAPWAP is defining WLAN switch protocols, and MANET is defining IP over wireless mesh and ad-hoc networks. Modern WLAN switches are Layer 3/4 aware and include many traditional IETF defined functions such as ARP, DHCP and firewalling in combination with wireless functions such as mobility. The lack of industry-standard metrics for benchmarking makes it very difficult for vendors to compare and improve the performance of their devices. We propose that BMWG should take up work in support of these technologies. Metrics are required for roaming and scalability performance (related to CAPWAP) and multi-hop performance and recovery times (related to MANET), as well as general WLAN switch data plane performance. The proposed work item is as follows: - extend existing LAN switch benchmarking terminology and methodology (RFC 1242, RFC 2285 etc.) to wireless LAN switching devices - create new wireless-specific terminology and methodology for mobility, scalability, and mesh networks The IEEE 802.11T Task Group has determined that such work does not fall within their scope, and so there should be no issue with taking it up in BMWG. Scope ------- The scope of the proposed work will cover benchmarking terminology and methodology for measuring the performance of wireless LAN switches and wireless LAN meshes, with particular focus on extending RFC 1242, RFC 2544, RFC 2285 and RFC 2889 to such systems. Benchmarking and performance of 802.11 PHY or link layer functions is a non-objective. Status of existing drafts ------------------------- No drafts currently exist in support of this work. For background on related BMWG wireless discussions, see: http://www3.ietf.org/proceedings/04nov/bmwg.html#cmr Please help contribute to a successful meeting by reading the above I-D(s) and references *before* we meet. To offer comments on BMWG work in progress or the agenda itself, please send email to: bmwg@ietf.org Alternatively, to offer potential agenda items, please email: acmorton@att.com _______________________________________________ bmwg mailing list bmwg@ietf.org https://www1.ietf.org/mailman/listinfo/bmwg _______________________________________________ bmwg mailing list bmwg@ietf.org https://www1.ietf.org/mailman/listinfo/bmwg
- [bmwg] Draft Agenda for bmwg at IETF-67 Al Morton
- RE: [bmwg] Draft Agenda for bmwg at IETF-67 Poretsky, Scott
- RE: [bmwg] Draft Agenda for bmwg at IETF-67 Poretsky, Scott