Re: [bmwg] Re-Charter Text for review

["Fernando Calabria (fcalabri)" <[e-mail]>]

+1 



On 4/28/14, 9:07 AM, "Barry Constantine" <[e-mail]>
wrote:

>Hi Al,
>
>I think the charter is well written and with topics that are very
>relevant.
>
>Thank you,
>Barry Constantine
>
>JDSU Network and Service Enablement
>Principal Member Technical Staff
>301-325-7069
>
>-----Original Message-----
>From: bmwg [mailto:[e-mail]] On Behalf Of MORTON, ALFRED C
>(AL)
>Sent: Thursday, April 24, 2014 1:21 PM
>To: [e-mail]
>Subject: [bmwg] Re-Charter Text for review
>
>BMWG, 
>
>Over the last year, we've developed this set of paragraphs for specific
>work items one at a time.  Now it's time for one last look (by May 2) and
>then we declare consensus and move to the next step (unless the wheels
>come-off).  Reply with your comments on the text below.
>
>Al and Sarah,
>bmwg co-chairs
>
>-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-==-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>
>Description of Working Group:
>
>
>The Benchmarking Methodology Working Group (BMWG) will continue to
>produce a series of recommendations concerning the key performance
>characteristics of internetworking technologies, or benchmarks for
>network devices, systems, and services. Taking a view of networking
>divided into planes, the scope of work includes benchmarks for the
>management, control, and forwarding planes.
>
>Each recommendation will describe the class of equipment, system, or
>service being addressed; discuss the performance characteristics that are
>pertinent to that class; clearly identify a set of metrics that aid in
>the description of those characteristics; specify the methodologies
>required to collect said metrics; and lastly, present the requirements
>for the common, unambiguous reporting of benchmarking results.
>
>The set of relevant benchmarks will be developed with input from the
>community of users (e.g., network operators and testing organizations)
>and from those affected by the benchmarks when they are published
>(networking and test equipment manufacturers). When possible, the
>benchmarks and other terminology will be developed jointly with
>organizations that are willing to share their expertise. Joint review
>requirements for a specific work area will be included in the detailed
>description of the task, as listed below.
>
>To better distinguish the BMWG from other measurement initiatives in the
>IETF, the scope of the BMWG is limited to the characterization of
>implementations of various internetworking technologies using controlled
>stimuli in a laboratory environment. Said differently, the BMWG does not
>attempt to produce benchmarks for live, operational networks. Moreover,
>the benchmarks produced by this WG shall strive to be vendor independent
>or otherwise have universal applicability to a given technology class.
>
>Because the demands of a particular technology may vary from deployment
>to deployment, a specific non-goal of the Working Group is to define
>acceptance criteria or performance requirements.
>
>An ongoing task is to provide a forum for discussion regarding the
>advancement of measurements designed to provide insight on the
>capabilities and operation of inter-networking technology implementations.
>
>The BMWG will communicate with the operations community through
>organizations such as NANOG, RIPE, and APRICOT.
>
>In addition to its current work items, the BMWG is explicitly tasked to
>develop benchmarks and methodologies for the following technologies:
>
>Traffic Management: Develop the methods to characterize the capacity of
>traffic management features in network devices, such as classification,
>policing, shaping, and active queue management. Existing terminology will
>be used where appropriate. Configured operation will be verified as a
>part of the methodology. The goal is a methodology to assess the maximum
>forwarding performance that a network device can sustain without dropping
>or impairing packets, or compromising the accuracy of multiple instances
>of traffic management functions. This is the benchmark for comparison
>between devices. Another goal is to devise methods that utilize flows
>with congestion-aware transport as part of the traffic load and still
>produce repeatable results in the isolated test environment.
>
>IPv6 Neighbor Discovery: Large address space in IPv6 subnets presents
>several networking challenges, as described in RFC 6583. Indexes to
>describe the performance of network devices, such as the number of
>reachable devices on a sub-network, are useful benchmarks to the
>operations community. The working group will develop the necessary
>terminology and methodologies to measure such benchmarks.
>
>In-Service Software Upgrade: Develop new methods and benchmarks to
>characterize the upgrade of network devices while in-service, considering
>both data and control plane operations and impacts.
>These devices are generally expected to maintain control plane session
>integrity, including routing connections. Quantification of Upgrade
>impact will include packet loss measurement, and other forms of recovery
>behavior will be noted accordingly. The work will produce a definition of
>ISSU, which will help refine the scope. Liaisons will be established as
>needed.
>
>Data Center Benchmarking: This work will define additional terms,
>benchmarks, and methods applicable to data center performance
>evaluations. 
>This includes data center specific congestion scenarios, switch buffer
>analysis, microburst, head of line blocking, while also using a wide mix
>of traffic conditions. Some aspects from BMWG's past work are not
>meaningful when testing switches that implement new IEEE specifications
>in the area of data center bridging. For example, throughput as defined
>in RFC 1242 cannot be measured when testing devices that implement three
>new IEEE specifications: priority-based flow control (802.1Qbb); priority
>groups (802.1Qaz); and congestion notification (802.1Qau).
>This work will update RFC1242, RFC2544, RFC2889 (and other key RFCs), and
>exchange Liaisons with relevant SDOs, especially at WG Last Call.
>
>VNF and related Infrastructure Benchmarking: Benchmarking Methodologies
>have reliably characterized many physical devices. This work item extends
>and enhances the methods to virtual network functions (VNF) and their
>unique supporting infrastructure. First, the new task space will be
>considered to ensure that common issues are recognized from the start.
>Benchmarks for platform capacity and performance characteristics of
>virtual routers, switches, and related components will follow, including
>comparisons between physical and virtual network functions.
>
>_______________________________________________
>bmwg mailing list
>[e-mail]
>https://www.ietf.org/mailman/listinfo/bmwg
>
>_______________________________________________
>bmwg mailing list
>[e-mail]
>https://www.ietf.org/mailman/listinfo/bmwg