Re: [bmwg] Revised Draft Text for Re-Chartering

"MORTON, ALFRED C (AL)" <acmorton@att.com> Tue, 03 April 2018 17:56 UTC

Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 51E8512EA76 for <bmwg@ietfa.amsl.com>; Tue, 3 Apr 2018 10:56:56 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.601
X-Spam-Level:
X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WBy9MTbbm1CX for <bmwg@ietfa.amsl.com>; Tue, 3 Apr 2018 10:56:52 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D3D931243F6 for <bmwg@ietf.org>; Tue, 3 Apr 2018 10:56:52 -0700 (PDT)
Received: from pps.filterd (m0048589.ppops.net [127.0.0.1]) by m0048589.ppops.net-00191d01. (8.16.0.21/8.16.0.21) with SMTP id w33Huptk036063; Tue, 3 Apr 2018 13:56:52 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0048589.ppops.net-00191d01. with ESMTP id 2h4e9kg9a2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 03 Apr 2018 13:56:51 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id w33HukQu018774; Tue, 3 Apr 2018 12:56:47 -0500
Received: from zlp30493.vci.att.com (zlp30493.vci.att.com [135.46.181.176]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id w33Huh9L018691; Tue, 3 Apr 2018 12:56:43 -0500
Received: from zlp30493.vci.att.com (zlp30493.vci.att.com [127.0.0.1]) by zlp30493.vci.att.com (Service) with ESMTP id C280640004A7; Tue, 3 Apr 2018 17:56:43 +0000 (GMT)
Received: from tlpd252.dadc.sbc.com (unknown [135.31.184.157]) by zlp30493.vci.att.com (Service) with ESMTP id 9E40A4000486; Tue, 3 Apr 2018 17:56:43 +0000 (GMT)
Received: from dadc.sbc.com (localhost [127.0.0.1]) by tlpd252.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id w33Huh7R064353; Tue, 3 Apr 2018 12:56:43 -0500
Received: from mail-blue.research.att.com (mail-blue.research.att.com [135.207.178.11]) by tlpd252.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id w33HucQV064066; Tue, 3 Apr 2018 12:56:38 -0500
Received: from exchange.research.att.com (njbdcas1.research.att.com [135.197.255.61]) by mail-blue.research.att.com (Postfix) with ESMTP id 9B500F0A2E; Tue, 3 Apr 2018 13:56:37 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njbdcas1.research.att.com ([fe80::78d3:43d5:1186:18f5%15]) with mapi id 14.03.0389.001; Tue, 3 Apr 2018 13:56:36 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "bmwg@ietf.org" <bmwg@ietf.org>, Warren Kumari <warren@kumari.net>
Thread-Topic: Revised Draft Text for Re-Chartering
Thread-Index: AdOxb2rKXpi1VX+tRZuJhTfFaQDwzgP2TelAAor2kJA=
Date: Tue, 3 Apr 2018 17:56:35 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF4A8E1A92@njmtexg5.research.att.com>
References: <4D7F4AD313D3FC43A053B309F97543CF490BD78B@njmtexg5.research.att.com> <4D7F4AD313D3FC43A053B309F97543CF490F60C8@njmtexg5.research.att.com>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF490F60C8@njmtexg5.research.att.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
x-originating-ip: [135.16.251.116]
Content-Type: multipart/mixed; boundary="_002_4D7F4AD313D3FC43A053B309F97543CF4A8E1A92njmtexg5researc_"
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-04-03_08:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1711220000 definitions=main-1804030184
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/7zyvU8wiCKjf6FNqa-g1qQ2Y9eY>
Subject: Re: [bmwg] Revised Draft Text for Re-Chartering
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 03 Apr 2018 17:57:00 -0000

Hi Warren,

The comment week closed on the 21st, and there were
no comments in overtime :-) so we can comfortably declare 
WG consensus on the revised text and milestones and
pass to you for AD review and Last Call/IESG.

<clean text file attached>

Thanks,
Al
bmwg co-chair

> -----Original Message-----
> From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of MORTON, ALFRED C
> (AL)
> Sent: Wednesday, March 21, 2018 3:41 PM
> To: bmwg@ietf.org
> Subject: Re: [bmwg] Revised Draft Text for Re-Chartering
> 
> *** Security Advisory: This Message Originated Outside of AT&T ***.
> Reference http://cso.att.com/EmailSecurity/IDSP.html for more information.
> 
> BMWG,
> 
> We have slightly revised the Proposed Milestones as part of
> Re-Charter Review in our IETF-101 session.
> 
> Revised proposal is below.
> 
> Final Comment period is one week, by March 21st, 2018, to the
> bmwg-list please.
> 
> regards,
> Al
> bmwg co-chair
> 
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> BMWG - Benchmarking Methodology Working Group  (as of March 20, 2018)
> 
> Description of Working Group:
> 
>   The Benchmarking Methodology Working Group (BMWG) will continue to
>   produce a series of recommendations concerning the key performance
>   characteristics of internetworking technologies, or benchmarks for
>   network devices, systems, and services. Taking a view of networking
>   divided into planes, the scope of work includes benchmarks for the
>   management, control, and forwarding planes.
> 
>   The scope of BMWG has been extended to develop methods for
>   virtual network functions (VNF) and their unique supporting
>   infrastructure (such as SDN Controllers and vSwitches).
>   Benchmarks for platform capacity and performance characteristics
>   of virtual routers, firewalls (and other security functions),
>   signaling control gateways, and other forms of gateways are included.
>   The benchmarks will foster comparisons between physical and virtual
>   network functions, and also cover unique features of
>   Network Function Virtualization systems. Also, with the emergence
>   of virtualized test systems, specifications for test system
>   calibration are also in-scope.
> 
>   Each recommendation will describe the class of network function, system,
>   or service being addressed; discuss the performance characteristics that
>   are pertinent to that class; clearly identify a set of metrics that aid
>   in the description of those characteristics; specify the methodologies
>   required to collect said metrics; and lastly, present the requirements
>   for the common, unambiguous reporting of benchmarking results.
> 
>   The set of relevant benchmarks will be developed with input from the
>   community of users (e.g., network operators and testing organizations)
>   and from those affected by the benchmarks when they are published
>   (networking and test equipment manufacturers). When possible, the
>   benchmarks and other terminologies will be developed jointly with
>   organizations that are willing to share their expertise. Joint review
>   requirements for a specific work area will be included in the
>   description of the task.
> 
>   To better distinguish the BMWG from other measurement initiatives in the
>   IETF, the scope of the BMWG is limited to the characterization of
>   implementations of various internetworking technologies
>   using controlled stimuli in a laboratory environment. Said differently,
>   the BMWG does not attempt to produce benchmarks for live, operational
>   networks. Moreover, the benchmarks produced by this WG shall strive to
>   be vendor independent or otherwise have universal applicability to a
>   given technology class.
> 
>   Because the demands of a particular technology may vary from deployment
>   to deployment, a specific non-goal of the Working Group is to define
>   acceptance criteria or performance requirements.
> 
>   An ongoing task is to provide a forum for development and
>   advancement of measurements which provide insight on the
>   capabilities and operation of implementations of inter-networking
>   technology. Ideally, BMWG should communicate with the operations
>   community through organizations such as NANOG, RIPE, and APRICOT.
> 
> 
> Goals and Milestones: (preliminary list, we have an active draft for all
> topics)
> 
>   Aug 2018 - Methodology for Next-Gen Firewall Benchmarking to IESG Review
>   Dec 2018 - Update to RFC2544 Back-to-back Frame Benchmarking to IESG
> Review
>   Dec 2019 - Methodology for EVPN Benchmarking to IESG Review
>   Dec 2019 - Considerations for Benchmarking Network Virtualization
> Platforms
>              to IESG Review
>   Dec 2019 - Draft on Network Service Layer Abstract Models applied to
>              Benchmarking to IESG Review
>   Dec 2019 - Draft on General VNF Benchmarking Automation to IESG Review
> 
> 
> > -----Original Message-----
> > From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of MORTON, ALFRED C
> > (AL)
> > Sent: Thursday, March 1, 2018 10:13 AM
> > To: bmwg@ietf.org
> > Subject: [bmwg] Revised Draft Text for Re-Chartering
> >
> > *** Security Advisory: This Message Originated Outside of AT&T ***.
> > Reference http://cso.att.com/EmailSecurity/IDSP.html for more
> information.
> >
> > BMWG,
> >
> > We have slightly revised text to review during our
> > Virtual Interim Meeting today (the first draft text
> > was sent to bmwg-list in October, 2017).
> > I'll summarize the changes during the meeting.
> >
> > regards,
> > Al
> > bmwg-co-chair
> >
> > -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> >
> > BMWG - Benchmarking Methodology Working Group
> >
> > Description of Working Group:
> >
> >   The Benchmarking Methodology Working Group (BMWG) will continue to
> >   produce a series of recommendations concerning the key performance
> >   characteristics of internetworking technologies, or benchmarks for
> >   network devices, systems, and services. Taking a view of networking
> >   divided into planes, the scope of work includes benchmarks for the
> >   management, control, and forwarding planes.
> >
> >   The scope of BMWG has been extended to develop methods for
> >   virtual network functions (VNF) and their unique supporting
> >   infrastructure (such as SDN Controllers and vSwitches).
> >   Benchmarks for platform capacity and performance characteristics
> >   of virtual routers, firewalls (and other security functions),
> >   signaling control gateways, and other forms of gateways are included.
> >   The benchmarks will foster comparisons between physical and virtual
> >   network functions, and also cover unique features of
> >   Network Function Virtualization systems. Also, with the emergence
> >   of virtualized test systems, specifications for test system
> >   calibration are also in-scope.
> >
> >   Each recommendation will describe the class of network function,
> system,
> >   or service being addressed; discuss the performance characteristics
> that
> >   are pertinent to that class; clearly identify a set of metrics that
> aid
> >   in the description of those characteristics; specify the methodologies
> >   required to collect said metrics; and lastly, present the requirements
> >   for the common, unambiguous reporting of benchmarking results.
> >
> >   The set of relevant benchmarks will be developed with input from the
> >   community of users (e.g., network operators and testing organizations)
> >   and from those affected by the benchmarks when they are published
> >   (networking and test equipment manufacturers). When possible, the
> >   benchmarks and other terminologies will be developed jointly with
> >   organizations that are willing to share their expertise. Joint review
> >   requirements for a specific work area will be included in the
> >   description of the task.
> >
> >   To better distinguish the BMWG from other measurement initiatives in
> the
> >   IETF, the scope of the BMWG is limited to the characterization of
> >   implementations of various internetworking technologies
> >   using controlled stimuli in a laboratory environment. Said
> differently,
> >   the BMWG does not attempt to produce benchmarks for live, operational
> >   networks. Moreover, the benchmarks produced by this WG shall strive to
> >   be vendor independent or otherwise have universal applicability to a
> >   given technology class.
> >
> >   Because the demands of a particular technology may vary from
> deployment
> >   to deployment, a specific non-goal of the Working Group is to define
> >   acceptance criteria or performance requirements.
> >
> >   An ongoing task is to provide a forum for development and
> >   advancement of measurements which provide insight on the
> >   capabilities and operation of implementations of inter-networking
> >   technology. Ideally, BMWG should communicate with the operations
> >   community through organizations such as NANOG, RIPE, and APRICOT.
> >
> >
> > Goals and Milestones: (preliminary list, we have an active draft for all
> > topics)
> >
> >   Dec 2018 - Methodology for Next-Gen Firewall Benchmarking to IESG
> Review
> >   Dec 2018 - Update to RFC2544 Back-to-back Frame Benchmarking to IESG
> > Review
> >   Dec 2019 - Methodology for EVPN Benchmarking to IESG Review
> >   Dec 2019 - Considerations for Benchmarking Network Virtualization
> > Platforms
> >              to IESG Review
> >   Dec 2019 - Draft on Network Service Layer Abstract Models applied to
> >              Benchmarking to IESG Review
> >   Dec 2019 - Draft on Automated VNF Benchmarking to IESG Review
> >
> > _______________________________________________
> > bmwg mailing list
> > bmwg@ietf.org
> > https://urldefense.proofpoint.com/v2/url?u=https-
> > 3A__www.ietf.org_mailman_listinfo_bmwg&d=DwICAg&c=LFYZ-
> >
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=lB6OG6NJlVPcNq_hFaAplcHHWP5AQ
> > v2kMcsxYDTMFDo&s=xE6icgQqHQn-lu-aS00Fu45JJF6kvx92rmWZFwP1XFM&e=
> 
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.ietf.org_mailman_listinfo_bmwg&d=DwICAg&c=LFYZ-
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=C2slT6LxJ9UfLkLuYslijFwaU0UXD
> 4RGtuk9YWX1P9s&s=jfXw9LmZxlYJ12OfFN-txwQeLTKJVguCt4E24TqI8bs&e=