Re: [bmwg] Alissa Cooper's Discuss on draft-ietf-bmwg-vswitch-opnfv-03: (with DISCUSS and COMMENT)

"MORTON, ALFRED C (AL)" <acmorton@att.com> Thu, 08 June 2017 12:19 UTC

Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 8E5051201F2; Thu, 8 Jun 2017 05:19:52 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -5.401
X-Spam-Level:
X-Spam-Status: No, score=-5.401 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-2.8, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cOtoPsylavev; Thu, 8 Jun 2017 05:19:42 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2F8721293EE; Thu, 8 Jun 2017 05:19:42 -0700 (PDT)
Received: from pps.filterd (m0049463.ppops.net [127.0.0.1]) by m0049463.ppops.net-00191d01. (8.16.0.17/8.16.0.17) with SMTP id v58CEtgA019497; Thu, 8 Jun 2017 08:19:37 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049463.ppops.net-00191d01. with ESMTP id 2axv32prmw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 08 Jun 2017 08:19:36 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v58CJaQF126040; Thu, 8 Jun 2017 07:19:36 -0500
Received: from dalint03.pst.cso.att.com (dalint03.pst.cso.att.com [135.31.133.161]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id v58CJTtO126012 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 8 Jun 2017 07:19:29 -0500
Received: from clpi183.sldc.sbc.com (clpi183.sldc.sbc.com [135.41.1.46]) by dalint03.pst.cso.att.com (RSA Interceptor); Thu, 8 Jun 2017 12:19:21 GMT
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v58CJLOn028957; Thu, 8 Jun 2017 07:19:21 -0500
Received: from mail-blue.research.att.com (mail-blue.research.att.com [135.207.178.11]) by clpi183.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id v58CJAp6028265; Thu, 8 Jun 2017 07:19:10 -0500
Received: from exchange.research.att.com (njmtcas2.research.att.com [135.207.255.47]) by mail-blue.research.att.com (Postfix) with ESMTP id 18A8AF0457; Thu, 8 Jun 2017 08:19:10 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njmtcas2.research.att.com ([fe80::d550:ec84:f872:cad9%15]) with mapi id 14.03.0319.002; Thu, 8 Jun 2017 08:19:09 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: Alissa Cooper <alissa@cooperw.in>
CC: IESG <iesg@ietf.org>, "draft-ietf-bmwg-vswitch-opnfv@ietf.org" <draft-ietf-bmwg-vswitch-opnfv@ietf.org>, Sarah Banks <sbanks@encrypted.net>, "bmwg-chairs@ietf.org" <bmwg-chairs@ietf.org>, "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: Alissa Cooper's Discuss on draft-ietf-bmwg-vswitch-opnfv-03: (with DISCUSS and COMMENT)
Thread-Index: AQHS39hnThUImon/oEeFX43rE6nRmqIZ9L4wgAElqAD//8eUcA==
Date: Thu, 08 Jun 2017 12:19:09 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF25FD70B9@njmtexg5.research.att.com>
References: <149687236248.2644.15874239410315728328.idtracker@ietfa.amsl.com> <4D7F4AD313D3FC43A053B309F97543CF25FD6E1B@njmtexg5.research.att.com> <7CEC6054-B033-4818-B0EE-7A7A1137A2F7@cooperw.in>
In-Reply-To: <7CEC6054-B033-4818-B0EE-7A7A1137A2F7@cooperw.in>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [73.178.187.36]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-RSA-Inspected: yes
X-RSA-Classifications: public
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-08_05:, , signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706080220
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/2jF3U4gB6IDV6E0I1QpfCZ2ITB8>
Subject: Re: [bmwg] Alissa Cooper's Discuss on draft-ietf-bmwg-vswitch-opnfv-03: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 08 Jun 2017 12:19:52 -0000

> -----Original Message-----
> From: Alissa Cooper [mailto:alissa@cooperw.in]
> Sent: Thursday, June 08, 2017 7:37 AM
> To: MORTON, ALFRED C (AL)
> 
> Hi Al,
> 
> > On Jun 7, 2017, at 6:57 PM, MORTON, ALFRED C (AL) <acmorton@att.com>
> wrote:
> >
> > Hi Alissa,
> > Thanks for your review, please see reply below.
> > Al (for the co-authors)
> >
> >> -----Original Message-----
> >> From: Alissa Cooper [mailto:alissa@cooperw.in]
> >> Sent: Wednesday, June 07, 2017 5:53 PM
> >> To: The IESG
> >> Cc: draft-ietf-bmwg-vswitch-opnfv@ietf.org; Sarah Banks; bmwg-
> >> chairs@ietf.org; sbanks@encrypted.net; bmwg@ietf.org
> >> Subject: Alissa Cooper's Discuss on draft-ietf-bmwg-vswitch-opnfv-03:
> >> (with DISCUSS and COMMENT)
> > ...
> >>
> >> ---------------------------------------------------------------------
> -
> >> DISCUSS:
> >> ---------------------------------------------------------------------
> -
> >>
> >> Picking up on the thread that started with the Gen-ART review, I'm not clear on
> >> the position of this document when it comes to repeatability. If all of the
> >> parameters listed in Section 3.3 (assuming they apply) are configured and
> >> documented, is it assumed that the benchmarks will be repeatable and comparable
> >> with hardware implementation benchmarks?
> > [ACM]
> >>>>
> > Of course, it has been the goal of the project to produce
> > repeatable results, and a large set of the parameters
> > believed to be critical was provided so that the benchmarking
> > community could better appreciate the increase in configuration
> > complexity inherent in this work. So, yes, this set was assumed
> > sufficient for the infrastructure in use by the VSPERF project
> > to obtain repeatable results from test-to-test.
> >>>> We could add summary above as a new paragraph in section 3.3.
> >
> > Benchmark Comparability between virtual and physical/hardware
> > implementations of equivalent functions will likely place more
> > detailed and exact requirements on the *testing systems*
> > (in terms of stream generation, algorithms to search
> > for max values, and their configurations of course).
> > This is another area for standardization to appreciate,
> > now that we have a few years testing experience.
> > However, the is a topic for a future draft.
> >
...
> >> I think this needs to be more clear in the document.
> > [ACM]
> > If the >>>paragraph<<< accomplishes that for you,
> > with some minor tweaks to word it less like a reply, let me know.
> 
> I think both of your first two paragraphs above make useful points for
> inclusion in the draft, so if you’re able to write up some text that
> incorporates them, that would resolve my issue.
[ACM] 
OK, the paragraph beginning 
"Benchmark Comparability between virtual and physical/hardware..."
seems to fit best in section 3.1 (Comparison with Physical Network Functions).

Al

> 
> Thanks,
> Alissa
> 
...