Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

"MORTON, ALFRED C (AL)" <acm@research.att.com> Wed, 03 June 2020 18:52 UTC

Return-Path: <acm@research.att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 99D413A0FAC for <bmwg@ietfa.amsl.com>; Wed, 3 Jun 2020 11:52:11 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.796
X-Spam-Level:
X-Spam-Status: No, score=-1.796 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gPWLJkKBjLl3 for <bmwg@ietfa.amsl.com>; Wed, 3 Jun 2020 11:52:10 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0a-00191d01.pphosted.com [67.231.149.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EE8CF3A0E37 for <bmwg@ietf.org>; Wed, 3 Jun 2020 11:52:00 -0700 (PDT)
Received: from pps.filterd (m0049297.ppops.net [127.0.0.1]) by m0049297.ppops.net-00191d01. (8.16.0.42/8.16.0.42) with SMTP id 053Iq0Eu012833; Wed, 3 Jun 2020 14:52:00 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049297.ppops.net-00191d01. with ESMTP id 31eew7kufx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 03 Jun 2020 14:51:59 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id 053IpoXu001429; Wed, 3 Jun 2020 13:51:50 -0500
Received: from zlp30495.vci.att.com (zlp30495.vci.att.com [135.46.181.158]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id 053Ipj8H001348 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 3 Jun 2020 13:51:45 -0500
Received: from zlp30495.vci.att.com (zlp30495.vci.att.com [127.0.0.1]) by zlp30495.vci.att.com (Service) with ESMTP id 48586404B590; Wed, 3 Jun 2020 18:51:45 +0000 (GMT)
Received: from clph811.sldc.sbc.com (unknown [135.41.107.12]) by zlp30495.vci.att.com (Service) with ESMTP id 2506D404B58C; Wed, 3 Jun 2020 18:51:45 +0000 (GMT)
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clph811.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id 053Ipirh100728; Wed, 3 Jun 2020 13:51:44 -0500
Received: from mail-blue.research.att.com (mail-blue.research.att.com [135.207.178.11]) by clph811.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id 053IpacL099615; Wed, 3 Jun 2020 13:51:36 -0500
Received: from exchange.research.att.com (njbdcas1.research.att.com [135.197.255.61]) by mail-blue.research.att.com (Postfix) with ESMTPS id E42F510A3023; Wed, 3 Jun 2020 14:51:35 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njbdcas1.research.att.com ([fe80::8c6b:4b77:618f:9a01%11]) with mapi id 14.03.0468.000; Wed, 3 Jun 2020 14:51:35 -0400
From: "MORTON, ALFRED C (AL)" <acm@research.att.com>
To: "bmonkman@netsecopen.org" <bmonkman@netsecopen.org>, "'Simon Edwards'" <simon@selabs.uk>
CC: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]
Thread-Index: AdY5hnfAwWz4XwJFTbSITAGUlRwjvAAUhwWAAAXMXCD//+logIAAN15g///dQYCAAC7E8A==
Date: Wed, 3 Jun 2020 18:51:34 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF0108A5F8BD@njmtexg5.research.att.com>
References: <DBBPR09MB30618401FAD3184921486E79B6880@DBBPR09MB3061.eurprd09.prod.outlook.com> <027801d639b7$0d4ac290$27e047b0$@netsecopen.org>, <4D7F4AD313D3FC43A053B309F97543CF0108A5F74A@njmtexg5.research.att.com> <MN2PR19MB3711D1A07CC4F368EE7BD9FDFE880@MN2PR19MB3711.namprd19.prod.outlook.com> <4D7F4AD313D3FC43A053B309F97543CF0108A5F7C6@njmtexg5.research.att.com> <029a01d639cd$427aa0a0$c76fe1e0$@netsecopen.org>
In-Reply-To: <029a01d639cd$427aa0a0$c76fe1e0$@netsecopen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [69.141.203.172]
Content-Type: multipart/alternative; boundary="_000_4D7F4AD313D3FC43A053B309F97543CF0108A5F8BDnjmtexg5resea_"
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216, 18.0.687 definitions=2020-06-03_13:2020-06-02, 2020-06-03 signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 lowpriorityscore=0 cotscore=-2147483648 impostorscore=0 adultscore=0 priorityscore=1501 mlxscore=0 phishscore=0 bulkscore=0 clxscore=1015 suspectscore=0 mlxlogscore=999 spamscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2004280000 definitions=main-2006030146
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/CBF-LFRWxNC39A-OXMZ2GwGOhhE>
Subject: Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 03 Jun 2020 18:52:17 -0000

It’s beginning to look like I won’t complete a review this week, and I have already promised to review a doc over the weekend before a Monday deadline. So, bottom line is that I’m with you, hoping others weigh-in with reviews to help!

Al

From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of bmonkman@netsecopen.org
Sent: Wednesday, June 3, 2020 1:35 PM
To: MORTON, ALFRED C (AL) <acm@research.att.com>om>; 'Simon Edwards' <simon@selabs.uk>
Cc: bmwg@ietf.org
Subject: Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

It doesn’t hurt either. 😊

Should we hold off until you complete your review before we submit a new draft? I am hoping others weigh in on this as well.

Brian

From: MORTON, ALFRED C (AL) <acm@research.att.com<mailto:acm@research.att.com>>
Sent: June 3, 2020 1:04 PM
To: Brian Monkman <bmonkman@netsecopen.org<mailto:bmonkman@netsecopen.org>>; 'Simon Edwards' <simon@selabs.uk<mailto:simon@selabs.uk>>
Cc: bmwg@ietf.org<mailto:bmwg@ietf.org>
Subject: RE: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

Hi Brian,  no other pauses or speed-bumps yet, but I need to complete my
review of the most recent version.

sorry, I know that doesn’t help!
Al

From: Brian Monkman [mailto:bmonkman@netsecopen.org]
Sent: Wednesday, June 3, 2020 12:21 PM
To: MORTON, ALFRED C (AL) <acm@research.att.com<mailto:acm@research.att.com>>; 'Simon Edwards' <simon@selabs.uk<mailto:simon@selabs.uk>>
Cc: bmwg@ietf.org<mailto:bmwg@ietf.org>
Subject: Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

Hi Al,

Are there any other items that caused you pause that could be addressed the same way?

Brian

Get Outlook for iOS<https://urldefense.proofpoint.com/v2/url?u=https-3A__aka.ms_o0ukef&d=DwMF-g&c=LFYZ-o9_HUMeMTSQicvjIg&r=_6cen3Hn-e_hOm0BhY7aIpA58dd19Z9qGQsr8-6zYMI&m=gwfQJfXOl1ZubaA9a-aKfQ8n5AY2sKsb3Ml9NAklRwQ&s=ai2z_d2vwJeZx_EibZkLkoE_ZWS26ZSoAJq9RZNf6bM&e=>
________________________________
From: MORTON, ALFRED C (AL) <acm@research.att.com<mailto:acm@research.att.com>>
Sent: Wednesday, June 3, 2020 12:12:27 PM
To: bmonkman@netsecopen.org<mailto:bmonkman@netsecopen.org> <bmonkman@netsecopen.org<mailto:bmonkman@netsecopen.org>>; 'Simon Edwards' <simon@selabs.uk<mailto:simon@selabs.uk>>
Cc: bmwg@ietf.org<mailto:bmwg@ietf.org> <bmwg@ietf.org<mailto:bmwg@ietf.org>>
Subject: RE: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]


Thanks for your question, Simon, and the history, Brian.



I confess that this question (why 50%?) has occurred to me in other contexts, and it may help to add a sentence to two of rationale. So if the technical folks can help with suggestions, that would be great!



regards,

Al



From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of bmonkman@netsecopen.org<mailto:bmonkman@netsecopen.org>
Sent: Wednesday, June 3, 2020 10:56 AM
To: 'Simon Edwards' <simon@selabs.uk<mailto:simon@selabs.uk>>
Cc: bmwg@ietf.org<mailto:bmwg@ietf.org>
Subject: Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]



Simon,



This requirement was agreed to and adopted by the working group within NetSecOPEN. It first appeared in the IETF individual draft on October 14, 2018. (https://tools.ietf.org/html/draft-balarajah-bmwg-ngfw-performance-05<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf..org_html_draft-2Dbalarajah-2Dbmwg-2Dngfw-2Dperformance-2D05&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=mylQC4CnyBjr1JS-Qbiw5d392Llnmp_6LxMRXJO8NVI&s=8ElQf-XUyhkke33v7BvLLf5mEBDCEU2mlwlFDpxHJD8&e=>)BDCEU2mlwlFDpxHJD8&e=>). It was clarified and expanded on March 5, 2019 in https://tools.ietf.org/html/draft-ietf-bmwg-ngfw-performance-00<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf..org_html_draft-2Dietf-2Dbmwg-2Dngfw-2Dperformance-2D00&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=mylQC4CnyBjr1JS-Qbiw5d392Llnmp_6LxMRXJO8NVI&s=xbiPb0v60VZvEM2cumbN3J41IrOw2hMUK7HHaR-HN8o&e=>IrOw2hMUK7HHaR-HN8o&e=>. It’s form has largely been unchanged since then.



I will leave it to the technical folks to expand on this more. However, I am saying all of this because it has been in the position to be reviewed and commented on by the BMWG community for awhile. Our assumption is that given no one appears to have an issue with it that we hit the mark.



Brian



From: bmwg <bmwg-bounces@ietf.org<mailto:bmwg-bounces@ietf.org>> On Behalf Of Simon Edwards
Sent: June 3, 2020 5:10 AM
To: bmwg@ietf.org<mailto:bmwg@ietf.org>
Subject: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]



Hi all,



In a number of sections, but specifically '7.8.3.2.  Test Equipment Configuration Parameters', there are requirements to measure with 50% of the maximum connections/ sec measured in the HTTP/S throughput tests



E.g. "Target objective for scenarios 1 and 2: 50% of the maximum connections per second measured in test scenario..."



I'm sure this 50% value is the product of much thought and discussion, rather than an arbitrary choice. Is anyone able to explain the reason for the specific '50%' value (as opposed to 25%, 75% or whatever) or could you please point to documentation around that decision made by the group?



I'm asking just to understand. I don't disagree with the decision : )



Very best wishes,

Simon