Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

"MORTON, ALFRED C (AL)" <acm@research.att.com> Wed, 03 June 2020 17:04 UTC

Return-Path: <acm@research.att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id E478E3A0B04 for <bmwg@ietfa.amsl.com>; Wed, 3 Jun 2020 10:04:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.799
X-Spam-Level:
X-Spam-Status: No, score=-1.799 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zYdsEa-2F380 for <bmwg@ietfa.amsl.com>; Wed, 3 Jun 2020 10:04:41 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id EB8BD3A0B03 for <bmwg@ietf.org>; Wed, 3 Jun 2020 10:04:40 -0700 (PDT)
Received: from pps.filterd (m0049463.ppops.net [127.0.0.1]) by m0049463.ppops.net-00191d01. (8.16.0.42/8.16.0.42) with SMTP id 053GqBSs042736; Wed, 3 Jun 2020 13:04:40 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049463.ppops.net-00191d01. with ESMTP id 31e7ttsm6a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 03 Jun 2020 13:04:39 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id 053H4c1i062808; Wed, 3 Jun 2020 12:04:39 -0500
Received: from zlp30493.vci.att.com (zlp30493.vci.att.com [135.46.181.176]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id 053H4aFh062767 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 3 Jun 2020 12:04:37 -0500
Received: from zlp30493.vci.att.com (zlp30493.vci.att.com [127.0.0.1]) by zlp30493.vci.att.com (Service) with ESMTP id D5F8C400AF93; Wed, 3 Jun 2020 17:04:36 +0000 (GMT)
Received: from clph811.sldc.sbc.com (unknown [135.41.107.12]) by zlp30493.vci.att.com (Service) with ESMTP id AEB1C400AF73; Wed, 3 Jun 2020 17:04:36 +0000 (GMT)
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clph811.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id 053H4aW0002230; Wed, 3 Jun 2020 12:04:36 -0500
Received: from mail-green.research.att.com (mail-green.research.att.com [135.207.255.15]) by clph811.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id 053H4UBM001605; Wed, 3 Jun 2020 12:04:30 -0500
Received: from exchange.research.att.com (njbdcas1.research.att.com [135.197.255.61]) by mail-green.research.att.com (Postfix) with ESMTPS id 6C22E10A1907; Wed, 3 Jun 2020 13:04:29 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njbdcas1.research.att.com ([fe80::8c6b:4b77:618f:9a01%11]) with mapi id 14.03.0468.000; Wed, 3 Jun 2020 13:04:29 -0400
From: "MORTON, ALFRED C (AL)" <acm@research.att.com>
To: Brian Monkman <bmonkman@netsecopen.org>, "'Simon Edwards'" <simon@selabs.uk>
CC: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]
Thread-Index: AdY5hnfAwWz4XwJFTbSITAGUlRwjvAAUhwWAAAXMXCD//+logIAAN15g
Date: Wed, 3 Jun 2020 17:04:27 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF0108A5F7C6@njmtexg5.research.att.com>
References: <DBBPR09MB30618401FAD3184921486E79B6880@DBBPR09MB3061.eurprd09.prod.outlook.com> <027801d639b7$0d4ac290$27e047b0$@netsecopen.org>, <4D7F4AD313D3FC43A053B309F97543CF0108A5F74A@njmtexg5.research.att.com> <MN2PR19MB3711D1A07CC4F368EE7BD9FDFE880@MN2PR19MB3711.namprd19.prod.outlook.com>
In-Reply-To: <MN2PR19MB3711D1A07CC4F368EE7BD9FDFE880@MN2PR19MB3711.namprd19.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [69.141.203.172]
Content-Type: multipart/alternative; boundary="_000_4D7F4AD313D3FC43A053B309F97543CF0108A5F7C6njmtexg5resea_"
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216, 18.0.687 definitions=2020-06-03_13:2020-06-02, 2020-06-03 signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 spamscore=0 bulkscore=0 suspectscore=0 malwarescore=0 adultscore=0 cotscore=-2147483648 priorityscore=1501 lowpriorityscore=0 phishscore=0 clxscore=1015 impostorscore=0 mlxscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2004280000 definitions=main-2006030133
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/qQw3dmWamXN-4VFuU6fkBGr552M>
Subject: Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 03 Jun 2020 17:04:43 -0000

Hi Brian,  no other pauses or speed-bumps yet, but I need to complete my
review of the most recent version.

sorry, I know that doesn't help!
Al

From: Brian Monkman [mailto:bmonkman@netsecopen.org]
Sent: Wednesday, June 3, 2020 12:21 PM
To: MORTON, ALFRED C (AL) <acm@research.att.com>om>; 'Simon Edwards' <simon@selabs.uk>
Cc: bmwg@ietf.org
Subject: Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

Hi Al,

Are there any other items that caused you pause that could be addressed the same way?

Brian

Get Outlook for iOS<https://urldefense.proofpoint.com/v2/url?u=https-3A__aka.ms_o0ukef&d=DwMF-g&c=LFYZ-o9_HUMeMTSQicvjIg&r=_6cen3Hn-e_hOm0BhY7aIpA58dd19Z9qGQsr8-6zYMI&m=gwfQJfXOl1ZubaA9a-aKfQ8n5AY2sKsb3Ml9NAklRwQ&s=ai2z_d2vwJeZx_EibZkLkoE_ZWS26ZSoAJq9RZNf6bM&e=>
________________________________
From: MORTON, ALFRED C (AL) <acm@research.att.com<mailto:acm@research.att.com>>
Sent: Wednesday, June 3, 2020 12:12:27 PM
To: bmonkman@netsecopen.org<mailto:bmonkman@netsecopen.org> <bmonkman@netsecopen.org<mailto:bmonkman@netsecopen.org>>; 'Simon Edwards' <simon@selabs.uk<mailto:simon@selabs.uk>>
Cc: bmwg@ietf.org<mailto:bmwg@ietf.org> <bmwg@ietf.org<mailto:bmwg@ietf.org>>
Subject: RE: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]


Thanks for your question, Simon, and the history, Brian.



I confess that this question (why 50%?) has occurred to me in other contexts, and it may help to add a sentence to two of rationale. So if the technical folks can help with suggestions, that would be great!



regards,

Al



From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of bmonkman@netsecopen.org<mailto:bmonkman@netsecopen.org>
Sent: Wednesday, June 3, 2020 10:56 AM
To: 'Simon Edwards' <simon@selabs.uk<mailto:simon@selabs.uk>>
Cc: bmwg@ietf.org<mailto:bmwg@ietf.org>
Subject: Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]



Simon,



This requirement was agreed to and adopted by the working group within NetSecOPEN. It first appeared in the IETF individual draft on October 14, 2018. (https://tools.ietf.org/html/draft-balarajah-bmwg-ngfw-performance-05<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dbalarajah-2Dbmwg-2Dngfw-2Dperformance-2D05&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=mylQC4CnyBjr1JS-Qbiw5d392Llnmp_6LxMRXJO8NVI&s=8ElQf-XUyhkke33v7BvLLf5mEBDCEU2mlwlFDpxHJD8&e=>)BDCEU2mlwlFDpxHJD8&e=>). It was clarified and expanded on March 5, 2019 in https://tools.ietf.org/html/draft-ietf-bmwg-ngfw-performance-00<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Dbmwg-2Dngfw-2Dperformance-2D00&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=mylQC4CnyBjr1JS-Qbiw5d392Llnmp_6LxMRXJO8NVI&s=xbiPb0v60VZvEM2cumbN3J41IrOw2hMUK7HHaR-HN8o&e=>IrOw2hMUK7HHaR-HN8o&e=>. It's form has largely been unchanged since then.



I will leave it to the technical folks to expand on this more. However, I am saying all of this because it has been in the position to be reviewed and commented on by the BMWG community for awhile. Our assumption is that given no one appears to have an issue with it that we hit the mark.



Brian



From: bmwg <bmwg-bounces@ietf.org<mailto:bmwg-bounces@ietf.org>> On Behalf Of Simon Edwards
Sent: June 3, 2020 5:10 AM
To: bmwg@ietf.org<mailto:bmwg@ietf.org>
Subject: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]



Hi all,



In a number of sections, but specifically '7.8.3.2.  Test Equipment Configuration Parameters', there are requirements to measure with 50% of the maximum connections/ sec measured in the HTTP/S throughput tests



E.g. "Target objective for scenarios 1 and 2: 50% of the maximum connections per second measured in test scenario..."



I'm sure this 50% value is the product of much thought and discussion, rather than an arbitrary choice. Is anyone able to explain the reason for the specific '50%' value (as opposed to 25%, 75% or whatever) or could you please point to documentation around that decision made by the group?



I'm asking just to understand. I don't disagree with the decision : )



Very best wishes,

Simon