Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

"MORTON, ALFRED C (AL)" <acm@research.att.com> Wed, 03 June 2020 16:12 UTC

Return-Path: <acm@research.att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 882B13A079A for <bmwg@ietfa.amsl.com>; Wed, 3 Jun 2020 09:12:42 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.799
X-Spam-Level:
X-Spam-Status: No, score=-1.799 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id u7KSET2Honn7 for <bmwg@ietfa.amsl.com>; Wed, 3 Jun 2020 09:12:41 -0700 (PDT)
Received: from mx0a-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id AFB023A0797 for <bmwg@ietf.org>; Wed, 3 Jun 2020 09:12:40 -0700 (PDT)
Received: from pps.filterd (m0049459.ppops.net [127.0.0.1]) by m0049459.ppops.net-00191d01. (8.16.0.42/8.16.0.42) with SMTP id 053GC5V0035698; Wed, 3 Jun 2020 12:12:39 -0400
Received: from tlpd255.enaf.dadc.sbc.com (sbcsmtp3.sbc.com [144.160.112.28]) by m0049459.ppops.net-00191d01. with ESMTP id 31d98h7vq8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 03 Jun 2020 12:12:39 -0400
Received: from enaf.dadc.sbc.com (localhost [127.0.0.1]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id 053GCc5V083582; Wed, 3 Jun 2020 11:12:39 -0500
Received: from zlp30499.vci.att.com (zlp30499.vci.att.com [135.46.181.149]) by tlpd255.enaf.dadc.sbc.com (8.14.5/8.14.5) with ESMTP id 053GCZ0f083562 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 3 Jun 2020 11:12:35 -0500
Received: from zlp30499.vci.att.com (zlp30499.vci.att.com [127.0.0.1]) by zlp30499.vci.att.com (Service) with ESMTP id B4ED54047691; Wed, 3 Jun 2020 16:12:35 +0000 (GMT)
Received: from clph811.sldc.sbc.com (unknown [135.41.107.12]) by zlp30499.vci.att.com (Service) with ESMTP id 9310B4047690; Wed, 3 Jun 2020 16:12:35 +0000 (GMT)
Received: from sldc.sbc.com (localhost [127.0.0.1]) by clph811.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id 053GCZI8114952; Wed, 3 Jun 2020 11:12:35 -0500
Received: from mail-azure.research.att.com (mail-azure.research.att.com [135.207.255.18]) by clph811.sldc.sbc.com (8.14.5/8.14.5) with ESMTP id 053GCTFe114687; Wed, 3 Jun 2020 11:12:29 -0500
Received: from exchange.research.att.com (njbdcas1.research.att.com [135.197.255.61]) by mail-azure.research.att.com (Postfix) with ESMTPS id BB0EC10A199B; Wed, 3 Jun 2020 12:12:28 -0400 (EDT)
Received: from njmtexg5.research.att.com ([fe80::b09c:ff13:4487:78b6]) by njbdcas1.research.att.com ([fe80::8c6b:4b77:618f:9a01%11]) with mapi id 14.03.0468.000; Wed, 3 Jun 2020 12:12:28 -0400
From: "MORTON, ALFRED C (AL)" <acm@research.att.com>
To: "bmonkman@netsecopen.org" <bmonkman@netsecopen.org>, "'Simon Edwards'" <simon@selabs.uk>
CC: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]
Thread-Index: AdY5hnfAwWz4XwJFTbSITAGUlRwjvAAUhwWAAAXMXCA=
Date: Wed, 3 Jun 2020 16:12:27 +0000
Message-ID: <4D7F4AD313D3FC43A053B309F97543CF0108A5F74A@njmtexg5.research.att.com>
References: <DBBPR09MB30618401FAD3184921486E79B6880@DBBPR09MB3061.eurprd09.prod.outlook.com> <027801d639b7$0d4ac290$27e047b0$@netsecopen.org>
In-Reply-To: <027801d639b7$0d4ac290$27e047b0$@netsecopen.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [69.141.203.172]
Content-Type: multipart/alternative; boundary="_000_4D7F4AD313D3FC43A053B309F97543CF0108A5F74Anjmtexg5resea_"
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216, 18.0.687 definitions=2020-06-03_13:2020-06-02, 2020-06-03 signatures=0
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 spamscore=0 cotscore=-2147483648 bulkscore=0 mlxlogscore=999 adultscore=0 priorityscore=1501 impostorscore=0 malwarescore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 phishscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2004280000 definitions=main-2006030129
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/0TKk8zoWwcgYBzrbWZzosfBCTLc>
Subject: Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 03 Jun 2020 16:12:43 -0000

Thanks for your question, Simon, and the history, Brian.

I confess that this question (why 50%?) has occurred to me in other contexts, and it may help to add a sentence to two of rationale. So if the technical folks can help with suggestions, that would be great!

regards,
Al

From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of bmonkman@netsecopen.org
Sent: Wednesday, June 3, 2020 10:56 AM
To: 'Simon Edwards' <simon@selabs.uk>
Cc: bmwg@ietf.org
Subject: Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

Simon,

This requirement was agreed to and adopted by the working group within NetSecOPEN. It first appeared in the IETF individual draft on October 14, 2018. (https://tools.ietf.org/html/draft-balarajah-bmwg-ngfw-performance-05<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dbalarajah-2Dbmwg-2Dngfw-2Dperformance-2D05&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=mylQC4CnyBjr1JS-Qbiw5d392Llnmp_6LxMRXJO8NVI&s=8ElQf-XUyhkke33v7BvLLf5mEBDCEU2mlwlFDpxHJD8&e=>)BDCEU2mlwlFDpxHJD8&e=>). It was clarified and expanded on March 5, 2019 in https://tools.ietf.org/html/draft-ietf-bmwg-ngfw-performance-00<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Dbmwg-2Dngfw-2Dperformance-2D00&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=mylQC4CnyBjr1JS-Qbiw5d392Llnmp_6LxMRXJO8NVI&s=xbiPb0v60VZvEM2cumbN3J41IrOw2hMUK7HHaR-HN8o&e=>IrOw2hMUK7HHaR-HN8o&e=>. It's form has largely been unchanged since then.

I will leave it to the technical folks to expand on this more. However, I am saying all of this because it has been in the position to be reviewed and commented on by the BMWG community for awhile. Our assumption is that given no one appears to have an issue with it that we hit the mark.

Brian

From: bmwg <bmwg-bounces@ietf.org<mailto:bmwg-bounces@ietf.org>> On Behalf Of Simon Edwards
Sent: June 3, 2020 5:10 AM
To: bmwg@ietf.org<mailto:bmwg@ietf.org>
Subject: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

Hi all,

In a number of sections, but specifically '7.8.3.2.  Test Equipment Configuration Parameters', there are requirements to measure with 50% of the maximum connections/ sec measured in the HTTP/S throughput tests

E.g. "Target objective for scenarios 1 and 2: 50% of the maximum connections per second measured in test scenario..."

I'm sure this 50% value is the product of much thought and discussion, rather than an arbitrary choice. Is anyone able to explain the reason for the specific '50%' value (as opposed to 25%, 75% or whatever) or could you please point to documentation around that decision made by the group?

I'm asking just to understand. I don't disagree with the decision : )

Very best wishes,
Simon