Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

Brian Monkman <bmonkman@netsecopen.org> Wed, 03 June 2020 16:20 UTC

Return-Path: <bmonkman@netsecopen.org>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B14E03A07ED for <bmwg@ietfa.amsl.com>; Wed, 3 Jun 2020 09:20:59 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.798
X-Spam-Level:
X-Spam-Status: No, score=-1.798 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=netsecopen-org.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id g6c14PcjjGct for <bmwg@ietfa.amsl.com>; Wed, 3 Jun 2020 09:20:57 -0700 (PDT)
Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DB8463A0809 for <bmwg@ietf.org>; Wed, 3 Jun 2020 09:20:53 -0700 (PDT)
Received: by mail-qk1-x735.google.com with SMTP id g28so2762616qkl.0 for <bmwg@ietf.org>; Wed, 03 Jun 2020 09:20:53 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netsecopen-org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:thread-topic:thread-index:date:message-id :references:in-reply-to:accept-language:content-language :mime-version; bh=cImIueCP7bo/JuXZoU4Xfrqi1baenGbUq0wLdwB81RU=; b=kFx2Eu7lSNMPJP7Mo4kjZNzDUX964/HiJiQOsnhsu5PQSgdWfL1L0UOpJo6HeNY1jz Xiw1ESDfexHPFyiPn51fwRUkqb5sZu2j5LE2n95oCa6VA1eUnwejx3FdV/Ur1U1bB/1m CTRxfWep80T68VnQA3aRIiWFvmOb0g6ZYA1HbKCjnZ4pYp25A2iBkrz4w5fNNasVT1bs 6NFElODp+i8EyPTZ4TjmOEnU3Z5CGtrYnkRJBX2CTDCwTGZQzffSNmLU5CyPGTsBmeW2 TRsi2Jgh7uvXzAaVjnanMovM41S7Z6hc4xY98BvUrdi+oVV40jYfwkKErr/r1PZhGVN6 KUYA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:thread-topic:thread-index :date:message-id:references:in-reply-to:accept-language :content-language:mime-version; bh=cImIueCP7bo/JuXZoU4Xfrqi1baenGbUq0wLdwB81RU=; b=pax0vBtM8l/HAfT7cjZRTeEoSG2mSQ7pd6FX954lIqqmiLyXSAqSogDCcreCiuUmQY IAQYkrY8HbCvLcjth24kFI4UBN5L2tO/8yNTHCFTsGe2jUwuz70TRpgcBVVad017sKVT TMIGTYp18GW3cbgmpVNnxtni7D+m/4Ejbm/7knPsFWZHCjUwpkkpwG4HFIdT+P6AEZrr vZnYwGzVPCFHV6T5A1hZc/pu67vn69lrAt6l3xKK7JlD+zId1F96ba7x0944Ka14vPp8 nnCa103t7lGPmT/aGuiVA9ftVvq1PBu3wFApMT6p6atDz3pzjQQzvzulHnvtHb0JJvCW 9HjA==
X-Gm-Message-State: AOAM531/A6UkpUxigzbF4FGlapHzzBAigaxGUH0kZr9PY6IRPYjMP7kF FuG9s59RPZv59BRS8jMHF8dohQ==
X-Google-Smtp-Source: ABdhPJyOYDEMiP2sFrHXVXdv/1oUD5O9J35dKfx1AYoJPx+84QeRUlk/MaR+ES1BTD4hJvnTO7gBnA==
X-Received: by 2002:a37:a111:: with SMTP id k17mr436415qke.376.1591201252886; Wed, 03 Jun 2020 09:20:52 -0700 (PDT)
Received: from MN2PR19MB3711.namprd19.prod.outlook.com ([2603:1036:302:40cb::5]) by smtp.gmail.com with ESMTPSA id q24sm2019230qkj.103.2020.06.03.09.20.52 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Jun 2020 09:20:52 -0700 (PDT)
From: Brian Monkman <bmonkman@netsecopen.org>
To: "MORTON, ALFRED C (AL)" <acm@research.att.com>, 'Simon Edwards' <simon@selabs.uk>
CC: "bmwg@ietf.org" <bmwg@ietf.org>
Thread-Topic: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]
Thread-Index: ATBFNDEzFbulDxPH4bKZe8HJJqcDGSRlMiRkQTE5MDMwOTYwMw==
X-MS-Exchange-MessageSentRepresentingType: 1
Date: Wed, 3 Jun 2020 16:20:51 +0000
Message-ID: <MN2PR19MB3711D1A07CC4F368EE7BD9FDFE880@MN2PR19MB3711.namprd19.prod.outlook.com>
References: <DBBPR09MB30618401FAD3184921486E79B6880@DBBPR09MB3061.eurprd09.prod.outlook.com> <027801d639b7$0d4ac290$27e047b0$@netsecopen.org>, <4D7F4AD313D3FC43A053B309F97543CF0108A5F74A@njmtexg5.research.att.com>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF0108A5F74A@njmtexg5.research.att.com>
Accept-Language: en-US
Content-Language: en-CA
X-MS-Has-Attach:
X-MS-Exchange-Organization-SCL: -1
X-MS-TNEF-Correlator:
X-MS-Exchange-Organization-RecordReviewCfmType: 0
Content-Type: multipart/alternative; boundary="_000_MN2PR19MB3711D1A07CC4F368EE7BD9FDFE880MN2PR19MB3711namp_"
MIME-Version: 1.0
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/TWhVC4I8gOiA7BSyuoIYqnpvOBU>
Subject: Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 03 Jun 2020 16:21:00 -0000

Hi Al,

Are there any other items that caused you pause that could be addressed the same way?

Brian

Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: MORTON, ALFRED C (AL) <acm@research.att.com>
Sent: Wednesday, June 3, 2020 12:12:27 PM
To: bmonkman@netsecopen.org <bmonkman@netsecopen.org>rg>; 'Simon Edwards' <simon@selabs.uk>
Cc: bmwg@ietf.org <bmwg@ietf.org>
Subject: RE: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]


Thanks for your question, Simon, and the history, Brian.



I confess that this question (why 50%?) has occurred to me in other contexts, and it may help to add a sentence to two of rationale. So if the technical folks can help with suggestions, that would be great!



regards,

Al



From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of bmonkman@netsecopen.org
Sent: Wednesday, June 3, 2020 10:56 AM
To: 'Simon Edwards' <simon@selabs.uk>
Cc: bmwg@ietf.org
Subject: Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]



Simon,



This requirement was agreed to and adopted by the working group within NetSecOPEN. It first appeared in the IETF individual draft on October 14, 2018. (https://tools.ietf.org/html/draft-balarajah-bmwg-ngfw-performance-05<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dbalarajah-2Dbmwg-2Dngfw-2Dperformance-2D05&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=mylQC4CnyBjr1JS-Qbiw5d392Llnmp_6LxMRXJO8NVI&s=8ElQf-XUyhkke33v7BvLLf5mEBDCEU2mlwlFDpxHJD8&e=>)BDCEU2mlwlFDpxHJD8&e=>). It was clarified and expanded on March 5, 2019 in https://tools.ietf.org/html/draft-ietf-bmwg-ngfw-performance-00<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Dbmwg-2Dngfw-2Dperformance-2D00&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=mylQC4CnyBjr1JS-Qbiw5d392Llnmp_6LxMRXJO8NVI&s=xbiPb0v60VZvEM2cumbN3J41IrOw2hMUK7HHaR-HN8o&e=>IrOw2hMUK7HHaR-HN8o&e=>. It’s form has largely been unchanged since then.



I will leave it to the technical folks to expand on this more. However, I am saying all of this because it has been in the position to be reviewed and commented on by the BMWG community for awhile. Our assumption is that given no one appears to have an issue with it that we hit the mark.



Brian



From: bmwg <bmwg-bounces@ietf.org<mailto:bmwg-bounces@ietf.org>> On Behalf Of Simon Edwards
Sent: June 3, 2020 5:10 AM
To: bmwg@ietf.org<mailto:bmwg@ietf.org>
Subject: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]



Hi all,



In a number of sections, but specifically '7.8.3.2.  Test Equipment Configuration Parameters', there are requirements to measure with 50% of the maximum connections/ sec measured in the HTTP/S throughput tests



E.g. "Target objective for scenarios 1 and 2: 50% of the maximum connections per second measured in test scenario..."



I'm sure this 50% value is the product of much thought and discussion, rather than an arbitrary choice. Is anyone able to explain the reason for the specific '50%' value (as opposed to 25%, 75% or whatever) or could you please point to documentation around that decision made by the group?



I'm asking just to understand. I don't disagree with the decision : )



Very best wishes,

Simon