Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

bmonkman@netsecopen.org Wed, 03 June 2020 17:34 UTC

Return-Path: <bmonkman@netsecopen.org>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D92FF3A0C67 for <bmwg@ietfa.amsl.com>; Wed, 3 Jun 2020 10:34:46 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.797
X-Spam-Level:
X-Spam-Status: No, score=-1.797 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, RCVD_IN_DNSWL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=netsecopen-org.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XFCvGJWGIjZB for <bmwg@ietfa.amsl.com>; Wed, 3 Jun 2020 10:34:44 -0700 (PDT)
Received: from mail-qk1-x72f.google.com (mail-qk1-x72f.google.com [IPv6:2607:f8b0:4864:20::72f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B18033A0C63 for <bmwg@ietf.org>; Wed, 3 Jun 2020 10:34:44 -0700 (PDT)
Received: by mail-qk1-x72f.google.com with SMTP id n11so3006875qkn.8 for <bmwg@ietf.org>; Wed, 03 Jun 2020 10:34:44 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netsecopen-org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:thread-index:content-language; bh=NmbjpOdpmDTE9hZr49HYvNXD3rWvGESKCnmHRYlEJV4=; b=UKyeHWmwanx0EtMSzC7FFap+yk9I8oT0XZVJuIY30ZFI2uyGMiVK0SY5NoL2xpxbnA 2uwVTgDnSpk6TSDNw/JUF+DjuZQ1h8+ahGYY4G/45ztdHSJNR79yyMWLZqHSdHA4vQlH nmLFGrMtW78Kf3onNCCXDwsk31NoWMfyKgrIvsdtC6NNi6HKzclcOwW6lJ2N0b/AmK+t J2yUXHmY1yqhM7y/WdBZ2AWkYqgewHY2Jr5hO5C4jq7yn5kFeIyfyUrNwFwwiQDE2p2B zteF7fd6AlsfNWdcInwpstitLByZVqaFR5bDcA1p6R39TBxNr2zCy6kLDm4xytbgw9fQ Pfjg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:references:in-reply-to:subject:date :message-id:mime-version:thread-index:content-language; bh=NmbjpOdpmDTE9hZr49HYvNXD3rWvGESKCnmHRYlEJV4=; b=gZG1kbmWhDvrtwzKBflePlPrQSfbwsl/Sp3JyuAeuKyOw53QxjtsAnI2rePGSQcjq+ J93G5JQUWvL2PWCH+8j4LfDRVUNskpuWs+wfcDyIpr2FuRHN9i0K6k1TyPs6h4SKE3TX Ie1UqtkdsHr4IMyMPPJpEhkLhve1VI3oi/VqMONo7/3FcPhghixAah/X00LeMi1gPN28 6hjoPOakgUPF4imwkpG8X3CzaZe6jsD7rUbxb8vK/JhIARv9iCTywwKXCAbs3cHT3tMd FC/PbVamjm9fRlGOEzBw6LseXfuDP2OMSkQrm4PGXZj3FRi2Xvq38+WTmEZ8hLmon50n A9iA==
X-Gm-Message-State: AOAM531pkuXrWsw+x0/itncGJePZNPNEzBrOYrpfXGyCbs25Ba3ama67 bQTkVoqSQuT371/Yqsg0cDXpk6Ml8FPUJQ==
X-Google-Smtp-Source: ABdhPJy26KHu/3JfjS/TOhLsMSNte2COjW7hV9XQl6737FvzVNoUQ0qiLdu3IAHOy0VPArAHrFm44g==
X-Received: by 2002:a37:b446:: with SMTP id d67mr750942qkf.136.1591205683471; Wed, 03 Jun 2020 10:34:43 -0700 (PDT)
Received: from WINDOWSU6SOVGL (c-98-235-212-118.hsd1.pa.comcast.net. [98.235.212.118]) by smtp.gmail.com with ESMTPSA id l19sm2424373qtq.13.2020.06.03.10.34.41 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Jun 2020 10:34:42 -0700 (PDT)
From: <bmonkman@netsecopen.org>
To: "'MORTON, ALFRED C \(AL\)'" <acm@research.att.com>, "'Simon Edwards'" <simon@selabs.uk>
Cc: <bmwg@ietf.org>
References: <DBBPR09MB30618401FAD3184921486E79B6880@DBBPR09MB3061.eurprd09.prod.outlook.com> <027801d639b7$0d4ac290$27e047b0$@netsecopen.org>, <4D7F4AD313D3FC43A053B309F97543CF0108A5F74A@njmtexg5.research.att.com> <MN2PR19MB3711D1A07CC4F368EE7BD9FDFE880@MN2PR19MB3711.namprd19.prod.outlook.com> <4D7F4AD313D3FC43A053B309F97543CF0108A5F7C6@njmtexg5.research.att.com>
In-Reply-To: <4D7F4AD313D3FC43A053B309F97543CF0108A5F7C6@njmtexg5.research.att.com>
Date: Wed, 3 Jun 2020 13:34:39 -0400
Message-ID: <029a01d639cd$427aa0a0$c76fe1e0$@netsecopen.org>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="----=_NextPart_000_029B_01D639AB.BB6A8740"
X-Mailer: Microsoft Outlook 16.0
Thread-Index: AQGlTKTnD6W7FccTsuGZe8HJJqcDGQNb0+YDAhSNGwMCJDaNhAMMsSBhqNP2mPA=
Content-Language: en-ca
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/9PStHIryKbiuUSKJ12DrwkF3EG4>
Subject: Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 03 Jun 2020 17:34:47 -0000

It doesn’t hurt either. 😊

 

Should we hold off until you complete your review before we submit a new draft? I am hoping others weigh in on this as well.

 

Brian

 

From: MORTON, ALFRED C (AL) <acm@research.att.com> 
Sent: June 3, 2020 1:04 PM
To: Brian Monkman <bmonkman@netsecopen.org>rg>; 'Simon Edwards' <simon@selabs.uk>
Cc: bmwg@ietf.org
Subject: RE: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

 

Hi Brian,  no other pauses or speed-bumps yet, but I need to complete my 

review of the most recent version.

 

sorry, I know that doesn’t help!

Al

 

From: Brian Monkman [mailto:bmonkman@netsecopen.org] 
Sent: Wednesday, June 3, 2020 12:21 PM
To: MORTON, ALFRED C (AL) <acm@research.att.com <mailto:acm@research.att.com> >; 'Simon Edwards' <simon@selabs.uk <mailto:simon@selabs.uk> >
Cc: bmwg@ietf.org <mailto:bmwg@ietf.org> 
Subject: Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

 

Hi Al,

 

Are there any other items that caused you pause that could be addressed the same way?

 

Brian

 

Get Outlook for iOS <https://urldefense.proofpoint.com/v2/url?u=https-3A__aka.ms_o0ukef&d=DwMF-g&c=LFYZ-o9_HUMeMTSQicvjIg&r=_6cen3Hn-e_hOm0BhY7aIpA58dd19Z9qGQsr8-6zYMI&m=gwfQJfXOl1ZubaA9a-aKfQ8n5AY2sKsb3Ml9NAklRwQ&s=ai2z_d2vwJeZx_EibZkLkoE_ZWS26ZSoAJq9RZNf6bM&e=> 

  _____  

From: MORTON, ALFRED C (AL) <acm@research.att.com <mailto:acm@research.att.com> >
Sent: Wednesday, June 3, 2020 12:12:27 PM
To: bmonkman@netsecopen.org <mailto:bmonkman@netsecopen.org>  <bmonkman@netsecopen.org <mailto:bmonkman@netsecopen.org> >; 'Simon Edwards' <simon@selabs.uk <mailto:simon@selabs.uk> >
Cc: bmwg@ietf.org <mailto:bmwg@ietf.org>  <bmwg@ietf.org <mailto:bmwg@ietf.org> >
Subject: RE: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02] 

 

Thanks for your question, Simon, and the history, Brian.

 

I confess that this question (why 50%?) has occurred to me in other contexts, and it may help to add a sentence to two of rationale. So if the technical folks can help with suggestions, that would be great!

 

regards,

Al

 

From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of bmonkman@netsecopen.org <mailto:bmonkman@netsecopen.org> 
Sent: Wednesday, June 3, 2020 10:56 AM
To: 'Simon Edwards' <simon@selabs.uk <mailto:simon@selabs.uk> >
Cc: bmwg@ietf.org <mailto:bmwg@ietf.org> 
Subject: Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

 

Simon,

 

This requirement was agreed to and adopted by the working group within NetSecOPEN. It first appeared in the IETF individual draft on October 14, 2018. (https://tools.ietf.org/html/draft-balarajah-bmwg-ngfw-performance-05 <https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dbalarajah-2Dbmwg-2Dngfw-2Dperformance-2D05&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=mylQC4CnyBjr1JS-Qbiw5d392Llnmp_6LxMRXJO8NVI&s=8ElQf-XUyhkke33v7BvLLf5mEBDCEU2mlwlFDpxHJD8&e=> ). It was clarified and expanded on March 5, 2019 in https://tools.ietf.org/html/draft-ietf-bmwg-ngfw-performance-00 <https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Dbmwg-2Dngfw-2Dperformance-2D00&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=mylQC4CnyBjr1JS-Qbiw5d392Llnmp_6LxMRXJO8NVI&s=xbiPb0v60VZvEM2cumbN3J41IrOw2hMUK7HHaR-HN8o&e=> . It’s form has largely been unchanged since then.

 

I will leave it to the technical folks to expand on this more. However, I am saying all of this because it has been in the position to be reviewed and commented on by the BMWG community for awhile. Our assumption is that given no one appears to have an issue with it that we hit the mark.

 

Brian

 

From: bmwg <bmwg-bounces@ietf.org <mailto:bmwg-bounces@ietf.org> > On Behalf Of Simon Edwards
Sent: June 3, 2020 5:10 AM
To: bmwg@ietf.org <mailto:bmwg@ietf.org> 
Subject: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

 

Hi all,

 

In a number of sections, but specifically '7.8.3.2.  Test Equipment Configuration Parameters', there are requirements to measure with 50% of the maximum connections/ sec measured in the HTTP/S throughput tests

 

E.g. "Target objective for scenarios 1 and 2: 50% of the maximum connections per second measured in test scenario..."

 

I'm sure this 50% value is the product of much thought and discussion, rather than an arbitrary choice. Is anyone able to explain the reason for the specific '50%' value (as opposed to 25%, 75% or whatever) or could you please point to documentation around that decision made by the group?

 

I'm asking just to understand. I don't disagree with the decision : )

 

Very best wishes,

Simon