Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]

bmonkman@netsecopen.org Wed, 03 June 2020 14:55 UTC

Return-Path: <bmonkman@netsecopen.org>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 181073A08E9 for <bmwg@ietfa.amsl.com>; Wed, 3 Jun 2020 07:55:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.898
X-Spam-Level:
X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=netsecopen-org.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OhlZadS8IeEq for <bmwg@ietfa.amsl.com>; Wed, 3 Jun 2020 07:55:45 -0700 (PDT)
Received: from mail-qt1-x835.google.com (mail-qt1-x835.google.com [IPv6:2607:f8b0:4864:20::835]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 5CACB3A08D4 for <bmwg@ietf.org>; Wed, 3 Jun 2020 07:55:45 -0700 (PDT)
Received: by mail-qt1-x835.google.com with SMTP id w9so2267791qtv.3 for <bmwg@ietf.org>; Wed, 03 Jun 2020 07:55:45 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netsecopen-org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:thread-index:content-language; bh=Up3sA6GIs/kirXBmlfyb7ur8K/d+0644zoDT95Ixys0=; b=KzaBEsd8YwFgVQcT4AK++iELbcVF9IxtIaw+VUR4G1TH6X0qJKSoK9if9PsSibbMqr KpQ28REXix/JGNV4UFE+1SmxhnS69LZYxRFmDIzInRIdMFW72/x8T306JED0zd6ofDyO Vgw7B7FmruXsRUFE7M2dHOwS882paXRdeCI6lG/bY7Z932ZTvmqctSpR9q34X7HFDQsw 9JYGL8rriVtsFGmB8D/DA7FIGBesjr3V0TERlIXc2pWKI7nwy+WtKxDKhQ0UK/ptpSt6 e4W3qx8AsLU1PS6YDXQyl1mNCes/PSjonYPzTZaJ802lIibF72Xk65injoD+tUHT2sGn xEbA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:references:in-reply-to:subject:date :message-id:mime-version:thread-index:content-language; bh=Up3sA6GIs/kirXBmlfyb7ur8K/d+0644zoDT95Ixys0=; b=RgIsKYzx4erlYx33rfVnUonFBnYvCPfCmdHLWNX9dYfza+rvGNnDV7tf1wT0ilkGis Z288CGJG5mxzL2/XmvvQxgksSwlxL8mtZT6WStR95avJ4gdtPdy69sn1kMEyvA2jr6Mk mSeIdus1IvWoNcxeybOiHVFhc0WyyW7vJchHv6mjPuq9CpXQEvXzptX3YkFEDF1mLmAM I7CdfMP0kA70XeDELOBZesERougorbkEwk+UnWPL5Gp+DW+t4CsRXg60YgxHRbrOK6Ce TVj+nY93yQwxGg6eKTXQ373uSm9hHg3FixN9AWSXs5lAWAy7RO/LXatFiHlryeT8HeHm aXxQ==
X-Gm-Message-State: AOAM531V8xl/JYCsQqhJLPcCd1YyMzJE/IJ6AzNzJOFvzOu3xCvQ4mx2 5wrR8Hb+y2HYcUAxte/nI04UIHSdIMmgww==
X-Google-Smtp-Source: ABdhPJy5d156ocyfNhLqSJPdHmXEFmQH8lJobs42NW4RgN6u4CZTrPOwfO0XRcttG5JX86BBZwoJgA==
X-Received: by 2002:ac8:2dc3:: with SMTP id q3mr32015123qta.141.1591196144195; Wed, 03 Jun 2020 07:55:44 -0700 (PDT)
Received: from WINDOWSU6SOVGL (c-98-235-212-118.hsd1.pa.comcast.net. [98.235.212.118]) by smtp.gmail.com with ESMTPSA id m53sm2253312qtb.64.2020.06.03.07.55.43 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Jun 2020 07:55:43 -0700 (PDT)
From: <bmonkman@netsecopen.org>
To: "'Simon Edwards'" <simon@selabs.uk>
Cc: <bmwg@ietf.org>
References: <DBBPR09MB30618401FAD3184921486E79B6880@DBBPR09MB3061.eurprd09.prod.outlook.com>
In-Reply-To: <DBBPR09MB30618401FAD3184921486E79B6880@DBBPR09MB3061.eurprd09.prod.outlook.com>
Date: Wed, 3 Jun 2020 10:55:41 -0400
Message-ID: <027801d639b7$0d4ac290$27e047b0$@netsecopen.org>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="----=_NextPart_000_0279_01D63995.8639E5E0"
X-Mailer: Microsoft Outlook 16.0
Thread-Index: AQGlTKTnD6W7FccTsuGZe8HJJqcDGako0deQ
Content-Language: en-ca
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/p4YnxzLtfMJ20idAX2ZljMQzFgU>
Subject: Re: [bmwg] Query about 50% values in [Benchmarking Methodology for Network Security Device Performance draft 02]
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 03 Jun 2020 14:55:47 -0000

Simon,

 

This requirement was agreed to and adopted by the working group within
NetSecOPEN. It first appeared in the IETF individual draft on October 14,
2018.
(https://tools.ietf.org/html/draft-balarajah-bmwg-ngfw-performance-05). It
was clarified and expanded on March 5, 2019 in
https://tools.ietf.org/html/draft-ietf-bmwg-ngfw-performance-00. It's form
has largely been unchanged since then.

 

I will leave it to the technical folks to expand on this more. However, I am
saying all of this because it has been in the position to be reviewed and
commented on by the BMWG community for awhile. Our assumption is that given
no one appears to have an issue with it that we hit the mark.

 

Brian

 

From: bmwg <bmwg-bounces@ietf.org> On Behalf Of Simon Edwards
Sent: June 3, 2020 5:10 AM
To: bmwg@ietf.org
Subject: [bmwg] Query about 50% values in [Benchmarking Methodology for
Network Security Device Performance draft 02]

 

Hi all,

 

In a number of sections, but specifically '7.8.3.2.  Test Equipment
Configuration Parameters', there are requirements to measure with 50% of the
maximum connections/ sec measured in the HTTP/S throughput tests

 

E.g. "Target objective for scenarios 1 and 2: 50% of the maximum connections
per second measured in test scenario..."

 

I'm sure this 50% value is the product of much thought and discussion,
rather than an arbitrary choice. Is anyone able to explain the reason for
the specific '50%' value (as opposed to 25%, 75% or whatever) or could you
please point to documentation around that decision made by the group?

 

I'm asking just to understand. I don't disagree with the decision : )

 

Very best wishes,

Simon