Re: [bmwg] Roman Danyliw's Discuss on draft-ietf-bmwg-ngfw-performance-13: (with DISCUSS and COMMENT)
bmonkman@netsecopen.org Thu, 03 February 2022 00:19 UTC
Return-Path: <bmonkman@netsecopen.org>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F3F433A0DA1 for <bmwg@ietfa.amsl.com>; Wed, 2 Feb 2022 16:19:03 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Level:
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=netsecopen-org.20210112.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L-8UIZZMxVy4 for <bmwg@ietfa.amsl.com>; Wed, 2 Feb 2022 16:18:58 -0800 (PST)
Received: from mail-qk1-x733.google.com (mail-qk1-x733.google.com [IPv6:2607:f8b0:4864:20::733]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C5AB03A0D9B for <bmwg@ietf.org>; Wed, 2 Feb 2022 16:18:58 -0800 (PST)
Received: by mail-qk1-x733.google.com with SMTP id g145so1233675qke.3 for <bmwg@ietf.org>; Wed, 02 Feb 2022 16:18:58 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=netsecopen-org.20210112.gappssmtp.com; s=20210112; h=from:to:cc:references:in-reply-to:subject:date:message-id :mime-version:content-transfer-encoding:thread-index :content-language; bh=J1y1pshEZK0fwBizWgPj1NJ58/0WmdZdnbks43WkwRk=; b=5/FBAfxfynha6Sm4OGk+dc1f7ksSiiE5mGFy2qtTrnL1gQstG+h+En60KTCxePO3Pk N7kAzIyqpKaM59fDn0tdJYDNeCaeKdJPUvyWUCLSsUMWEa4GsGVlBMYfT2CSY6C+aUIv VC6CvSi2nHZ+DXWlz46MnAhZJ0LmlKbo2S/5lXhQfi6qXM5xNM0fKO9v14uufglr3zpG O2OHLbbIU6DIdETA3X7GFq9+e907vN2rcUIoi8wMbsb2QhynWmX1D+q2lxe1OQOjZzfX xRXgnPLus8VFwpNEp2BgJwhnD2MYhVQqBbyUu1MyaDgLRZemke+IUA61XZYwVgxOs9Sv sATw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:references:in-reply-to:subject:date :message-id:mime-version:content-transfer-encoding:thread-index :content-language; bh=J1y1pshEZK0fwBizWgPj1NJ58/0WmdZdnbks43WkwRk=; b=0RTL+5/BaNG53dR8IfxwcQXtfMyK8QyvH6eyvXNvtGq6X7Rf/P5V7F07wr8j5vF8hZ +TF3FESem4CvZyCmQrC/lOH42smgIfCmSw0e4Hb7pZ7f+IIEeBrD2Pf6FWgbHsulRi84 blefBplEsqweJ3QR2A9i8WiQOs7iD6Bz8CrKeM2UzYDhfAJ1jYIvBauZ9KtpqUkkqVSl nFhzD54wkvvPbfuKYq8uUr6X3KMZQ1ZhFVHlkjldEPmkPsd8NsIl9I9/ZwYtmKEhs4Ig anp7JTsGL7uSIN+Y63UkPeNsHFqiCXmr3/9t6tiVDfZxQpV16zkuOoBHcpFTaw6UtHql EVUg==
X-Gm-Message-State: AOAM533FjAEy/JK/VQMjzNSuxMz/6pWs1RsX2xKMUvZ/00xT4o49TsvY HCuy9Smqfng+GdMkX8WxpJg6yv/Vlt1Ppg==
X-Google-Smtp-Source: ABdhPJyzLihY4pzc/H7Ig196zSLZo3cPLrwREif8IBu1te8+yaZXBIjPXUdQobHhjnN+S1Ty0tW6VQ==
X-Received: by 2002:a05:620a:146a:: with SMTP id j10mr21819303qkl.745.1643847537057; Wed, 02 Feb 2022 16:18:57 -0800 (PST)
Received: from DESKTOP42TMNEU (c-98-235-212-118.hsd1.pa.comcast.net. [98.235.212.118]) by smtp.gmail.com with ESMTPSA id 2sm12619352qtf.9.2022.02.02.16.18.56 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 02 Feb 2022 16:18:56 -0800 (PST)
From: bmonkman@netsecopen.org
To: "'MORTON JR., AL'" <acmorton@att.com>, 'Carsten Rossenhoevel' <cross@eantc.de>, bala@netsecopen.org
Cc: 'Sarah Banks' <sbanks@encrypted.net>, 'Warren Kumari' <warren@kumari.net>, bmwg@ietf.org
References: <164384157725.26994.16348654460944534798@ietfa.amsl.com> <CH0PR02MB798001C2AE648557231EA114D3279@CH0PR02MB7980.namprd02.prod.outlook.com>
In-Reply-To: <CH0PR02MB798001C2AE648557231EA114D3279@CH0PR02MB7980.namprd02.prod.outlook.com>
Date: Wed, 02 Feb 2022 19:18:34 -0500
Message-ID: <28fa01d81893$a1c103f0$e5430bd0$@netsecopen.org>
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
X-Mailer: Microsoft Outlook 16.0
Thread-Index: AQHcXMbmSPXXRgt1VXosEwdw/9DlzQL5UeiCrGChz7A=
Content-Language: en-us
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/QMPq-05N5wkfz5O_XiuSCP11ttM>
Subject: Re: [bmwg] Roman Danyliw's Discuss on draft-ietf-bmwg-ngfw-performance-13: (with DISCUSS and COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 03 Feb 2022 00:19:05 -0000
Thanks for the feedback Al. These are all very useful comments. However, given the complexity of the questions/comments and given that the IESG meeting is early tomorrow I am concerned that there won't be enough time to provide an adequate answer. We will give it our best shot. Brian -----Original Message----- From: MORTON JR., AL <acmorton@att.com> Sent: Wednesday, February 2, 2022 6:14 PM To: Carsten Rossenhoevel <cross@eantc.de>; bmonkman@netsecopen.org; bala@netsecopen.org Cc: Sarah Banks <sbanks@encrypted.net>; 'Warren Kumari' <warren@kumari.net>; bmwg@ietf.org Subject: FW: Roman Danyliw's Discuss on draft-ietf-bmwg-ngfw-performance-13: (with DISCUSS and COMMENT) Authors, Roman's DISCUSS ballot is particularly important to address. Specifically, the points above ---------------------------------------------------------------------- COMMENT: line, but all the points should be addressed eventually. If you can provide some proposed resolutions before the IESG meets Thursday, it will give our AD Warren some backup for any exchanges with Roman during the live meeting. Status as of Wednesday evening: There are surprisingly few IESG ballots cast (4) https://datatracker.ietf.org/doc/draft-ietf-bmwg-ngfw-performance/ballot/ I previously sent the link to the webex for the meeting, so you can listen-in and possibly gain some insights. Al -----Original Message----- From: Roman Danyliw via Datatracker <noreply@ietf.org> Sent: Wednesday, February 2, 2022 5:40 PM To: The IESG <iesg@ietf.org> Cc: draft-ietf-bmwg-ngfw-performance@ietf.org; bmwg-chairs@ietf.org; bmwg@ietf.org; Al Morton <acm@research.att.com>; acm@research.att.com Subject: Roman Danyliw's Discuss on draft-ietf-bmwg-ngfw-performance-13: (with DISCUSS and COMMENT) Roman Danyliw has entered the following ballot position for draft-ietf-bmwg-ngfw-performance-13: Discuss When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.) Please refer to https://urldefense.com/v3/__https://www.ietf.org/blog/handling-iesg-ballot-positions/__;!!BhdT!wz4NWwr2EhVf2SVpgeLrJ37BbODitrSIIGTSdWLeVIHQAuDPrMeDZojEcJCm$ for more information about how to handle DISCUSS and COMMENT positions. The document, along with other ballot positions, can be found here: https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-ietf-bmwg-ngfw-performance/__;!!BhdT!wz4NWwr2EhVf2SVpgeLrJ37BbODitrSIIGTSdWLeVIHQAuDPrMeDZpdwKyaS$ ---------------------------------------------------------------------- DISCUSS: ---------------------------------------------------------------------- ** A key element of successfully running the throughput tests described in Section 7, appears to be ensuring how to configure the device under test. Section 4.2. helpfully specifies feature sets with recommendations configurations. However, it appears there are elements of under-specification given the level of detail specified with normative language. Specifically: -- Section 4.2.1 seems unspecified regarding all the capabilities in Table 1 and 2. The discussion around vulnerabilities (CVEs) does not appear to be relevant to configuration of anti-spyware, anti-virus, anti-botnet, DLP, and DDOS. -- Recognizing that NGFW, NGIPS and UTM are not precise product categories, offerings in this space commonly rely on statistical models or AI techniques (e.g., machine learning) to improve detection rates and reduce false positives to realize the capabilities in Table 1 and 2. If even possible, how should these settings be tuned? How should the training period be handled when describing the steps of the test regime (e.g., in Section 4.3.4? Section 7.2.4?) ** Appendix A. The KPI measures don’t seem precise here – CVEs are unlikely to be the measure seen on the wire. Wouldn’t it be exploits associated with a particular vulnerability (that’s numbered via CVE)? There can be a one-to-many relationship between the vulnerability and exploits (e.g., multiple products affected by a single CVE); or the multiple implementations of an exploit. ---------------------------------------------------------------------- COMMENT: ---------------------------------------------------------------------- ** Abstract. NGFW, NGIPS and UTM are fuzzy product categories. Do you want to them somewhere? How do they differ in functionality? UTM is mentioned here, but not again in the document. ** Section 1. The requirements for network security element performance and effectiveness have increased tremendously since then. In the eighteen years since [RFC3511] was published, recommending test methodology and terminology for firewalls, requirements and expectations for network security elements has increased tremendously. I don’t follow how the intent of these two sentences is different. Given the other text in this paragraph, these sentences also appear redundant. ** Section 3. Per “This document focuses on advanced, …”, what makes a testing method “advanced”? ** Section 4.2. The abstract said that testing for NGFW, NGIPS and UTM would be provided. This section is silent on UTM. ** Section 4.2. Should the following additional features be noted as a feature of NGFWs and NGIPS (Tables 1 and 2)? -- reconnaissance detection -- geolocation or network topology-based classification/filtering ** Section 4.2. Thanks for the capability taxonomies describe here. Should it be noted that “Table 1 and 2 are approximate taxonomies of features commonly found in currently deployed NGFW and NGIDS. The features provided by specific implementations may be named differently and not necessarily have configuration settings that align to the taxonomy.” ** Table 1. Is there a reason that DPI and Anti-Evasion (listed in Table 2 for NGIPS) are not mentioned here (for NGFW). I don’t see how many (all?) of the features listed as RECOMMENDED could be done without it. ** Table 3. For Anti-Botnet, should it read “detects and blocks”? ** Table 3. For Web Filtering, is this scoped to be classification and threat detection by URI? ** Table 3. This table is missing a description for DoS from Table 1 and DPI and Anti-Evasion from Table 2. ** Section 4.2. Per “Logging SHOULD be enabled.” How does this “SHOULD” align with “logging and reporting” being a RECOMMENDED in Table 1 and 2? Same question on “Application Identification and Control SHOULD be configured” ** Section 4.3.1.1. Why is such well-formed and well-behaved traffic assumed for a security device? ** Section 4.3.1. What cipher suites should be used for TLS 1.3 based tests? The text is prescriptive for TLS 1.2 (using a RECOMMEND) but simply restates all of those registered by RFC8446. ** Section 9. Given that the configurations of these test will include working exploits, it would be helpful to provide a reminder on the need control access to them. ** Section A.1. In parallel, the CVEs will be sent to the DUT/SUT as encrypted and as well as clear text payload formats using a traffic generator. This guidance doesn’t seem appropriate for all cases. Couldn’t the vulnerability being exploited involve a payload in the unencrypted part or a phase in the communication exchange before a secure channel is negotiated? ** Editorial nits -- Section 1. Editorial. s/for firewalls initially/for firewalls/ -- Section 5. Typo. s/as test equipments/as test equipment/
- [bmwg] Roman Danyliw's Discuss on draft-ietf-bmwg… Roman Danyliw via Datatracker
- [bmwg] FW: Roman Danyliw's Discuss on draft-ietf-… MORTON JR., AL
- Re: [bmwg] Roman Danyliw's Discuss on draft-ietf-… bmonkman
- Re: [bmwg] Roman Danyliw's Discuss on draft-ietf-… Carsten Rossenhoevel
- Re: [bmwg] Roman Danyliw's Discuss on draft-ietf-… bmonkman
- Re: [bmwg] Roman Danyliw's Discuss on draft-ietf-… Warren Kumari
- Re: [bmwg] Roman Danyliw's Discuss on draft-ietf-… bmonkman
- Re: [bmwg] Roman Danyliw's Discuss on draft-ietf-… Roman Danyliw