Re: [bmwg] Roman Danyliw's Discuss on draft-ietf-bmwg-ngfw-performance-13: (with DISCUSS and COMMENT)

[Warren Kumari <[e-mail]>]

[ - Al (he is in the chairs expansion)  ]

Hi Roman.

The authors have posted a new version of this document.
Link to your DISCUSS for easy clickin':
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ngfw-performance/
Link to Diff:
https://www.ietf.org/rfcdiff?url1=draft-ietf-bmwg-ngfw-performance-13&url2=draft-ietf-bmwg-ngfw-performance-14&difftype=--html


I believe that the authors have attempted to answer / address your discuss
positions in the draft, and Carsten Rossenhövel has also answered (or
attempted to answer) many of the positions or get more information in email
(see above/ below).

Could you please review the updated document and the email discussion and
clear your DISCUSS or let us know what else is needed to satisfy?

Thank you,
W



On Wed, Feb 09, 2022 at 10:34 AM, <[e-mail]> wrote:

> Roman,
>
> Do you have response to the questions Carsten asked in response to you?
>
> Brian
>
> -----Original Message-----
> From: Carsten Rossenhoevel <[e-mail]> Sent: Wednesday, February 2,
> 2022 7:48 PM
> To: Roman Danyliw <[e-mail]>; The IESG <[e-mail]> Cc:
> [e-mail]; [e-mail];
> [e-mail]; Al Morton <[e-mail]> Subject: Re: Roman
> Danyliw's Discuss on draft-ietf-bmwg-ngfw-performance-13: (with DISCUSS and
> COMMENT)
>
> Dear Roman,
>
> thank you for your comments and questions. Primarily, the document is
> focused on performance benchmarking as mentioned in 4.2.1. This is the area
> where the authors and contributors saw a major gap in test methodology.
> Today's datasheets of commercial firewalls are sometimes difficult to
> interpret. Performance benchmarking methodology applies across all types of
> security features (per Table 1).
>
> Thus, we describe performance benchmarking methodology in great detail.
> Sections 4.3 and 7 focus on representative and reproducible load profiles.
> (Note - Training phases would not include threat emulation. This is normal
> and not mentioned otherwise in the document - do you recommend we change
> any text / add an explanation?)
>
> Advanced security features use a wide range of functions and methods, as
> you mentioned. It would be very difficult (and a race against time) to
> cover white-box test methodology exhaustively for such security features in
> a standards document. Therefore in this document, we:
>
> a) describe the recommended and optional security features at high level
> (section 4.2)
>
> b) suggest evaluating the correct functional security effectiveness of
> these features in advance of performance benchmarking, outside of the core
> performance benchmarking methodology (mentioned in section 4.2.1, second
> paragraph)
>
> c) safeguard against the false testing of a firewall without proper
> security features enabled (mentioned in section 4.2.1, first paragraph and
> 2nd sentence in second paragraph)
>
> d) describe security effectiveness test methodology verifying protection
> against common vulnerabilities and exposures (CVEs) in Appendix A. This
> test methodology is a black-box approach not aiming to verify/compare
> advanced concepts such as sandboxing, zero-day threat protection, and other
> AI-based functions. We are working on test methodology for these areas in
> the NetSecOPEN group; but it is difficult to come up with reasonable test
> solutions (specifically if they should be reproducible and test
> tool-agnostic), which is why this document does not include detailed
> methodology yet.
>
> Regarding "CVE" nomenclature, I agree that the language in Appendix A is
> not absolutely precise. The term CVE has become informally used for the
> (NIST/Mitre) database entries, the firewall capability to protect, and for
> the test tool attack type. The authors are open to improving the
> terminology. What do you suggest (without making each sentence a mouthful)?
>
> I also agree with your observation that CVEs are often not atomic and that
> test tool implementations and firewall implementations may cover parts of a
> CVE. This is an issue of the CVE database - there is no consistent
> numbering of sub-CVEs. We have suffered from this issue in our security
> effectiveness methodology verification program we have conducted since
> early 2021. Our goal is to create a standard document that is not tied to
> any specific test tool or lab implementation. Two test labs (UNH-IOL and
> EANTC) have worked with two test tool vendors (Spirent and Keysight) and
> multiple firewall vendors. We have evaluated thousands of CVEs, comparing
> test coverage and results.
>
> Each test tool vendor uses individual implementations to emulate threats
> aligned with CVEs, and their implementations sometimes differ in the
> sub-CVE coverage. Firewall vendors also have their individual code and do
> not always cover all sub-CVEs - often for good reasons that need to be
> analyzed on the application layer individually (very high effort). We
> concluded that we have to accept the state of the art in CVE granularity
> and that we have to live with the uncertainties of sub-CVE coverage in
> general. (In detail we will continue to work with test tool and firewall
> vendors to align implementations and aim for best-working CVE threat
> protection.) If you have any ideas how this draft document could help
> improve the industry more quickly, we are seriously open for suggestions.
>
> Re your other comments:
>
> - The introduction section and tables 1/2/3 are being rephrased
> editorially to increase their readability and add more explanations, based
> on feedback from other AD reviewers.
>
> - In section 4.2, I am not sure if I understand your comment about
> logging. According to RFC 2119, "RECOMMEND" and "SHOULD" consistently
> describe the same level of requirement?
>
> - Well-defined TCP stack attributes are required for a network security
> device because transport layer attributes influence its performance
> greatly. Many issues found in EANTC tests in the past originated from
> incorrect TCP stack attributes, and non-reproducible behavior of a firewall
> is often related to undocumented/uncontrolled TCP stack attributes.
>
> - From our understanding, TLS 1.3 cipher suites are all considered
> adequate in the industry today, while some of the TLS 1.2 cipher suites
> are considered deprecated. This is why we specify them in detail. That
> said, the note below the list of TLS 1.2 recommended ciphers encourages the
> use of ciphers appropriate for evolving use cases. This might lead to a
> selection of specific TLS 1.3 ciphers in the future as well.
>
> - Good point regarding the use of emulated exploits in section 9. We agree
> to add a note.
>
> - Regarding 2nd paragraph of A.1, second sentence ("In parallel..."), we
> agree that the text is imprecise. Threats that apply to the connection
> setup cannot be encrypted. We'll edit the text to apply to these cases as
> well.
>
> Best regards, Carsten
>
> Am 2/2/2022 um 11:39 PM schrieb Roman Danyliw via Datatracker:
>
> Roman Danyliw has entered the following ballot position for
> draft-ietf-bmwg-ngfw-performance-13: Discuss
>
> When responding, please keep the subject line intact and reply to all
> email addresses included in the To and CC lines. (Feel free to cut this
> introductory paragraph, however.)
>
> Please refer to
> https://www.ietf.org/blog/handling-iesg-ballot-positions/ for more
> information about how to handle DISCUSS and COMMENT positions.
>
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-ngfw-performance/
>
> ----------------------------------------------------------------------
> DISCUSS:
> ----------------------------------------------------------------------
>
> ** A key element of successfully running the throughput tests described in
> Section 7, appears to be ensuring how to configure the device under test.
> Section 4.2. helpfully specifies feature sets with recommendations
> configurations. However, it appears there are elements of
> under-specification given the level of detail specified with normative
> language. Specifically:
>
> -- Section 4.2.1 seems unspecified regarding all the capabilities in Table
> 1 and 2. The discussion around vulnerabilities (CVEs) does not appear to be
> relevant to configuration of anti-spyware, anti-virus, anti-botnet, DLP,
> and DDOS.
>
> -- Recognizing that NGFW, NGIPS and UTM are not precise product
> categories, offerings in this space commonly rely on statistical models or
> AI techniques (e.g., machine learning) to improve detection rates and
> reduce false positives to realize the capabilities in Table 1 and 2. If
> even possible, how should these settings be tuned? How should the training
> period be handled when describing the steps of the test regime (e.g., in
> Section 4.3.4? Section 7.2.4?)
>
> ** Appendix A. The KPI measures don’t seem precise here – CVEs are
> unlikely to be the measure seen on the wire. Wouldn’t it be exploits
> associated with a particular vulnerability (that’s numbered via CVE)? There
> can be a one-to-many relationship between the vulnerability and exploits
> (e.g., multiple products affected by a single CVE); or the multiple
> implementations of an exploit.
>
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
>
> ** Abstract. NGFW, NGIPS and UTM are fuzzy product categories. Do you want
> to them somewhere? How do they differ in functionality? UTM is mentioned
> here, but not again in the document.
>
> ** Section 1.
> The requirements
> for network security element performance and effectiveness have increased
> tremendously since then. In the eighteen years since
> [RFC3511] was published, recommending test methodology and terminology for
> firewalls, requirements and expectations for network security elements has
> increased tremendously.
>
> I don’t follow how the intent of these two sentences is different. Given
> the other text in this paragraph, these sentences also appear redundant.
>
> ** Section 3. Per “This document focuses on advanced, …”, what makes a
> testing method “advanced”?
>
> ** Section 4.2. The abstract said that testing for NGFW, NGIPS and UTM
> would be provided. This section is silent on UTM.
>
> ** Section 4.2. Should the following additional features be noted as a
> feature of NGFWs and NGIPS (Tables 1 and 2)?
>
> -- reconnaissance detection
>
> -- geolocation or network topology-based classification/filtering
>
> ** Section 4.2. Thanks for the capability taxonomies describe here. Should
> it be noted that “Table 1 and 2 are approximate taxonomies of features
> commonly found in currently deployed NGFW and NGIDS. The features provided
> by specific implementations may be named differently and not necessarily
> have configuration settings that align to the taxonomy.”
>
> ** Table 1. Is there a reason that DPI and Anti-Evasion (listed in Table 2
> for
> NGIPS) are not mentioned here (for NGFW). I don’t see how many (all?) of
> the features listed as RECOMMENDED could be done without it.
>
> ** Table 3. For Anti-Botnet, should it read “detects and blocks”?
>
> ** Table 3. For Web Filtering, is this scoped to be classification and
> threat detection by URI?
>
> ** Table 3. This table is missing a description for DoS from Table 1 and
> DPI and Anti-Evasion from Table 2.
>
> ** Section 4.2. Per “Logging SHOULD be enabled.” How does this
> “SHOULD” align with “logging and reporting” being a RECOMMENDED in Table 1
> and 2? Same question on “Application Identification and Control SHOULD be
> configured”
>
> ** Section 4.3.1.1. Why is such well-formed and well-behaved traffic
> assumed for a security device?
>
> ** Section 4.3.1. What cipher suites should be used for TLS 1.3 based
> tests? The text is prescriptive for TLS 1.2 (using a RECOMMEND) but simply
> restates all of those registered by RFC8446.
>
> ** Section 9. Given that the configurations of these test will include
> working exploits, it would be helpful to provide a reminder on the need
> control access to them.
>
> ** Section A.1.
> In parallel, the CVEs will be sent to the DUT/SUT as encrypted and as well
> as clear text payload formats using a traffic generator.
>
> This guidance doesn’t seem appropriate for all cases. Couldn’t the
> vulnerability being exploited involve a payload in the unencrypted part or
> a phase in the communication exchange before a secure channel is
> negotiated?
>
> ** Editorial nits
> -- Section 1. Editorial. s/for firewalls initially/for firewalls/
>
> -- Section 5. Typo. s/as test equipments/as test equipment/
>
> --
> Carsten Rossenhövel
> Managing Director, EANTC AG (European Advanced Networking Test Center)
> Salzufer 14, 10587 Berlin, Germany office +49.30.3180595-21, fax
> +49.30.3180595-10, mobile [phone] [e-mail], https://www.
> eantc.de
>
> Place of Business/Sitz der Gesellschaft: Berlin, Germany
> Chairman/Vorsitzender des Aufsichtsrats: Herbert Almus Managing
> Directors/Vorstand: Carsten Rossenhövel, Gabriele Schrenk Registered: HRB
> 73694, Amtsgericht Charlottenburg, Berlin, Germany EU VAT No: DE812824025
>