Re: [auth48] AUTH48: RFC-to-be 9411 <draft-ietf-bmwg-ngfw-performance-15> for your review

[[e-mail]]

Authors,

While reviewing this document during AUTH48, please resolve (as necessary) the following questions, which are also in the XML file.

1) <!--[rfced] Section 1. FYI, we have updated instances where "ACL" is expanded as "access control rule" to be "access control list". Please review and let us know if further updates are needed.

Original:
   Firewalls have evolved significantly from the days of simple ACL 
   filters.
   ...
   In addition, a realistic number of access control rules (ACL) SHOULD
   be configured on the DUT/SUT where ACLs are configurable and
   reasonable based on the deployment scenario.
   
Current:
   Firewalls have evolved significantly from the days of simple access
   control list (ACL) filters.
   ...
   In addition, a realistic number of access control lists (ACLs) SHOULD
   be configured on the DUT/SUT where ACLs are configurable and
   reasonable based on the deployment scenario.
-->   


2) <!-- [rfced] Section 4.2.1. Would you like to provide a reference for the Common Vulnerabilities and Exposures (CVE) list?

Original:
   This section defines
   the selection of the security vulnerability sets from the Common
   Vulnerabilities and Exposures (CVE) list for testing.
-->


3) <!-- [rfced] Section 4.3.1.3.  FYI, we have updated the following sentence by removing "be". Please let us know if any changes are necessary:

Original:

   *  If multiple IP blocks are used, they MUST be consist of multiple
      unique, discontinuous static address blocks.

Current:

   *  If multiple IP blocks are used, they MUST consist of multiple
      unique, discontinuous static address blocks.
-->


4) <!-- [rfced] Section 4.3.1.4.  We have updated the following HTTP/2 terms to match RFC 9113. Please let us know if any updates are needed:

Original:

   *  Maximum Frame size: 16384 bytes

   *  Initial Window size: 65535 bytes

   *  HPACK Header table size: 4096 bytes

   *  Server PUSH enable: false


Current (using lowercase and changing "Header table" to "header field table"):

   *  Maximum frame size: 16384 bytes

   *  Initial window size: 65535 bytes

   *  HPACK header field table size: 4096 bytes

   *  Server push enable: false
-->


5) <!--[rfced] Section 4.3.2.4. Is "any port" all encompassing? Should "UDP port 443" be removed?

Original:
  HTTP/3 server pool listens on UDP port 443 or
  any port.

Perhaps:
  The HTTP/3 server pool listens on any port.
-->  


6) <!-- [rfced] Section 4.3.3.  We are having difficulty parsing the following. Will the server be ready to accept "connection states" or just "connections"? Is the initialization of the transport stack part of the init phase or the ramp up phase of the traffic load profile? Are the HTTP(S) servers bound or are their connections inbound? Are clients added throughout the testing, as implied by "When a client endpoint is needed", or only during the init phase, as described in Section 4.3.4? Please let us know how to make this clearer.

Original:
   At the beginning of the test, the server endpoint
   initializes and will be ready to accept connection states including
   initialization of the TCP or QUIC stack as well as bound HTTP and
   HTTPS servers.  When a client endpoint is needed, it will initialize
   and be given attributes such as a MAC and IP address.  The behavior
   of the client is to sweep through the given server IP space,
   generating a recognizable service by the DUT.   

Possibly (clarifying load phase and server connections; tweaking the wording of client behavior):
   At the beginning of the test (the init phase; see Section 4.3.4), 
   the server endpoint initializes, and the server endpoint will be 
   ready to accept TCP or QUIC connections as well as inbound HTTP and 
   HTTPS requests.  The client endpoints initialize and are given 
   attributes such as a MAC and IP address. After the init phase of 
   the test, each client sweeps through the given server IP space,
   generating a service recognizable by the DUT.    
-->


7) <!-- [rfced] Section 4.3.3.1. Does the following suggestion improve the readability of the sentence?

Original:
   If using SNI, the server MUST then perform an SNI name check 
   with the proposed FQDN compared to the domain embedded in
   the certificate.

Perhaps (changing "with...compared to" to "by comparing...to"):
   If using SNI, the server MUST then perform an SNI name check 
   by comparing the proposed FQDN to the domain embedded in
   the certificate.
-->


8) <!-- [rfced] Section 5. Does the following update improve the readability of the sentence?

Original:

   3.  Ensure that any ancillary switching or routing functions added in
       the test equipment do not limit the performance by introducing
       network metrics such as packet loss and latency.

Perhaps (removing "network metrics such as" because the terms "packet loss and latency" don't refer to metrics in this case):

   3.  Ensure that any ancillary switching or routing functions added in
       the test equipment do not limit performance by introducing
       packet loss or latency.
-->


9) <!--[rfced] Section 6.1. As both of the bullet points listed below list multiple items, should their contents be separated into multiple bullet points?

Original:
   *  If the DUT/SUT is a Virtual Network Function (VNF), host
      (server) hardware and software details, interface
      acceleration type such as DPDK and SR-IOV, used CPU cores,
      used RAM, resource sharing (e.g.  Pinning details and NUMA
      Node) configuration details, hypervisor version, virtual
      switch version
      ...
   *  If the test equipment is a virtual solution, the host
      (server) hardware and software details, interface
      acceleration type such as DPDK and SR-IOV, used CPU cores,
      used RAM, resource sharing (e.g.  Pinning details and NUMA
      Node) configuration details, hypervisor version, virtual
      switch version      
-->	      


10) <!-- [rfced] Section 7.1.3.1. We are having difficulty parsing the following sentence. Does TLS inspection impact the mix of encrypted traffic at the application layer? 

Original:
   In case the DUT/SUT is configured without TLS inspection, the test 
   report MUST explain the implications of this to the relevant 
   application traffic mix encrypted traffic.

Possibly:
   If the DUT/SUT is configured without TLS inspection, the test 
   report MUST explain how this impacts the mix of relevant encrypted 
   traffic at the application layer.
-->


11) <!-- [rfced] Section 7.2.4 and also 7.6.4. Do the following updates clarify these two sentences?

Original:
   The test procedure is designed to measure the TCP connections per
   second rate of the DUT/SUT at the sustaining period of the traffic
   load profile.  
   ...
   The test procedure is designed to measure the TCP or QUIC connections
   per second rate of the DUT/SUT at the sustaining period of the
   traffic load profile.  

Perhaps (clarifying what is being measured and when):
   The test procedure is designed to measure the DUT/SUT's rate of TCP 
   connections per second during the sustaining period of the traffic
   load profile.  
   ...
   The test procedure is designed to measure the DUT/SUT's rate of TCP
   or QUIC connections per second during the sustaining period of the
   traffic load profile.  
-->


12) <!--[rfced] Section 7.3.4.1. "Initial inspected throughput" is not defined in Section 7.3.3.2, but that section does define "Target inspected throughput" and "Initial throughput". Please review and let us know how to update.

Original:
   Configure the traffic load profile of the test equipment to establish
   "Initial inspected throughput" as defined in Section 7.3.3.2.
   ...
   The traffic load profile MUST be defined as described in
   Section 4.3.4.  The DUT/SUT MUST reach the "Initial inspected
   throughput" during the sustain phase. 
-->


13) <!--[rfced] Sections 7.5.4.1 and 7.9.4.1. We note that Sections 7.5.3.2 and 7.9.3.2 do not contain definitions of "Initial concurrent TCP connections", but it does contain "Initial concurrent connections". Please review and let us know how to update. 

Original:
   Configure test equipment to establish "Initial concurrent TCP
   connections" defined in Section 7.5.3.2.
   ...
   Configure test equipment to establish "Initial concurrent TCP
   connections" defined in Section 7.9.3.2.   
-->   


14) <!-- [rfced] Informative References. FYI, we updated the Wikipedia reference to include the title, date, and a dated URI. Please let us know if any updates are necessary.

Original:
   [Wiki-NGFW]
              "",
              <https://en.wikipedia.org/wiki/Next-generation_firewall>.

Current:
   [Wiki-NGFW]
              Wikipedia, "Next-generation firewall", January 2023,
              <https://en.wikipedia.org/w/index.php?title=Next-
              generation_firewall&oldid=1133673904>.
-->


15) <!-- [rfced] Informative References. FYI, we updated the title of this reference to the title of the URL provided.

Original:
   [fastly]   Oku, K. and J. Iyengar, "Can QUIC match TCP's
              computational efficiency?", 30 April 2020,
              <https://www.fastly.com/blog/measuring-quic-vs-tcp-
              computational-efficiency>.

Current:
   [fastly]   Oku, K. and J. Iyengar, "QUIC vs TCP: Which is
              Better?", 30 April 2020,
              <https://www.fastly.com/blog/measuring-quic-vs-tcp-
              computational-efficiency>.
-->


16) <!--[rfced] Appendix A.5. FYI, to parallel the other terminology defined in Appendix A.5, we have updated the text as follows. Please let us know of any objections.

Original:
   -  Heavily impacted: Considered as "Heavily impacted" if any
      deviation of traffic forwarding rate is greater than 10% (i.e. 
      large spikes) or reduced the background HTTP(S) throughput
      greater than 10%

Current:
   -  Heavy impact: Considered as "heavy impact" if any
      deviation of the traffic forwarding rate is greater than 10% (i.e.,
      large spikes) or reduced the background HTTP(S) throughput
      greater than 10%
-->


17) <!--[rfced] Appendix A.6.2. Should the section number be included here?

Original:
   While generating background traffic (in sustain phase), send the CVE
   traffic as defined in the parameter section.

Perhaps:
   While generating background traffic (in sustain phase), send the CVE
   traffic as defined in the parameter section (Appendix A.3.2).
-->   


18) <!-- [rfced] Appendix A.6.2. Does the following reordering of the conditional phrase improve the readability of the sentence? 

Original:
   In addition, the DUT/SUT should either report the detected
   vulnerabilities in the log correctly or if, for example, a different
   naming convention is used, there MUST be reference material available
   that will allow for verification that the correct vulnerability was
   detected. 

Perhaps (moving the example conditional phrase to the end):
   In addition, the DUT/SUT should report the detected 
   vulnerabilities in the log correctly, or there MUST be reference 
   material available that will allow for verification that the correct 
   vulnerability was detected if, for example, a different naming 
   convention is used.
-->


19) <!-- [rfced] Formatting Questions:

a) Section 4.2. Would you like to format Figure 3 as a <table> instead?

b) Please review the lists in the "Test Equipment Configuration Parameters" sections listed below to ensure correctness. We updated to use bulleted lists for the parameters (the original xml contained a mix of <ul empty="true" spacing="normal"> and <t>).

7.2.3.2
7.3.3.2
7.4.3.2
7.5.3.2
7.6.3.2
7.7.3.2
7.8.3.2
7.9.3.2
                                                                        
In particular, in Section 7.2.3.2, should the last two paragraphs (after the Note) be part of the bulleted list? Also, in Section 7.6.3.2, should the last paragraph be part of the bulleted list?  

c) Please review whether any of the notes in this document should be in the <aside> element. It is defined as "a container for content that is semantically less important or tangential to the content that surrounds it" (https://authors.ietf.org/en/rfcxml-vocabulary#aside).
-->


20) <!--[rfced] FYI, to make the use of the terms "target objective" and "initial objective" consistent, we have updated '"Initial objective"' to 'initial objectives' and '"Target objective"' to 'target objectives' when the terms were referring to multiple objectives. Please let us know if any changes are needed. For example:

Original:
   Configure the traffic load profile of the test equipment to establish
   the "Initial objective" as defined in Section 7.4.3.2.

Current (made 'initial objective' plural because there are two initial objectives specified in Section 7.4.3.2): 
   Configure the traffic load profile of the test equipment to establish
   the initial objectives as defined in Section 7.4.3.2.
-->


21) <!-- [rfced] Throughout the text, the KPIs appear to be capitalized inconsistently. May we make the KPIs lower case when used generally in the text and upper case in section titles and when they are being defined?

For example:
Original:
   The duration for the ramp
   up phase MUST be configured long enough that the test equipment
   does not overwhelm the DUT/SUTs stated performance metrics
   defined in Section 6.3 namely, TCP or QUIC Connections Per
   Second, Inspected Throughput, Concurrent TCP or QUIC Connections,
   and Application Transactions Per Second.
   ...
   *  Application Transactions Per Second
   
      The average number of successfully completed transactions per
      second.  For a particular transaction to be considered successful,
      all data MUST have been transferred in its entirety.  In case of
      HTTP(S) transactions, it MUST have a valid status code (200 OK).

Perhaps:
   The duration for the ramp up phase
   MUST be configured long enough that the test equipment does not
   overwhelm the DUT's/SUT's stated performance metrics defined in
   Section 6.3, namely TCP/QUIC connections per second, inspected
   throughput, concurrent TCP/QUIC connections, and application
   transactions per second.
   ...
   Application Transactions Per Second
      The average number of successfully completed transactions per
      second.  For a particular transaction to be considered successful,
      all data MUST have been transferred in its entirety.  In case of
      an HTTP(S) transaction, it MUST have a valid status code (200 OK).

List of the specific KPIs that are capitalized inconsistently:
Concurrent TCP/QUIC Connections vs. concurrent TCP/QUIC connections
Connections Per Second vs. connections per second vs. Connections per second vs.  Connections per Second
   Note that the following is consistently capitalized: Initial connections per second
Handshake Rate vs. handshake rate
HTTP/HTTPS Throughput vs. HTTP/HTTPS throughput
HTTP/HTTPS Transaction Latency vs. HTTP/HTTPS transaction latency
Initial throughput vs. initial throughput
Inspected Throughput vs. inspected throughput
Transactions Per Second vs. transactions per second
-->


22) <!-- [rfced] Terminology 

a) May we update "KBytes" to "KB" to match more common usage?

b) Please review the use of "TCP/QUIC connections". Does this mean "TCP or QUIC", "TCP and QUIC", or "TCP and/or QUIC"?

c) Please review the use of "TCP/HTTP connections". Does this mean "TCP connections carrying HTTP traffic"?
-->


23) <!-- [rfced] Please review the "Inclusive Language" portion of the online Style Guide <https://www.rfc-editor.org/styleguide/part2/#inclusive_language> and let us know if any changes are needed.

For example, please consider whether "dummy" should be updated.
-->


Thank you.

RFC Editor/ap/jm


On 2/28/23 5:23 PM, [e-mail] wrote:

*****IMPORTANT*****

Updated 2023/02/28

RFC Author(s):
--------------

Instructions for Completing AUTH48

Your document has now entered AUTH48.  Once it has been reviewed and 
approved by you and all coauthors, it will be published as an RFC.  
If an author is no longer available, there are several remedies 
available as listed in the FAQ (https://www.rfc-editor.org/faq/).

You and you coauthors are responsible for engaging other parties 
(e.g., Contributors or Working Group) as necessary before providing 
your approval.

Planning your review 
---------------------

Please review the following aspects of your document:

*  RFC Editor questions

   Please review and resolve any questions raised by the RFC Editor 
   that have been included in the XML file as comments marked as 
   follows:

   <!-- [rfced] ... -->

   These questions will also be sent in a subsequent email.

*  Changes submitted by coauthors 

   Please ensure that you review any changes submitted by your 
   coauthors.  We assume that if you do not speak up that you 
   agree to changes submitted by your coauthors.

*  Content 

   Please review the full content of the document, as this cannot 
   change once the RFC is published.  Please pay particular attention to:
   - IANA considerations updates (if applicable)
   - contact information
   - references

*  Copyright notices and legends

   Please review the copyright notice and legends as defined in
   RFC 5378 and the Trust Legal Provisions 
   (TLP – https://trustee.ietf.org/license-info/).

*  Semantic markup

   Please review the markup in the XML file to ensure that elements of  
   content are correctly tagged.  For example, ensure that <sourcecode> 
   and <artwork> are set correctly.  See details at 
   <https://authors.ietf.org/rfcxml-vocabulary>.

*  Formatted output

   Please review the PDF, HTML, and TXT files to ensure that the 
   formatted output, as generated from the markup in the XML file, is 
   reasonable.  Please note that the TXT will have formatting 
   limitations compared to the PDF and HTML.


Submitting changes
------------------

To submit changes, please reply to this email using ‘REPLY ALL’ as all 
the parties CCed on this message need to see your changes. The parties 
include:

   *  your coauthors
   
   *  [e-mail] (the RPC team)

   *  other document participants, depending on the stream (e.g., 
      IETF Stream participants are your working group chairs, the 
      responsible ADs, and the document shepherd).
     
   *  [e-mail], which is a new archival mailing list 
      to preserve AUTH48 conversations; it is not an active discussion 
      list:
     
     *  More info:
        https://mailarchive.ietf.org/arch/msg/ietf-announce/yb6lpIGh-4Q9l2USxIAe6P8O4Zc
     
     *  The archive itself:
        https://mailarchive.ietf.org/arch/browse/auth48archive/

     *  Note: If only absolutely necessary, you may temporarily opt out 
        of the archiving of messages (e.g., to discuss a sensitive matter).
        If needed, please add a note at the top of the message that you 
        have dropped the address. When the discussion is concluded, 
        [e-mail] will be re-added to the CC list and 
        its addition will be noted at the top of the message. 

You may submit your changes in one of two ways:

An update to the provided XML file
 — OR —
An explicit list of changes in this format

Section # (or indicate Global)

OLD:
old text

NEW:
new text

You do not need to reply with both an updated XML file and an explicit 
list of changes, as either form is sufficient.

We will ask a stream manager to review and approve any changes that seem
beyond editorial in nature, e.g., addition of new text, deletion of text, 
and technical changes.  Information about stream managers can be found in 
the FAQ.  Editorial changes do not require approval from a stream manager.


Approving for publication
--------------------------

To approve your RFC for publication, please reply to this email stating
that you approve this RFC for publication.  Please use ‘REPLY ALL’,
as all the parties CCed on this message need to see your approval.


Files 
-----

The files are available here:
   https://www.rfc-editor.org/authors/rfc9411.xml
   https://www.rfc-editor.org/authors/rfc9411.html
   https://www.rfc-editor.org/authors/rfc9411.pdf
   https://www.rfc-editor.org/authors/rfc9411.txt

Diff file of the text:
   https://www.rfc-editor.org/authors/rfc9411-diff.html
   https://www.rfc-editor.org/authors/rfc9411-rfcdiff.html (side by side)

For your convenience, we have also created an alt-diff file that will 
allow you to more easily view changes where text has been deleted or 
moved: 
   https://www.rfc-editor.org/authors/rfc9411-alt-diff.html 

Diff of the XML: 
   https://www.rfc-editor.org/authors/rfc9411-xmldiff1.html

The following files are provided to facilitate creation of your own 
diff files of the XML.  

Initial XMLv3 created using XMLv2 as input:
   https://www.rfc-editor.org/authors/rfc9411.original.v2v3.xml 

XMLv3 file that is a best effort to capture v3-related format updates 
only: 
   https://www.rfc-editor.org/authors/rfc9411.form.xml


Tracking progress
-----------------

The details of the AUTH48 status of your document are here:
   https://www.rfc-editor.org/auth48/rfc9411

Please let us know if you have any questions.  

Thank you for your cooperation,

RFC Editor

--------------------------------------
RFC9411 (draft-ietf-bmwg-ngfw-performance-15)

Title            : Benchmarking Methodology for Network Security Device Performance
Author(s)        : B. Balarajah, C. Rossenhoevel, B. Monkman
WG Chair(s)      : Sarah Banks, Al Morton

Area Director(s) : Warren Kumari, Robert Wilton