[auth48] [AD] Re: AUTH48: RFC-to-be 9693 <draft-ietf-bmwg-benchmarking-stateful-09> for your review

[Alanna Paloma <[e-mail]>]

Hi Gábor and Warren (AD)*,

*Warren - As the AD, please review and approve of the updated key word in Section 4.4:

Original:
   [RFC4814] REQUIRES pseudorandom port numbers, which the authors
   believe is a good approximation of the distribution of the source
   port numbers a NATxy gateway on the Internet may face with.

Current:
   As described in [RFC4814], pseudorandom port numbers are REQUIRED,
   which the authors believe is a good approximation of the distribution
   of the source port numbers a NATxy gateway on the Internet may be
   faced with.

See this diff file:
 https://www.rfc-editor.org/authors/rfc9693-auth48diff.html 


Gábor - Thank you for your replies.  We have updated as requested. Please note that we have some follow-up queries. 

> But both of them are rather lengthy. Thus, the best title could be simply:
> "Benchmarking Methodology for Stateful NATxy Gateways"
> Rationale: This RFC will be the first one talking about benchmarking of stateleful NAT boxes. The proposed title expresses the topic clearly and it is concise (it does not contain anything unnecessary). It should be noted that the current version is partial: it mentions pseudorandom port numbers but it does not mention pseudorandom IP addresses. This is not logical.
> I discussed it with my co-author, Keiichi Shima and he agrees with the proposed title.
> Is it possible the change the title to this version?

) As “NATxy” has not appeared in previous RFCs and is defined in the Abstract, may we further update the title?

Current:
   Benchmarking Methodology for Stateful NATxy Gateways

Perhaps:
   Benchmarking Methodology for Stateful NAT Gateways


>> 8) <!-- [rfced] For clarity, how may we rephrase "it may be computing efficiently
>> generated by preparing" in the text below?
>> 
>> Original:
>> 
>> It may be computing efficiently generated by preparing a
>> random permutation of the previously enumerated all possible four
>> tuples using Durstenfeld's random shuffle algorithm [DUST1964].
>> 
>> Perhaps:
>> 
>> Efficient computing may be generated by preparing a
>> random permutation of the previously enumerated all possible four
>> tuples using Durstenfeld's random shuffle algorithm [DUST1964].
>> 
>> -->
>> 
> I am sorry, I do not understand your version. I think that the ambiguity was caused in the original text by the first word of the sentence: "It". To that end I would clarify the sentence as follows:
> New:
> Pseudorandom enumeration of all possible four tuples may be computing efficiently generated by preparing a
> random permutation of the previously enumerated all possible four
> tuples using Durstenfeld's random shuffle algorithm [DUST1964].

) We have updated the text per your suggestion; however, “may be computing efficiently generated by…” still reads awkwardly. May we further clarify the text as follows?

Perhaps:
   Pseudorandom enumeration of all possible four tuples may compute efficiently
   by using Durstenfeld’s random shuffle algorithm [DUST1964] to prepare a
   random permutation of the previously enumerated all possible four tuples. 


> In Section 6.
> 
> ORIGINAL: The table MUST be complemented with reporting the relevant parameters of the DUT. If the DUT is a general-purpose computer and some software NATxy gateway implementation is tested, then the hardware description SHOULD include: computer type, CPU type, and number of active CPU cores, memory type, size and speed, network interface card type (reflecting also the speed), the fact that direct cable connections were used or the type of the switch used for interconnecting the Tester and the DUT.
> 
> EDITED: The table MUST be complemented with reporting the relevant parameters of the DUT. If the DUT is a general-purpose computer and some software NATxy gateway implementation is tested, then the hardware description SHOULD include: the computer type, CPU type and number of active CPU cores, memory type, size and speed, network interface card type (also reflecting the speed), the fact that direct cable connections were used, and the type of the switch used for interconnecting the Tester and the DUT. 
> 
> Here the problem is, that the Tester and the DUT are connected either by direct cables or through a switch. The original text had an "or" but the edited text has an "and" between the two. I understand that the original text did not sound well. Could you please rephrase it so that its original meaning should remain?
> 
> Moreover, I found a mistake that is NOT a results of the editing, but it was present in the original text, and thus it was definitely my fault. Could you please correct it?


) We have updated the text and reformatted it to a bulleted list to improve readability. Please let us know if further updates are needed.

Current:
   The table MUST be complemented with reporting the relevant parameters
   of the DUT. If the DUT is a general-purpose computer and some
   software NATxy gateway implementation is tested, then the hardware
   description SHOULD include the following:

   * computer type

   * CPU type

   * number of active CPU cores

   * memory type

   * size and speed

   * network interface card type (also reflecting the speed)

   * direct cable connections or the type of switch used for
      interconnecting the Tester and the DUT

...
The files have been posted here (please refresh):
 https://www.rfc-editor.org/authors/rfc9693.xml
 https://www.rfc-editor.org/authors/rfc9693.txt
 https://www.rfc-editor.org/authors/rfc9693.html
 https://www.rfc-editor.org/authors/rfc9693.pdf

The relevant diff files have been posted here:
 https://www.rfc-editor.org/authors/rfc9693-diff.html (comprehensive diff)
 https://www.rfc-editor.org/authors/rfc9693-auth48diff.html (AUTH48 changes)

Please review the document carefully and contact us with any further updates you may have.  Note that we do not make changes once a document is published as an RFC.

We will await approvals from each party listed on the AUTH48 status page below prior to moving this document forward in the publication process.

For the AUTH48 status of this document, please see:
 https://www.rfc-editor.org/auth48/rfc9693

Thank you,
RFC Editor/ap

> On Dec 6, 2024, at 12:43 PM, Dr. Lencse Gábor <[e-mail]> wrote:
> 
> Dear RFC Editor,
> Please see my answers inline.
> On 04/12/2024 00:24, [e-mail] wrote:
>> Authors,
>> 
>> While reviewing this document during AUTH48, please resolve (as necessary) the following questions, which are also in the XML file.
>> 
>> 1) <!-- [rfced] As RFC 4814 is mentioned in this document's Abstract and
>> Introduction, may we remove the reference to it from the title?
>> 
>> Original:
>> 
>> Benchmarking Methodology for Stateful NATxy Gateways using RFC 4814
>> Pseudorandom Port Numbers
>> 
>> Perhaps: 
>> 
>> Benchmarking Methodology for Stateful NATxy Gateways Using
>> Pseudorandom Port Numbers
>> 
>> -->
>> 
> I can accept it. Moreover, as I was thinking about the title, I noticed something. Originally, the title reflected the content well: we used a single IP address pair and pseudorandom port numbers (as recommended by RFC 4814). Later we introduced the usage of multiple, pseudorandom IP addresses, too. However, we did not update the title to reflect the change.
> To that end, one could use one of the following titles:
> "Benchmarking Methodology for Stateful NATxy Gateways Using Pseudorandom Port Numbers and Pseudorandom IP Addresses"
> or
> "Benchmarking Methodology for Stateful NATxy Gateways Using Pseudorandom Port Numbers and IP Addresses"
> But both of them are rather lengthy. Thus, the best title could be simply:
> "Benchmarking Methodology for Stateful NATxy Gateways"
> Rationale: This RFC will be the first one talking about benchmarking of stateleful NAT boxes. The proposed title expresses the topic clearly and it is concise (it does not contain anything unnecessary). It should be noted that the current version is partial: it mentions pseudorandom port numbers but it does not mention pseudorandom IP addresses. This is not logical.
> I discussed it with my co-author, Keiichi Shima and he agrees with the proposed title.
> Is it possible the change the title to this version?
> 
>> 
>> 2) <!-- [rfced] Per Section 3.6 of RFC 7322 ("RFC Style Guide"), abbreviations
>> must be expanded upon first use. To avoid expanding "NAPT" upon first use
>> here and stacking multiple sets of parentheses, we have rephrased as
>> follows (because "NAPT" is introduced and expanded later in this document).
>> Please let us know of any objections.
>> 
>> Original:
>> 
>> However, none of them discussed, how to apply [RFC4814] pseudorandom
>> port numbers, when benchmarking stateful NATxy (NAT44 (also called
>> NAPT) [RFC3022], NAT64 [RFC6146], and NAT66) gateways. (It should be
>> noted that stateful NAT66 is not an IETF specification but refers to
>> an IPv6 version of the stateful NAT44 specification.)
>> 
>> Current:
>> 
>> However, none of them discussed how to apply pseudorandom port numbers from
>> [RFC4814] when benchmarking stateful NATxy gateways (such as NAT44
>> [RFC3022], NAT64 [RFC6146], and NAT66). (It should be
>> noted that stateful NAT66 is not an IETF specification but refers to
>> an IPv6 version of the stateful NAT44 specification.)
>> 
>> -->
>> 
> OK.
>> 
>> 3) <!-- [rfced] In the sentence below, may we clarify "also in the case of
>> UDP using the same kind of entries as in the case of TCP" as follows?
>> 
>> Original:
>> 
>> * Connection: Although UDP itself is a connection-less protocol,
>> stateful NATxy gateways keep track of their translation mappings
>> in the form of a "connection" also in the case of UDP using the
>> same kind of entries as in the case of TCP.
>> 
>> Perhaps:
>> 
>> Connection: Although UDP itself is a connectionless protocol,
>> stateful NATxy gateways keep track of their translation mappings
>> in the form of a "connection" as well as in the case of UDP using the
>> same kind of entries as in TCP.
>> 
>> -->
>> 
> I am not a native speaker. If it sounds better, I am OK with it.
>> 
>> 4) <!-- [rfced] For clarity, may we update "in the order of a few times ten
>> thousand" to "in the order of a few tens of thousands"?
>> 
>> Original:
>> 
>> If it is possible, the size of the source port number range SHOULD be
>> larger (e.g. in the order of a few times ten thousand), whereas the size of
>> the destination port number range SHOULD be smaller (may vary from a few to
>> several hundreds or thousands as needed).
>> 
>> 
>> Perhaps:
>> 
>> If it is possible, the size of the source port number range SHOULD be
>> larger (e.g., in the order of a few tens of thousands), whereas the size of
>> the destination port number range SHOULD be smaller (may vary from a few to
>> several hundreds or thousands as needed).
>> 
>> -->
>> 
> I am not a native speaker, but I trust in you. 
> Just for clarity: - If there are 1,000 apples in a container, and we have a few containers, I would say: We have a few thousands of apples. - If there are 10,000 apples in a container, and we have a few containers, I would say: We have a few times 10,000 of apples. (This is the literal translation of what I would say in Hungarian.)
> In the latter case, would a native speaker say: "We have a few tens of thousands of apples."?
> 
>> 5) <!-- [rfced] FYI - To improve readability, we have reformatted the text below
>> to read as a bulleted list. Please let us know any objections.
>> 
>> Original:
>> 
>> When multiple IP addresses are used, then the port number ranges
>> should be even more restricted, as the number of potential network
>> flows is the product of the size of the source IP address range, the
>> size of the source port number range, the size of the destination IP
>> address range, and the size of the destination port number range.
>> 
>> Current:
>> 
>> When multiple IP addresses are used, then the port number ranges
>> should be even more restricted, as the number of potential network
>> flows is the product of the size of:
>> 
>> * the source IP address range,
>> 
>> * the source port number range,
>> 
>> * the destination IP address range, and
>> 
>> * the destination port number range.
>> 
>> -->
>> 
> Yes, I like it. I think this formatting helps the understanding.
>> 
>> 6) <!-- [rfced] How may we clarify "that is throughput" in the text below?
>> 
>> Original:
>> 
>> Test phase 1 serves two purposes:
>> 
>> 1. The connection tracking table of the DUT is filled. It is
>> important, because its maximum connection establishment rate may
>> be lower than its maximum frame forwarding rate (that is
>> throughput).
>> 
>> Perhaps:
>> 
>> Test phase 1 serves two purposes:
>> 
>> 1. The connection tracking table of the DUT is filled. This is important
>> because its maximum connection establishment rate may be lower than its
>> maximum frame forwarding rate (that is, its throughput).
>> 
>> -->
>> 
> Perfect. 
> 
>> 7) <!--[rfced] As "REQUIRES" is not a key word per RFCs 2119/8174, may we
>> rephrase this sentence to use "REQUIRED"?
>> 
>> Original:
>> 
>> [RFC4814] REQUIRES pseudorandom port numbers, which the authors
>> believe is a good approximation of the distribution of the source
>> port numbers a NATxy gateway on the Internet may face with.
>> 
>> Perhaps:
>> 
>> As described in [RFC4814], pseudorandom port numbers are REQUIRED,
>> which the authors believe is a good approximation of the distribution
>> of the source port numbers a NATxy gateway on the Internet may face with.
>> 
>> -->
>> 
> Yes, this was my intention. Thank you for the correction.
>> 
>> 8) <!-- [rfced] For clarity, how may we rephrase "it may be computing efficiently
>> generated by preparing" in the text below?
>> 
>> Original:
>> 
>> It may be computing efficiently generated by preparing a
>> random permutation of the previously enumerated all possible four
>> tuples using Durstenfeld's random shuffle algorithm [DUST1964].
>> 
>> Perhaps:
>> 
>> Efficient computing may be generated by preparing a
>> random permutation of the previously enumerated all possible four
>> tuples using Durstenfeld's random shuffle algorithm [DUST1964].
>> 
>> -->
>> 
> I am sorry, I do not understand your version. I think that the ambiguity was caused in the original text by the first word of the sentence: "It". To that end I would clarify the sentence as follows:
> New:
> Pseudorandom enumeration of all possible four tuples may be computing efficiently generated by preparing a
> random permutation of the previously enumerated all possible four
> tuples using Durstenfeld's random shuffle algorithm [DUST1964].
> 
> 
>> 
>> 9) <!-- [rfced] FYI - We have reformatted the text below to read as a bulleted
>> list to improve readability. Please review and let us know of any objections.
>> 
>> Original:
>> 
>> Procedure: The Initiator sends a specific number of test frames using
>> all different four tuples at a specific rate through the DUT. The
>> Responder counts the frames that are successfully translated by the
>> DUT. If the count of offered frames is equal to the count of
>> received frames, the rate of the offered stream is raised and the
>> test is rerun. If fewer frames are received than were transmitted,
>> the rate of the offered stream is reduced and the test is rerun.
>> 
>> Current:
>> 
>> The procedure is as follows:
>> 
>> * The Initiator sends a specific number of test frames using all
>> different four tuples at a specific rate through the DUT.
>> 
>> * The Responder counts the frames that are successfully translated
>> by the DUT.
>> 
>> * If the count of offered frames is equal to the count of received
>> frames, the rate of the offered stream is raised and the test is
>> rerun.
>> 
>> * If fewer frames are received than were transmitted, the rate of
>> the offered stream is reduced and the test is rerun.
>> 
>> -->
>> 
> Thank you! I definitely agree with it.
>> 10) <!-- [rfced] Please review whether any of the notes in this document
>> should be in the <aside> element. It is defined as "a container for 
>> content that is semantically less important or tangential to the 
>> content that surrounds it"
>> (https://authors.ietf.org/en/rfcxml-vocabulary#aside)
>> -->
>> 
> I prefer not using it.
>> 
>> 11) <!--[rfced] May we clarify the singular/plural usage in this sentence as
>> follows??
>> 
>> Original:
>> 
>> ...but it is RECOMMENDED to perform measurement series
>> through which the value of one or more parameter(s) is/are changed to
>> discover how the various values of the given parameter(s) influence
>> the performance of the DUT.
>> 
>> Perhaps:
>> 
>> ...but it is RECOMMENDED to perform measurement series
>> through which the value of each parameter is changed to
>> discover how the various values of the each given parameter influences
>> the performance of the DUT.
>> 
>> --> 
>> 
> I disagree with this solution. 
> There may be several parameters. In a given measurement series, only one of them changes. In another one, two of them changes.
> I saw that in the edited version that "is/are" was replaced by "are". I did not object it in my previous e-mail, because I thought that it could be better grammatically than my solution, but I feel that "is/are" fits better to the subjective ("one or more parameter(s)").
> 
>> 12) <!-- [rfced] FYI - We have added expansions for abbreviations upon first use
>> per Section 3.6 of RFC 7322 ("RFC Style Guide"). Please review each
>> expansion in the document carefully to ensure correctness.
>> 
>> Border Relay (BR) 
>> Mapping of Address and Port using Translation (MAP-T)
>> Datagram Congestion Control Protocol (DCCP)
>> Stream Control Transmission Protocol (SCTP)
>> 
>> -->
>> 
> Thank you.
>> 
>> 13) <!-- [rfced] Please review the "Inclusive Language" portion of the online 
>> Style Guide <https://www.rfc-editor.org/styleguide/part2/#inclusive_language>
>> and let us know if any changes are needed. Updates of this nature typically
>> result in more precise language, which is helpful for readers.
>> 
>> a. For example, please consider whether "black" should be updated.
> "Black box testing" should be kept. It is a well-known and widely-used term.
>> 
>> b. In addition, please consider whether "tradition" and "traditional" should
>> be updated for clarity. While the NIST website 
>> <https://www.nist.gov/nist-research-library/nist-technical-series-publications-author-instructions#table1> 
>> indicates that this term is potentially biased, it is also ambiguous. 
>> "Tradition" is a subjective term, as it is not the same for everyone.
>> 
>> -->
>> 
> The term "Traditional NAT" was defined by [RFC3022].
>> 
>> 
>> Thank you.
>> 
>> RFC Editor/kf/ap
>> 
> Once again, thank you very much for your careful editing work!
> Best regards,
> Gábor Lencse
> 
>> 
>> On Dec 3, 2024, at 3:23 PM, [e-mail] wrote:
>> 
>> *****IMPORTANT*****
>> 
>> Updated 2024/12/03
>> 
>> RFC Author(s):
>> --------------
>> 
>> Instructions for Completing AUTH48
>> 
>> Your document has now entered AUTH48. Once it has been reviewed and 
>> approved by you and all coauthors, it will be published as an RFC. 
>> If an author is no longer available, there are several remedies 
>> available as listed in the FAQ (https://www.rfc-editor.org/faq/)
>> 
>> You and you coauthors are responsible for engaging other parties 
>> (e.g., Contributors or Working Group) as necessary before providing 
>> your approval.
>> 
>> Planning your review 
>> ---------------------
>> 
>> Please review the following aspects of your document:
>> 
>> * RFC Editor questions
>> 
>> Please review and resolve any questions raised by the RFC Editor 
>> that have been included in the XML file as comments marked as 
>> follows:
>> 
>> <!-- [rfced] ... -->
>> 
>> These questions will also be sent in a subsequent email.
>> 
>> * Changes submitted by coauthors 
>> 
>> Please ensure that you review any changes submitted by your 
>> coauthors. We assume that if you do not speak up that you 
>> agree to changes submitted by your coauthors.
>> 
>> * Content 
>> 
>> Please review the full content of the document, as this cannot 
>> change once the RFC is published. Please pay particular attention to:
>> - IANA considerations updates (if applicable)
>> - contact information
>> - references
>> 
>> * Copyright notices and legends
>> 
>> Please review the copyright notice and legends as defined in
>> RFC 5378 and the Trust Legal Provisions 
>> (TLP – https://trustee.ietf.org/license-info)
>> 
>> * Semantic markup
>> 
>> Please review the markup in the XML file to ensure that elements of 
>> content are correctly tagged. For example, ensure that <sourcecode> 
>> and <artwork> are set correctly. See details at 
>> <https://authors.ietf.org/rfcxml-vocabulary>.
>> 
>> * Formatted output
>> 
>> Please review the PDF, HTML, and TXT files to ensure that the 
>> formatted output, as generated from the markup in the XML file, is 
>> reasonable. Please note that the TXT will have formatting 
>> limitations compared to the PDF and HTML.
>> 
>> 
>> Submitting changes
>> ------------------
>> 
>> To submit changes, please reply to this email using ‘REPLY ALL’ as all 
>> the parties CCed on this message need to see your changes. The parties 
>> include:
>> 
>> * your coauthors
>> 
>> * [e-mail] (the RPC team)
>> 
>> * other document participants, depending on the stream (e.g., 
>> IETF Stream participants are your working group chairs, the 
>> responsible ADs, and the document shepherd).
>> 
>> * [e-mail], which is a new archival mailing list 
>> to preserve AUTH48 conversations; it is not an active discussion 
>> list:
>> 
>> * More info:
>> https://mailarchive.ietf.org/arch/msg/ietf-announce/yb6lpIGh-4Q9l2USxIAe6P8O4Zc
>> 
>> * The archive itself:
>> https://mailarchive.ietf.org/arch/browse/auth48archive/
>> 
>> * Note: If only absolutely necessary, you may temporarily opt out 
>> of the archiving of messages (e.g., to discuss a sensitive matter).
>> If needed, please add a note at the top of the message that you 
>> have dropped the address. When the discussion is concluded, 
>> [e-mail] will be re-added to the CC list and 
>> its addition will be noted at the top of the message. 
>> 
>> You may submit your changes in one of two ways:
>> 
>> An update to the provided XML file
>> — OR —
>> An explicit list of changes in this format
>> 
>> Section # (or indicate Global)
>> 
>> OLD:
>> old text
>> 
>> NEW:
>> new text
>> 
>> You do not need to reply with both an updated XML file and an explicit 
>> list of changes, as either form is sufficient.
>> 
>> We will ask a stream manager to review and approve any changes that seem
>> beyond editorial in nature, e.g., addition of new text, deletion of text, 
>> and technical changes. Information about stream managers can be found in 
>> the FAQ. Editorial changes do not require approval from a stream manager.
>> 
>> 
>> Approving for publication
>> --------------------------
>> 
>> To approve your RFC for publication, please reply to this email stating
>> that you approve this RFC for publication. Please use ‘REPLY ALL’,
>> as all the parties CCed on this message need to see your approval.
>> 
>> 
>> Files 
>> -----
>> 
>> The files are available here:
>> https://www.rfc-editor.org/authors/rfc9693.xml
>> https://www.rfc-editor.org/authors/rfc9693.html
>> https://www.rfc-editor.org/authors/rfc9693.pdf
>> https://www.rfc-editor.org/authors/rfc9693.txt
>> 
>> Diff file of the text:
>> https://www.rfc-editor.org/authors/rfc9693-diff.html
>> https://www.rfc-editor.org/authors/rfc9693-rfcdiff.html (side by side)
>> 
>> Diff of the XML: 
>> https://www.rfc-editor.org/authors/rfc9693-xmldiff1.html
>> 
>> 
>> Tracking progress
>> -----------------
>> 
>> The details of the AUTH48 status of your document are here:
>> https://www.rfc-editor.org/auth48/rfc9693
>> 
>> Please let us know if you have any questions. 
>> 
>> Thank you for your cooperation,
>> 
>> RFC Editor
>> 
>> --------------------------------------
>> RFC9693 (draft-ietf-bmwg-benchmarking-stateful-09)
>> 
>> Title : Benchmarking Methodology for Stateful NATxy Gateways using RFC 4814 Pseudorandom Port Numbers
>> Author(s) : G. Lencse, K. Shima
>> WG Chair(s) : Sarah Banks, Giuseppe Fioccola
>> 
>> Area Director(s) : Warren Kumari, Mahesh Jethanandani
>> 
>> 
>>