Re: [bmwg] Scalability measurement results of the Jool stateful NAT64 solution

[Gábor LENCSE <[e-mail]>]

Dear Al,

3/3/2022 4:20 PM keltezéssel, MORTON JR., AL írta:
>
> Hi Gábor,
>
> I read through your measurements this morning, thanks for sharing them.
>

Thank you very much for being interested in them!

> Some small suggestions:
>
> In the tables of results, I thought it might be useful to have a table 
> that explains the abbreviations you used in the left-hand column. For 
> example, the abbreviation for connections per second, cps, doesn’t 
> appear in the text, and I had to look-around to find that throughput 
> is expressed in units of packets per second. But otherwise the tables 
> of results are very useful. You expressed the variation of repeated 
> measurements, etc.
>

Yes, it was really missing. Did you think something like this one?

    abbreviation          explanation
    ------------          -----------
    num. CPU cores        number of CPU cores
    src ports             size of the source port number range
    dst ports             size of the destination port number range
    num. conn.            number of connections = src ports * dst ports
    conntrack t. s.       size of the connection tracking table of the
                          DUT
    hash table size       size of the hash table of the DUT
    c.t.s/num.conn.       conntrack table size / number of connections
    num. experiments      number of experiments
    error                 the difference between the upper and the lower
                          bound of the binary search when it stops
    cps (median/min/max)  maximum connection establishment rate
                          (median, minimum, maximum)
    cps rel. scale up     the relative scale up of the maximum connection
                          establishment rate against the number of CPU
                          cores
    tp. rel. scale up     the relative scale up of the throughput
    throughput ratio (%)  the ratio of the throughput of iptables and the
                          throughput of IPv4 packet forwarding

        Figure 3: Explanation of the abbreviations for the scale up of
                   iptables against the number of CPU cores


I prepared it only as a sample. I plan to prepare similar ones for the 
other tables in the next version.


> Also, it would be good to use network addresses from the benchmarking 
> v4 and v6 address space. The nits check is flagging the current 
> addresses, which may have been the actual addresses used in your 
> experiment (and that’s normal practice, but nits will continue to flag 
> them unless you substitute...). See below.
>

In theory, I agree with your suggestion, however, I think that the usage 
of the documentation IP addresses would be inappropriate in this special 
case.
Why?
- I used the 10.0.0.1 and 10.0.0.2 private IPv4 addresses to express 
that "this is the private side of the NAT44 box".
- I used the 198.19.0.1 and 198.19.0.2 IPv4 addresses as well as the 
2001:2::1 and 2001:2::1 IPv6 addresses to express that "this drawing 
depicts a benchmarking setup".

What do you think of it?

Best regards,

Gábor

> Al
>
> Checking nits according to https://www.ietf.org/id-info/checklist :
>
> ----------------------------------------------------------------------------
>
> == There are 2 instances of lines with non-RFC6890-compliant IPv4 
> addresses
>
> in the document.If these are example addresses, they should be changed.
>
> == There are 2 instances of lines with private range IPv4 addresses in the
>
> document.If these are generic example addresses, they should be changed
>
> to use any of the ranges defined in RFC 6890 (or successor): 192.0.2.x,
>
> 198.51.100.x or 203.0.113.x.
>
> == There are 1 instance of lines with non-RFC3849-compliant IPv6 addresses
>
> in the document.If these are example addresses, they should be changed.
>
> *From:*bmwg <[e-mail]> *On Behalf Of *Gábor LENCSE
> *Sent:* Monday, February 21, 2022 4:01 PM
> *To:* [e-mail]
> *Subject:* [bmwg] Scalability measurement results of the Jool stateful 
> NAT64 solution
>
> Dear All,
>
>
> As an application of you draft "Benchmarking Methodology for Stateful 
> NATxy Gateways using RFC 4814 Pseudorandom Port Numbers", I have 
> measured the peformance of the Jool stateful NAT64 implementation. I 
> have examined, how its performance metrics (maximum connection 
> establisment rate and throughput) scale up with the number of CPU 
> cores, and how the number of connections degrade its performance. 
> (Jool scaled up much less well than iptables.)
>
> Taday I have uploaded the updated version of my v6ops I-D about the 
> results. You can find the new results in its Section 3: 
> https://datatracker.ietf.org/doc/html/draft-lencse-v6ops-transition-scalability-01#section-3 
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-lencse-v6ops-transition-scalability-01*section-3__;Iw!!BhdT!kifrO2k7qtNtx_d0VHH5xBEEO8NQ3xajmXH03ZaWpd7lqGLB0DjiL7-PUqRHWKSsdaUgIac7g-9L0jU$>
>
>
> Any comments, recommendations, questions, etc. are welcome!
>
> We plan to update our BMWG draft about the methodology in 1-2 weeks 
> (by adding the method for measuring connection tear down rate).
>
> Best regards,
>
> Gábor
>
> -------- Továbbított üzenet --------
>
> *Tárgy: *
>
> 	
>
> New Version Notification for 
> draft-lencse-v6ops-transition-scalability-01.txt
>
> *Dátum: *
>
> 	
>
> Mon, 21 Feb 2022 05:13:54 -0800
>
> *Feladó: *
>
> 	
>
> [e-mail]
>
> *Címzett: *
>
> 	
>
> Gabor Lencse <[e-mail]> <mailto:[e-mail]>
>
>
>
>
> A new version of I-D, draft-lencse-v6ops-transition-scalability-01.txt
> has been successfully submitted by Gabor Lencse and posted to the
> IETF repository.
>
> Name: draft-lencse-v6ops-transition-scalability
> Revision: 01
> Title: Scalability of IPv6 Transition Technologies for IPv4aaS
> Document date: 2022-02-21
> Group: Individual Submission
> Pages: 12
> URL: 
> https://www.ietf.org/archive/id/draft-lencse-v6ops-transition-scalability-01.txt 
> <https://urldefense.com/v3/__https:/www.ietf.org/archive/id/draft-lencse-v6ops-transition-scalability-01.txt__;!!BhdT!kifrO2k7qtNtx_d0VHH5xBEEO8NQ3xajmXH03ZaWpd7lqGLB0DjiL7-PUqRHWKSsdaUgIac7ifFqxTI$>
> Status: 
> https://datatracker.ietf.org/doc/draft-lencse-v6ops-transition-scalability/ 
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-lencse-v6ops-transition-scalability/__;!!BhdT!kifrO2k7qtNtx_d0VHH5xBEEO8NQ3xajmXH03ZaWpd7lqGLB0DjiL7-PUqRHWKSsdaUgIac7si8Vb2k$>
> Htmlized: 
> https://datatracker.ietf.org/doc/html/draft-lencse-v6ops-transition-scalability 
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-lencse-v6ops-transition-scalability__;!!BhdT!kifrO2k7qtNtx_d0VHH5xBEEO8NQ3xajmXH03ZaWpd7lqGLB0DjiL7-PUqRHWKSsdaUgIac7MZBB174$>
> Diff: 
> https://www.ietf.org/rfcdiff?url2=draft-lencse-v6ops-transition-scalability-01 
> <https://urldefense.com/v3/__https:/www.ietf.org/rfcdiff?url2=draft-lencse-v6ops-transition-scalability-01__;!!BhdT!kifrO2k7qtNtx_d0VHH5xBEEO8NQ3xajmXH03ZaWpd7lqGLB0DjiL7-PUqRHWKSsdaUgIac7ABCA1pM$>
>
> Abstract:
> Several IPv6 transition technologies have been developed to provide
> customers with IPv4-as-a-Service (IPv4aaS) for ISPs with an IPv6-only
> access and/or core network. All these technologies have their
> advantages and disadvantages, and depending on existing topology,
> skills, strategy and other preferences, one of these technologies may
> be the most appropriate solution for a network operator.
>
> This document examines the scalability of the five most prominent
> IPv4aaS technologies (464XLAT, Dual Stack Lite, Lightweight 4over6,
> MAP-E, MAP-T) considering two aspects: (1) how their performance
> scales up with the number of CPU cores, (2) how their performance
> degrades, when the number of concurrent sessions is increased until
> hardware limit is reached.
>
>
>
> The IETF Secretariat
>