Re: [bmwg] Scalability measurement results of the Jool stateful NAT64 solution

["MORTON JR., AL" <[e-mail]>]

Hi Gábor, please see below,
Al

From: Gábor LENCSE <[e-mail]>
Sent: Monday, March 7, 2022 4:14 PM
To: MORTON JR., AL <[e-mail]>; [e-mail]
Subject: Re: [bmwg] Scalability measurement results of the Jool stateful NAT64 solution

Dear Al,
3/3/2022 4:20 PM keltezéssel, MORTON JR., AL írta:
Hi Gábor,

I read through your measurements this morning, thanks for sharing them.

Thank you very much for being interested in them!



Some small suggestions:

In the tables of results, I thought it might be useful to have a table that explains the abbreviations you used in the left-hand column. For example, the abbreviation for connections per second, cps, doesn’t appear in the text, and I had to look-around to find that throughput is expressed in units of packets per second. But otherwise the tables of results are very useful. You expressed the variation of repeated measurements, etc.

Yes, it was really missing. Did you think something like this one?

   abbreviation          explanation

   ------------          -----------

   num. CPU cores        number of CPU cores

   src ports             size of the source port number range

   dst ports             size of the destination port number range

   num. conn.            number of connections = src ports * dst ports

   conntrack t. s.       size of the connection tracking table of the

                         DUT

   hash table size       size of the hash table of the DUT

   c.t.s/num.conn.       conntrack table size / number of connections

   num. experiments      number of experiments

   error                 the difference between the upper and the lower

                         bound of the binary search when it stops

   cps (median/min/max)  maximum connection establishment rate

                         (median, minimum, maximum)

   cps rel. scale up     the relative scale up of the maximum connection

                         establishment rate against the number of CPU

                         cores

   tp. rel. scale up     the relative scale up of the throughput

   throughput ratio (%)  the ratio of the throughput of iptables and the

                         throughput of IPv4 packet forwarding



       Figure 3: Explanation of the abbreviations for the scale up of

                  iptables against the number of CPU cores

I prepared it only as a sample. I plan to prepare similar ones for the other tables in the next version.
[acm]
That’s great. I’m sure that if I didn’t ask, someone else who was interested in your work would have...  They will appreciate your extra effort.

Also, it would be good to use network addresses from the benchmarking v4 and v6 address space. The nits check is flagging the current addresses, which may have been the actual addresses used in your experiment (and that’s normal practice, but nits will continue to flag them unless you substitute...). See below.

In theory, I agree with your suggestion, however, I think that the usage of the documentation IP addresses would be inappropriate in this special case.
Why?
- I used the 10.0.0.1 and 10.0.0.2 private IPv4 addresses to express that "this is the private side of the NAT44 box".
- I used the 198.19.0.1 and 198.19.0.2 IPv4 addresses as well as the 2001:2::1 and 2001:2::1 IPv6 addresses to express that "this drawing depicts a benchmarking setup".

What do you think of it?
[acm]
I think that the nits-checker doesn’t recognize the BMWG address space as “ok for drafts”, so that’s fine.
You might add the sentence about Net10 private network addresses in your draft, and see how far that will go...

thanks for your reply!


Best regards,

Gábor



Al

  Checking nits according to https://www.ietf.org/id-info/checklist<https://urldefense.com/v3/__https:/www.ietf.org/id-info/checklist__;!!BhdT!iCKjq5hOhZCwjYwjkgNuvukUrmo0e-HSynwO4-Hr21cqJU-5MxYgjbZyk2kNzkwqTybBYFxsB9N0Yrs$> :
  ----------------------------------------------------------------------------

  == There are 2 instances of lines with non-RFC6890-compliant IPv4 addresses
     in the document.  If these are example addresses, they should be changed.

  == There are 2 instances of lines with private range IPv4 addresses in the
     document.  If these are generic example addresses, they should be changed
     to use any of the ranges defined in RFC 6890 (or successor): 192.0.2.x,
     198.51.100.x or 203.0.113.x.

  == There are 1 instance of lines with non-RFC3849-compliant IPv6 addresses
     in the document.  If these are example addresses, they should be changed.





From: bmwg <[e-mail]><mailto:[e-mail]> On Behalf Of Gábor LENCSE
Sent: Monday, February 21, 2022 4:01 PM
To: [e-mail]<mailto:[e-mail]>
Subject: [bmwg] Scalability measurement results of the Jool stateful NAT64 solution

Dear All,

As an application of you draft "Benchmarking Methodology for Stateful NATxy Gateways using RFC 4814 Pseudorandom Port Numbers", I have measured the peformance of the Jool stateful NAT64 implementation. I have examined, how its performance metrics (maximum connection establisment rate and throughput) scale up with the number of CPU cores, and how the number of connections degrade its performance. (Jool scaled up much less well than iptables.)

Taday I have uploaded the updated version of my v6ops I-D about the results. You can find the new results in its Section 3: https://datatracker.ietf.org/doc/html/draft-lencse-v6ops-transition-scalability-01#section-3<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-lencse-v6ops-transition-scalability-01*section-3__;Iw!!BhdT!kifrO2k7qtNtx_d0VHH5xBEEO8NQ3xajmXH03ZaWpd7lqGLB0DjiL7-PUqRHWKSsdaUgIac7g-9L0jU$>

Any comments, recommendations, questions, etc. are welcome!

We plan to update our BMWG draft about the methodology in 1-2 weeks (by adding the method for measuring connection tear down rate).

Best regards,

Gábor

-------- Továbbított üzenet --------
Tárgy:
New Version Notification for draft-lencse-v6ops-transition-scalability-01.txt
Dátum:
Mon, 21 Feb 2022 05:13:54 -0800
Feladó:
[e-mail]<mailto:[e-mail]>
Címzett:
Gabor Lencse <[e-mail]><mailto:[e-mail]>



A new version of I-D, draft-lencse-v6ops-transition-scalability-01.txt
has been successfully submitted by Gabor Lencse and posted to the
IETF repository.

Name: draft-lencse-v6ops-transition-scalability
Revision: 01
Title: Scalability of IPv6 Transition Technologies for IPv4aaS
Document date: 2022-02-21
Group: Individual Submission
Pages: 12
URL: https://www.ietf.org/archive/id/draft-lencse-v6ops-transition-scalability-01.txt<https://urldefense.com/v3/__https:/www.ietf.org/archive/id/draft-lencse-v6ops-transition-scalability-01.txt__;!!BhdT!kifrO2k7qtNtx_d0VHH5xBEEO8NQ3xajmXH03ZaWpd7lqGLB0DjiL7-PUqRHWKSsdaUgIac7ifFqxTI$>
Status: https://datatracker.ietf.org/doc/draft-lencse-v6ops-transition-scalability/<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-lencse-v6ops-transition-scalability/__;!!BhdT!kifrO2k7qtNtx_d0VHH5xBEEO8NQ3xajmXH03ZaWpd7lqGLB0DjiL7-PUqRHWKSsdaUgIac7si8Vb2k$>
Htmlized: https://datatracker.ietf.org/doc/html/draft-lencse-v6ops-transition-scalability<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft-lencse-v6ops-transition-scalability__;!!BhdT!kifrO2k7qtNtx_d0VHH5xBEEO8NQ3xajmXH03ZaWpd7lqGLB0DjiL7-PUqRHWKSsdaUgIac7MZBB174$>
Diff: https://www.ietf.org/rfcdiff?url2=draft-lencse-v6ops-transition-scalability-01<https://urldefense.com/v3/__https:/www.ietf.org/rfcdiff?url2=draft-lencse-v6ops-transition-scalability-01__;!!BhdT!kifrO2k7qtNtx_d0VHH5xBEEO8NQ3xajmXH03ZaWpd7lqGLB0DjiL7-PUqRHWKSsdaUgIac7ABCA1pM$>

Abstract:
Several IPv6 transition technologies have been developed to provide
customers with IPv4-as-a-Service (IPv4aaS) for ISPs with an IPv6-only
access and/or core network. All these technologies have their
advantages and disadvantages, and depending on existing topology,
skills, strategy and other preferences, one of these technologies may
be the most appropriate solution for a network operator.

This document examines the scalability of the five most prominent
IPv4aaS technologies (464XLAT, Dual Stack Lite, Lightweight 4over6,
MAP-E, MAP-T) considering two aspects: (1) how their performance
scales up with the number of CPU cores, (2) how their performance
degrades, when the number of concurrent sessions is increased until
hardware limit is reached.



The IETF Secretariat