Re: [bmwg] Usage of multiple IP address with RFC 2544 / RFC 5180 compliant throughput tests

[Gabor LENCSE <[e-mail]>]

Dear Sarah,

Thank you very much for your answer. Please see my comments inline.

On 7/6/2023 9:36 PM, [e-mail] wrote:
> Hi Lencse,
> My apologies for the long delay in this; I’d been looking for an RFC 
> to see if this was covered, and I can’t find one.

Thank you very much for checking it! If you did not find one, then I 
tend to believe that there is no such an RFC, and therefore, I invented 
something new. :-)

> However, my thoughts…
> How OpenBSD chooses to handle (or not) the RSS item is an 
> implementation detail; you know it, which is nice, but we can’t know 
> this for every system on the Internet.

My idea is the following. As we do black box testing, we should do our 
best to test all DUTs under fair conditions.

> Using 1 pseudorandom src/dest IP or using <n> of them seems no 
> different than what test ports would generate (albeit in this case, 
> they’re all pseudorandom I suppose).

The problem is that by default a single IP address pair was used:

      198.18.0.2/24                                       198.19.0.2/24
               \  +--------------------------------------+  /
                \ |                                      | /
    +-------------|                Tester                |<------------+
    |             |                                      |             |
    |             +--------------------------------------+             |
    |                                                                  |
    |             +--------------------------------------+             |
    |             |                                      |             |
    +------------>|          DUT: IPv4 router            |-------------+
                / |                                      | \
               /  +--------------------------------------+  \
     198.18.0.1/24                                        198.19.0.1/24

This is how for example the Anritsu MP1590B RFC 2544 compliant Network 
Performance Tester works. It is sends out the very same test frames 
following RFC 2544 Appendix C.2.6.4 test frame format. In this case, 
either under Linux or under OpenBSD the interrupts are hashed to two CPU 
cores (one per direction). Originally, my siitperf did the same. Then Al 
Morton called my attention to RFC 4814 and I have added the support for 
pseudorandom port numbers. Then, if the RSS is set properly on the Linux 
box, then the interrupts are hashed to all CPU cores. (The proper 
command is something like: ethtool -N /interface-name/ rx-flow-hash udp4 
sdfn) However, there is no such command under OpenBSD. And I feel that 
this is not fair: the measurement results will be biased against OpenBSD.

This is why I thought that then it is my responsibility to implement a 
Tester that can use pseudorandom IP addresses from the rages: 198.18.0.2 
-- 198.18.255.254 and 198.19.0.2 -- 198.19.255.254 on the left sided and 
on the right side of the Tester, respectively. Thus independently from 
the operating system (and on the RSS setting), all CPU cores can be 
utilized by the interrupts, as it likely happens when Internet traffic 
is forwarded.

Now I have finished the implementation. Currently I do some self-tests 
to determine its performance and then I plan to use it for benchmarking.

> If I’m missing something let me know, but I wonder if, if we’re 
> confident that there might be some exceptions where the system under 
> test in the lab can’t mimic close to production, then we note it in 
> the document and keep moving forward with the work. Thoughts?

Yes, I agree with you that there can be cases, when we must say that we 
are not able to closely mimic the production conditions. But I would 
like to do all my best. And, in this case, it was to do some programming 
work.

I will keep informed you about the results.

Our draft (draft-ietf-bmwg-benchmarking-stateful-03.txt) deals only with 
stateful NATxy gateways, and stateless (router) testing is out of its 
scope. But, perhaps it will be worth starting another one which 
addresses the stateless case and thus updates (or amends) RFC 2544 / RFC 
5180.

What do you think?

Best regards,

Gábor


>
> Thank you,
> Sarah
>
>> On Jun 5, 2023, at 2:12 AM, Lencse Gábor <[e-mail]> wrote:
>>
>> Dear All,
>>
>> Is there any RFC that describes how to use multiple IP addresses with 
>> RFC 2544 / RFC 5180 compliant throughput tests?
>>
>> Originally, RFC 2544 used fixed test frame format. Then RFC 4814 
>> specified how to use pseudorandom source and destination port 
>> numbers. However, I did not find any guidance, how one should use 
>> multiple, pseudorandom source and destination IP addresses.
>>
>> It is an issue, because in some systems, like OpenBSD, one cannot set 
>> the Receive-Side Scaling (RSS) to include also the port numbers into 
>> the hash tuple for distributing the interrupts of the incoming 
>> packets among the CPU cores. Please see this e-mail for the details: 
>> https://marc.info/?l=openbsd-misc&m=166581934723445&w=2
>>
>> However, when the system is used to forward Internet traffic, then it 
>> faces with different source and destination IP addresses. Thus, the 
>> laboratory test results will not reflect the results experienced 
>> under operational conditions.
>>
>> Now I am working on implementing the usage of pseudorandom source and 
>> destination IP addresses in siitperf, my stateful and stateless NAT64 
>> tester, which can also be used to benchmark IPv4 or IPv6 routers. My 
>> main motivation is to support stateful NAT64 tests with OpenBSD PF, 
>> but I also would like to support stateless IPv4 and IPv6 packet 
>> forwarding measurements.
>>
>> Regarding the stateless IPv4 tests, I planned this setup:
>>
>>      198.18.[2-65534]/16                            198.19.[2-65534]/16
>>                \  +--------------------------------------+  /
>>                 \ |                                      | /
>>     +-------------|                Tester                |<------------+
>>     |             |                                      |             |
>>     |             +--------------------------------------+             |
>>     |                                                                  |
>>     |             +--------------------------------------+             |
>>     |             |                                      |             |
>>     +------------>|          DUT: IPv4 router            |-------------+
>>                 / |                                      | \
>>                /  +--------------------------------------+  \
>>      198.18.0.1/16                                        198.19.0.1/16
>>
>> I am aware that this solution sacrifices the opportunity of testing 
>> with 256 destination networks.
>>
>> Any suggestions are welcome!
>>
>> Best regards,
>>
>> Gábor
>>
>>
>> _______________________________________________
>> bmwg mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/bmwg
>