Re: [bmwg] How to apply RFC 4814 pseudorandom source and destination port numbers, when benchmarking tunneling technologies?

[David Newman <[e-mail]>]

We did not consider middleboxes when writing 4814, which was intended to
address deficiencies in benchmarking of one or a small set of switches
and routers.

Per standard bmwg practice at the time, we treated the DUT/SUT as a
single black box, where ingress and egress ports didn't know or care
what happened inside the box.

However, a possible problem with the pseudorandom-everywhere approach
can arise when ingress/egress ports and one or more intermediate
tunneling mechanisms require different and possibly conflicting patterns
to obtain optimal traffic distributions at tunneling vs. egress ports.

Obviously, if you're testing some sort of encapsulation scheme, then the
specifics of that protocol are fixed, but you, as the tester, need to
understand the traffic you're generating and whether it represents a
fair test for arbitrary boxes or has been specifically tailored to your
device or system.

The spirit of 4814 is to use your knowledge of the test environment and
protocols to provide a way to  scale test traffic that is free from bias
when used to compare arbitrary boxes that perform the function that is
tested.

The original intent was to scale endpoints, e.g. MAC addresses, IP
addresses, etc. If you're testing a protocol that is between two fixed
endpoints, then scaling addresses might not be a reasonable approach for
traffic scaling, even if it provides better performance because hashing
is easier.

You really have to use your best judgement, as a knowledgeable tester,
to be fair and not cheat.

Also, a wording nit: We have a strong allergy to the term "real-life" in
the context of bmwg benchmarking. As former bmwg chair Scott Bradner
once asked, "This begs the question: Real for whom?"

As with approaches to encapsulation, there is no valid one-size-fits-all
answer to that question.

dn and tcp


On 9/19/20 1:08 PM, Lencse Gábor wrote:
> Dear Authors of RFC 4814 and also BMWG Members,
> 
> RFC 4814 Section 4.5 recommends to use pseudorandom source and
> destination port numbers. My experience show that their usage is an
> essential precondition to receive realistic results, when multi-core
> devices are benchmarked. (I have recently added this feature to
> siitperf, my DPDK-based RFC 8219 compliant SIIT tester:
> https://github.com/lencsegabor/siitperf/tree/varport Please see my
> report on the BMWG mailing list for the details.)
> 
> However, I am not sure, how random ports should be used, when tunneling
> technologies are benchmarked. My MSc student is working on the
> implementation of a DS-Lite tester. He is going to build two separate
> Testers for the B4 and for the AFTR parts. For simplicity, let us
> consider B4 first (as the stateful nature of the AFTR further
> complicates the problem). The Single DUT test and traffic setup of RFC
> 8219 would look like this:
> 
>                            +--------------------+
>                            |                    |
>               +------------|IPv4   Tester   IPv6|<-------------+
>               |            |              tunnel|              |
>               |            +--------------------+              |
>               |                                                |
>               |            +--------------------+              |
>               |            |                    |              |
>               +----------->|IPv4     B4     IPv6|--------------+
>                            |              tunnel|
>                            +--------------------+
> 
> Although the arrows are unidirectional, bidirectional traffic is
> required. On the left side, the Tester may send IPv4 traffic with
> pseudorandom source and destination port numbers. The DUT encapsulates
> the IPv4 packets into IPv6 packets and forwards them to the IPv6 port of
> the DUT. However, on the right side, the DUT has to use fixed UDP port
> numbers, when sending IPv6 traffic, as there is a tunnel there. Of
> course, the encapsulated IPv4 packets may have pseudorandom UDP port
> numbers, but they are not taken into consideration by the NIC of the B4
> device. Thus the interrupts caused by the arrival of the IPv6 packets
> will not be hashed to all the cores of the B4 devices.
> 
> IMHO, we may say that it is NOT a problem regarding the benchmarking
> method, because of the following consideration:
> 
> We use pseudorandom port numbers for benchmarking in order to
> approximate the nature of the real life traffic, which usually has
> different IP addresses and port numbers. However, in this case, the real
> life traffic does have fixed port numbers due to the tunnel, thus having
> fixed UDP port numbers of over IPv6 complies with the real life traffic.
> (In addition to that, B4 is usually a small CE device, which perhaps
> does not have too many cores.)
> 
> Am I right, or do I miss something?
> 
> The situation of the AFTR is very different.
> 
>                            +--------------------+
>                            |                    |
>               +------------|IPv6   Tester   IPv4|<-------------+
>               |            |tunnel              |              |
>               |            +--------------------+              |
>               |                                                |
>               |            +--------------------+              |
>               |            |                    |              |
>               +----------->|IPv6    AFTR    IPv4|--------------+
>                            |tunnel              |
>                            +--------------------+
> 
> On the left side, the Tester needs to send IPv6 packets that encapsulate
> IPv4 packets. In real life, a high number of B4 devices are connected to
> an AFTR. Thus although the port numbers are fixed due to the tunnels,
> the IPv6 source addresses are different, and thus the interrupts caused
> by the packets are hashed more or less equally to the several CPU cores
> of the AFTR. (Of course, the encapsulated IPv4 packets have pseudorandom
> port numbers, but the NIC does not "see" them.)
> 
> Am I right, that here the real spirit of RFC 4814 is that the Tester
> should use different source IPv6 addresses?
> 
> Of course, their number (e.g. 256 or 65536) is a further question.
> 
> As for the IPv4 traffic in the other direction, varying port number may
> be (and should be) used. However, they may not be arbitrarily due to the
> stateful NAPT44 in the AFTR. To that end, I have an idea: the tester
> should send preliminary traffic in the left to right direction, where
> the encapsulated IPv4 packets have varying (not random but rather
> one-by-one increasing) port numbers, and it should store the observed
> port numbers on the right side. Then it may randomly select from among
> the observed port numbers, when generating traffic in the right to left
> direction. But it is beyond the scope of my current question.
> 
> I really appreciate any suggestions, comments, ideas!
> 
> Best regards,
> 
> Gábor Lencse
> 
> -----------------------------------------------------
> Dr. Gabor Lencse
> Professor
> Department of Telecommunications 
> Szechenyi Istvan University
> Egyetem ter 1.  Gyor, H-9026, HUNGARY
> FAX: [phone]
> E-Mail: [e-mail]
> URL: http://www.hit.bme.hu/people/lencse/index_en.htm 
>