Re: [bmwg] I-D Action: draft-ietf-bmwg-benchmarking-stateful-04.txt

["Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco)" <[e-mail]>]

Hi Gabor.

When preparing (my part of) MLRsearch presentation,
I wanted to refer to my earlier comments
on how do we (fd.io CSIT project)
use MLRsearch when testing stateful NAT(44).

But it seems such comments never went past the planning stage
(at least I do not see any relevant e-mails from me in bmwg archive),
so now is a good time to write them down.

There are some differences between what we did when creating our tests
and what do you propose in your draft,
stemming mainly from the fact that out traffic generator is less capable,
but on the other hand we have better (not totally black box)
access to the DUT.

Firstly, we can configure memory usage of our DUT.
Instead of discovering "connection tracking table capacity",
we just tweaked the memory usage for each scale (number of flows)
until the DUT stopped crashing. :)
We still keep testing at few different scales,
as the performance depends on various memory caches.

Most importantly, we can ask our DUT to tell us
how many sessions it is tracking,
so we do not do any verification trials.

Secondly, just by simple reconfiguration (disable and re-enable NAT),
our DUT is able to delete the content of the connection tracking table
so fast it is not worth measuring.

In principle, the test could override the time needed
for a connection tracking table entry to time out
(after seeing last activity), but we were not forced to do that yet.

Now, to bad news. Our traffic generator does not really have
convenient enough state table on the Responder side.
I mean, it can track UDP and even TCP sessions properly,
but once the trial ends, all is forgotten.
So we can do bidirectional traffic,
but we cannot do unidirectional traffic from the Responder to the Initiator.

Also, our control over port numbers is limited.
Source port number is always randomly selected by the traffic generator,
and the destination port number has to be just one value
(unless I want to add one TG configuration item per each value separately).

Now, MLRsearch wants to control both the trial load and the trial duration.
That may work for Phase 2 if the number of flows is small,
but it will not work for Phase 1.
There, number of frames is determined by the number of flows,
so the trial duration is computed from that and the trial load.
So I taught MLRsearch library how to deal with that.

There are also other subtle differences between your draft and CSIT.
For phase 1, we do not count the frames arriving at the Responder,
we count "confirmed sessions" on the Initiator side.
One reason is that a real client would not be happy to see zero frames.
Also, our DUT is tracking sessions differently when they are only half-open,
(at least for TCP) and we wanted a slightly more realistic scenario.
Of course, we use the fast table drop before each subsequent trial.

For phase 2 tests, we have separate suites,
and we do not do full Connection Establishment Rate search there.
Similarly to the memory situation,
we just hardcoded the loads that proved to be safe and low before,
and use the DUT session count to confirm that is still the case.
In general, we use somewhat heavier transactions for Phase 2,
but still way shorter than what realistic sessions would be.
Also, we only use simple L2 counters to see
the expected number of frames has been exchanged.
This allows us to reach loads where the protocol-level counters
become unreliable due to the traffic generator running out
of CPU cycles to keep them properly updated.

Also, we try to be smart with tracking sessions
possibly timing out on DUT during the next trial.
Firstly, every time we see a trial with zero loss,
we assume each timer has been reset.
If we suspect some sessions may time out during next trial,
we execute one of those hardcoded trials with known safe value.
(In principle. In practice we do not test scales that need that anymore.)
After each trial we should confirm DUT still tracks all the sessions,
but we do that only at the end of the search,
as I do not remember that check ever failing.

I think all the other tricks we apply
are outside the scope of your draft
(e.g. what to do when the traffic generator
cannot sustain the required load).

Vratko.

-----Original Message-----
From: bmwg <[e-mail]> On Behalf Of Gábor LENCSE
Sent: Wednesday, 13 September, 2023 08:31
To: [e-mail]
Subject: Re: [bmwg] I-D Action: draft-ietf-bmwg-benchmarking-stateful-04.txt

Dear BMWG Chairs and Members,

I have just posted the updated version of our I-D.  There were only some 
minor updates to Section 3.2 and Section 7.

With my co-author, Keiichi Shima, we believe that our draft is ready for 
WGLC.

Best regards,

Gábor


9/13/2023 8:26 AM keltezéssel, [e-mail] írta:
> Internet-Draft draft-ietf-bmwg-benchmarking-stateful-04.txt is now available.
> It is a work item of the Benchmarking Methodology (BMWG) WG of the IETF.
>
>     Title:   Benchmarking Methodology for Stateful NATxy Gateways using RFC 4814 Pseudorandom Port Numbers
>     Authors: Gábor Lencse
>              Keiichi Shima
>     Name:    draft-ietf-bmwg-benchmarking-stateful-04.txt
>     Pages:   28
>     Dates:   2023-09-12
>
> Abstract:
>
>     RFC 2544 has defined a benchmarking methodology for network
>     interconnect devices.  RFC 5180 addressed IPv6 specificities and it
>     also provided a technology update, but excluded IPv6 transition
>     technologies.  RFC 8219 addressed IPv6 transition technologies,
>     including stateful NAT64.  However, none of them discussed how to
>     apply RFC 4814 pseudorandom port numbers to any stateful NATxy
>     (NAT44, NAT64, NAT66) technologies.  We discuss why using
>     pseudorandom port numbers with stateful NATxy gateways is a difficult
>     problem.  We recommend a solution limiting the port number ranges and
>     using two test phases (phase 1 and phase 2).  We show how the classic
>     performance measurement procedures (e.g. throughput, frame loss rate,
>     latency, etc.) can be carried out.  We also define new performance
>     metrics and measurement procedures for maximum connection
>     establishment rate, connection tear down rate and connection tracking
>     table capacity measurements.
>
> The IETF datatracker status page for this Internet-Draft is:
> https://datatracker.ietf.org/doc/draft-ietf-bmwg-benchmarking-stateful/
>
> There is also an HTMLized version available at:
> https://datatracker.ietf.org/doc/html/draft-ietf-bmwg-benchmarking-stateful-04
>
> A diff from the previous version is available at:
> https://author-tools.ietf.org/iddiff?url2=draft-ietf-bmwg-benchmarking-stateful-04
>
> Internet-Drafts are also available by rsync at:
> rsync.ietf.org::internet-drafts
>
>
> _______________________________________________
> bmwg mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/bmwg
>

_______________________________________________
bmwg mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/bmwg