Re: [bmwg] draft-lencse-bmwg-benchmarking-stateful

[Edwin Cordeiro <[e-mail]>]

I understand that previous testing recommended using UDP instead of TCP and
it was not my intent to say we need to replace UDP with TCP.

But we have to consider if the goal of the new testing is just to have
metrics to compare the transition techniques or do we want to also evaluate
their scalability.

As I believe we also want to test the scalability, specially on the
comparison between stateless versus stateful alternatives, testing with TCP
seems necessary. To test scalability, we need to consider scenarios that
are closer to the real usage of such techniques. Therefore I'm not
suggesting we scrap UDP tests, but that we should add some TCP based tests
to cover such scenarios.

When testing for scalability with TCP, we would target answering things
like:
- Does the transition technique affect how fast the congestion window grows?
- Does it affect encryption protocols?
- Is it impacted by out of order or lost packets?

Best regards,

Edwin Cordeiro


On Tue, May 25, 2021 at 10:12 PM Gábor LENCSE <[e-mail]> wrote:

> Dear Edwin Cordeiro,
>
> Thank you very much for your review and support!
>
> I would like to note regarding testing with TCP that all RFC 2544, RFC
> 5180 and also RFC 8219 recommends testing with UDP.
>
> Additional testing with TCP seems to be feasible, but it requires some
> considerations and design decisions. For example, whereas you can simply
> send UDP datagrams as test traffic, but you need to establish a TCP
> connection first. The 3-way handshake may be done during the preliminary
> phase, and one can send TCP segments as test traffic in the real test phase.
> However, it is does not seem trivial to me, how deeply the Tester should
> follow the TCP protocol.
> On the one hand, perfectly implementing all the rules of the TCP protocol
> would impose a lot of limitations on the Tester, thus it would not be able
> to send test frames at the required rates. This would prohibit the
> execution of the benchmarking tests like throughput, frame loss rate,
> latency etc, :-(
> On the other hand, normal routers do not look into the TCP protocol
> header, but NAT boxes do. Thus sending TCP segments with improper Sequence
> number, Acknowledgement number, etc. fields could result in different
> problems in the DUT depending on how deeply it follows the TCP protocol.
>
> Best regards,
>
> Gábor
>
> 24/05/2021 13:51 keltezéssel, Edwin Cordeiro írta:
>
> Dear authors of draft-lencse-bmwg-benchmarking-stateful,
>
> I agree with the motivation and ideas of the draft and I support its
> adoption.
>
> I recommend that the TCP considerations for such benchmarking methodology
> should not be left for a future document, as TCP still is the most used
> protocol. So, the details on TCP difficulties and an extension of this
> benchmark to consider TCP, should be in the scope of this document.
>
> Best regards,
>
> Edwin Cordeiro
>
> _______________________________________________
> bmwg mailing [e-mail]://www.ietf.org/mailman/listinfo/bmwg
>
>
> _______________________________________________
> bmwg mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/bmwg
>