Re: [Tsv-art] Tsvart telechat review of draft-ietf-bmwg-ngfw-performance-13

[[e-mail]]

Hi Tommy,

Thanks for you reply. The general consensus amongst the authors and contributors is that there is a need to get into the weeds when it comes to TCP configuration of the test tools and elements of the test bed. This draft is meant to address the test tools in order to ensure (hopefully) results can be replicated. I can certainly understand your position when it comes to production environments. But that is not what we are concerned about. The various test tools we have looked at and worked with have varied default settings. Which is what led us to our approach.

That said however, Al Morton pointed us to RFC7414 (https://datatracker.ietf.org/doc/html/rfc7414) as a possible reference that lists/consolidates all the TCP requirements and defaults that are relevant or in-force today. We could reference that RFC and specific sections for the requirements we need. This RFC is over 7 years old. Is there anything more current?

Thoughts?

Brian

-----Original Message-----
From: Tommy Pauly via Datatracker <[e-mail]> 
Sent: Tuesday, May 24, 2022 11:09 PM
To: [e-mail]
Cc: [e-mail]; [e-mail]; [e-mail]
Subject: Tsvart telechat review of draft-ietf-bmwg-ngfw-performance-13

Reviewer: Tommy Pauly
Review result: Almost Ready

This document has been reviewed as part of the transport area review team's ongoing effort to review key IETF documents. These comments were written primarily for the transport area directors, but are copied to the document's authors and WG to allow them to address any issues raised and also to the IETF discussion list for information.

When done at the time of IETF Last Call, the authors should consider this review as part of the last-call comments they receive. Please always CC [e-mail] if you reply to or forward this review.

I agree with the points Lars made about this document being too specific and constrained with regards to TCP details, and not nearly as specific with other protocol details. This was brought up in my initial TSVART review, which I will quote here since it still applies:

"From a transport perspective, I’m concerned that some things are over-specified (details of TCP implementations) and others are underspecified (how throughput is measured, how loss and delay are tested)... I’d like to see transports
(TCP/UDP/QUIC/other) be treated more consistently throughout the document, particularly since non-TCP traffic will become increasingly relevant for the devices these tests are targeting.

...

The client configuration section 4.3.1.1 details TCP stack configuration, but does not address other transports. Discussing QUIC seems like it will be relevant soon.

Overall, for this section, I am struck that there’s a lot of detail that seems over-specified, with lots of normative language. For example, the TCP connection MUST end with a three- or four-way handshake. What if there’s a RST?
I don’t understand what we’re requiring of these TCP implementations apart from being a functional and compliant TCP implementation. How much of this is actually required?"

Given the IESG reviews, I do agree this needs to be addressed before moving forward.
While we could spend a long time with transport area folks trying to fix the details and flesh out equal levels of detail for QUIC and HTTP/1.1 / HTTP/2 / HTTP/3 configurations, I don't think that is appropriate for this document.

My suggestion would be to strike the details about TCP entirely, particularly the extraneous normative requirements. If your concern is how the test equipment will behave with 1000s of connections, express that as a top-level requirement for any transport; describe that the transports need to be tuned with common options to ensure fairness and consistent use of the available bandwidth, etc. Getting at the reasons will make it clearer. 

You also already say that "these are the defaults in most client operating systems".
Rather than duplicating what you currently believe are the defaults, just encourage the use of defaults.