[Tsv-art] Tsvart telechat review of draft-ietf-bmwg-ngfw-performance-13

[Tommy Pauly via Datatracker <[e-mail]>]

Reviewer: Tommy Pauly
Review result: Almost Ready

This document has been reviewed as part of the transport area review team's
ongoing effort to review key IETF documents. These comments were written
primarily for the transport area directors, but are copied to the document's
authors and WG to allow them to address any issues raised and also to the IETF
discussion list for information.

When done at the time of IETF Last Call, the authors should consider this
review as part of the last-call comments they receive. Please always CC
[e-mail] if you reply to or forward this review.

I agree with the points Lars made about this document being too specific
and constrained with regards to TCP details, and not nearly as specific
with other protocol details. This was brought up in my initial TSVART
review, which I will quote here since it still applies:

"From a transport perspective, I’m concerned that some things are over-specified
(details of TCP implementations) and others are underspecified (how throughput
is measured, how loss and delay are tested)... I’d like to see transports 
(TCP/UDP/QUIC/other) be treated more consistently throughout the document,
particularly since non-TCP traffic will become increasingly relevant for the
devices these tests are targeting.

...

The client configuration section 4.3.1.1 details TCP stack configuration, but
does not address other transports. Discussing QUIC seems like it will be
relevant soon.

Overall, for this section, I am struck that there’s a lot of detail that seems
over-specified, with lots of normative language. For example, the TCP
connection MUST end with a three- or four-way handshake. What if there’s a RST?
I don’t understand what we’re requiring of these TCP implementations apart from
being a functional and compliant TCP implementation. How much of this is
actually required?"

Given the IESG reviews, I do agree this needs to be addressed before moving forward.
While we could spend a long time with transport area folks trying to fix the details
and flesh out equal levels of detail for QUIC and HTTP/1.1 / HTTP/2 / HTTP/3 configurations,
I don't think that is appropriate for this document.

My suggestion would be to strike the details about TCP entirely, particularly the
extraneous normative requirements. If your concern is how the test equipment will
behave with 1000s of connections, express that as a top-level requirement for any
transport; describe that the transports need to be tuned with common options to
ensure fairness and consistent use of the available bandwidth, etc. Getting at the
reasons will make it clearer. 

You also already say that "these are the defaults in most client operating systems".
Rather than duplicating what you currently believe are the defaults, just encourage
the use of defaults.