[ippm] Re: Gorry Fairhurst's Discuss on draft-ietf-ippm-capacity-protocol-20: (with DISCUSS and COMMENT)

[[e-mail]]

Hi Gorry,

thanks for your comments. Responses marked [Authors] in line below. Please note, over here there's a bank holiday on 19. June. I'll not reply before Monday.

Regards,

Ruediger


---------------------------< snip >--------------------------------
DISCUSS:
----------------------------------------------------------------------

Thank you for this I-D that defines a method for measuring Network Capacity
metrics. The measurement flows generate Internet packet flows that are
rate-controlled. The primary concerns are about the use of Datagrams
(addressing and packet size), the security model, and the congestion control
functions. This is therefore a significant piece of transport work and there is
a much longer than a normal DISCUSS. I hope to update my ballot once
resolutions have been agreed.

DISCUSS Topics from the TSV-ART review.

The I-D needs to therefore consider implications around security. Thanks to the
TSVART team for their review. In the security design, I saw many changes and
thank the editors et al for these updates in the present revision. Thanks.

--------------

This review also noted the way that datagram size was handled - and I note that
this is an important part of any service over UDP. In my review, I found the
guidance in the following section is inadequate :
   /A simplified approach is to choose the
   default datagram size small enough to prevent fragmentation.  MTU
   size detection prior to a test is out of scope of this document.  A
   method for a path MTU size discovery, as for example proposed by
   [RFC8899], is left for future design and implementation./

- RFC8899 could indeed be applicable, but the requires support is not defined
in this specification, so this is more than a future implementation issue. This
will actually require changes to the protocol, especially if a method is to be
interoperable.

[Authors]: ADD ...for future design and implementation. Note, that such a mechanism will require a carefully adapted protocol design to ensure interoperability. Unless... all tests. Note also, that network debugging may benefit from fragmentation to be completely under the control of the test administrator (rather than a protocol).

[Authors]: The latter statement is based on operational experience.
---------------------

DISCUSS Topics 1. Normative Dependencies

I would like to discuss the normative dependence on two external references for
key components of the design.

There are two Normative References to external documents:

   [TR-471]   Morton, A,, Editor., "Broadband Forum TR-471: IP Layer
              Capacity Metrics and Measurement, Issue 4", September
              2024, <https://www.broadband-forum.org/technical/download/
              TR-471.pdf>.

   [Y.1540]   ITU-T, "Internet protocol data communication service - IP
              packet transfer and availability performance parameters",
              ITU-T Recommendation Y.1540, December 2019,
              <https://www.itu.int/rec/T-REC-Y.1540-201912-I/en>.

DISCUSS 1.1 What is the process by which these have been revised by the IETF?

- I have a concern about the external definitions for terms, metrics and
algorithms used in this I-D.

- I note that describes itself as a technical report, that has been approved by
members of the Forum and that this Technical Report is subject to change. The
Broadband Forum has submitted a number of revisions of the report TR-471 in
recent years. I would like to discuss whether the normative reference is to a
particular version? If not what is expected if this standard evolves, how will
this evolution be managed?

[Authors]: Current Ref is to a particular version: Morton, A, Editor., "Broadband Forum TR-471: IP Layer Capacity Metrics and Measurement, Issue 4", September 2024

TR-471 seems to provide relevant, although I have yet to review that in detail
and there is useful information to the use of the method being specified.

[Authors] Originally, the draft didn't explain any of the fields which saw prior documentation in other docs. The references have been checked during this LC by the authors. See draft-ietf-ippm-capacity-protocol-17, they have been added there.

DISCUSS 1.2 TR-471 is used to normatively define a set of parameters - this
makes the current I-D normatively dependent on changes to another spec. Has the
working group fully considered the implications of changes to these
specifications, rather than normatively defining the operation to be identical
to the current specification of this external reference?

[Authors]: Copying in the specifications from other docs doubles the specification, but could be done if this is the best way out of this dilemma.

Examples include:

useOwDelVar

ignoreOooDup: Boolean, see ReordDupIgnoreEnable in Table 3 of  [TR-471].

subIntPeriod: Test Sub-interval period in [ms], see dt (TestSubInterval) in
Table 1 of [TR-471], default 1000 ms. Why is this a suitable interval?

lowThresh, upperThresh: are defined in Table 3 of [TR-471].

testIntTime: see TestInterval, Table 3 of [TR-471].

dscpEcn: specified by [TR-471] (see also my discuss topic 8).

-------------------------

DISCUSS Topics 2. This I-D defines a protocol layered on top of UDP. Therefore,
what level of IETF review has been performed to evaluate the way the flow rate
controlled? The I-D is subject to the BCP that defines UDP Usage [RFC8085] that
sets out expectations for when the protocol is used in the Internet. It is
defined for performance testing, and as such it is possible generate
considerable Internet load.

DISCUSS 2.1 RFC 2914 defines the need for congestion control. Section 3.1.1,
3.1.2 and 3.1.5 of RFC 8085 provides the current BCP guidance on Congestion
Control. RFC9743 (BCP 133) provides guidance on how to specify new congestion
control methods. RFC 8084 provides an alternative, to define an envelope that
controls the overall envelope of a connection. I think these RFCs apply to this
I-D. - Was there any review by ICCRG or CCWG, or analysis of the impact from a
congestion control perspective?

[Authors] The algorithm is not part of this document. Algorithm properties and requirements are discussed by https://www.rfc-editor.org/rfc/rfc9097.html#name-method-of-measurement, and https://www.rfc-editor.org/rfc/rfc9097.html#name-rfc-8085-udp-guidelines-che checks RFC8085 guidelines.
Serious discussion with Magnus Westerlund during the LC of 9097 (by mail and at least two calls) resulted in Timeouts being adjusted, ramp-up conditions tweaked, and test stop requirements modification to provide reasonable protections in this area.

DISCUSS 2.2 At the end of Section 4.2:
To ensure readers are clear about the importance, I'd strongly recommend to add
a note stating that the described method is necessary also to avoid potentially
inducing congestion after there is a overload/loss on the control path.

[Authors]: 4.2, that's the Security Mode section. Please clarify: That note you'd like to be added is to state, that avoiding DoS or amplification attacks require authentication (as stated by RFC 8085)?

DISCUSS 2.3 In section 7.1: If I understand the sending rate decrease based on
feedback intervals - which seems like it is heading in the correct direction. I
do not see how these intervals are adjusted to the path RTT, could they be?

[Authors] See RFC9097: The offered load SHALL then be decreased, following the same process as when the feedback indicates the presence of one or more sequence number anomalies OR the delay range was above the upper threshold. Note, impairments as seen by the receiver drives the sender's rate. The protocol seeks to be as independent of RTT as possible (avoiding a shortcoming of existing TCP-based speed tests). The default of 50 ms status feedback messages has been shown to work well in both lab settings with no delay and between servers testing from the US to Europe. However, it is a configurable parameter. This is a discussion belonging to an RFC 9097 bis, if desired, not to this doc.

DISCUSS 2.4 In section 5.1. on the responsiveness (what CC experts often call
aggressive behaviour) of a flow: I note that the LOAD CONTROL as define uses a
statically configured watchdog interval -  which I think allows a flow to
contribute excess congestions for 3 seconds after this is detected, longer if
small levels of loss/congestion is not considered a trigger. For an arbitrary
path RTT this could be a large number of RTTs (e.g. 300 for a 10mS path). -
There are specific recommendations on the use of UDP that describe this case:
RFC 8085.

[Authors] The algo requirements for the method of measurements and metric are specified by RFC 9097 (standards track). There, avoiding false maximums due to some early (non-congestion related) impairments is an objective. Also, the trade-off is that a test with a less aggressive load control (that induces less congestion, loss, delay) needs to run much longer to find a true maximum. The latter results in both longer test times and considerably more network traffic (!). In general, this is why there is support for two algorithms (B and C) that differ in aggressiveness. These (and future possible additions) allow for better accommodation of vastly different underlying network technologies (e.g., PON, DOCSIS, 5G, Satellite,...).

-------------------

DISCUSS Topics 3: How do we know the suitability of the specified rate
adaptation mechanisms?

DISCUSS 3.1 In section 7.1 there is a description of another alternative
congestion control method. Algorithm (B, see [Y.1540]) is a CC algorithm
defined outside the IETF, please explain how this has been shown to be safe for
use as a CC in the Internet?

[Authors] RFC 9097 covers the algo by specifying requirements for the method of measurements and the metric.
Historically, we've had a standard Y.1540 in Feb. 2020 with the Annex B algo, and an accepted WG draft -00 for 9097 at the same time. Both docs are aligned and changes applied to 9097 were entered into the implementation. You can track that in Y.1540., which saw an update after 9097 was finalized.   All in all, 9097 is the relevant doc for the measurement algo. Any specification not meeting 9097 requirements isn't UDPST standard conformant, no matter which SDO publishes it.

DISCUSS 3.2 In section 7.1: The present I-D says:
   /Algorithm B also has two modes for increasing/decreasing the sending
   rate:/

If I understand the sending rate decrease based on feedback intervals - which
seems like it is heading in the correct direction. I did not see how these
intervals adjust to the path RTT. I would like to discuss how they adapt or why
they do not?

[Authors]: This is again an RFC 9097 topic. UDPSTP signals the relevant info and tries to do so fast. The default was chosen because it performed well in extensive testing across many different network technologies. The feedback interval and the RTT sensitivity are configurable and can be adjusted to accommodate new technologies and/or adjustments to existing ones.

DISCUSS 3.3 There seems little discussion of the risk to the network and other
flows when performing a fixed-rate test, or whether this is applicable usage
across an Internet path. The present I-D says:
   /The reasons to conduct a fixed-rate test include
   stable measurement at the maximum determined by the load rate
   adjustment algorithm, or the desire to test at a known subscribed
   rate without searching./

Also:
   /The client MAY communicate the desired fixed rate in its test
   activation request./

[Authors] Fixed rate tests proved useful for test&turn-up verification of new service as well as post-repair verification after service calls. The ability to confirm that the service is completely clean (no loss) and stable at 90 or 95% of the provisioned rate is important. However, the desire to support fixed rate tests is controlled by the server so an administrator could decide to reject test activations that want to do a fixed rate test. They can also choose to cap the maximum that a fixed rate test is allowed to request. This is all handled by the server when it interrogates the Test Activation Request PDU. In our open source OB-UDPST code the server administrator can even coerce the client's requested maximum down to some allowable level...this is all supported by the protocol.

I see that the test also maintains rttMinimum and rttVarSample, so this could
be used to develop an Internet circuit breaker as described in RFC 8085, but I
would be keen to understand what sets the acceptable envelop for use across the
Internet.

[Authors] See RFC9097, https://www.rfc-editor.org/rfc/rfc9097.html#name-load-rate-adjustment-pseudo, slowAdjCount and delay. Current implementation uses these as measurements only. One of the challenges is that different network technologies can result in very different congestion induced maximum delays. Short PON paths may see no more 10-20 ms at their max while DOCSIS may experience well over a 1 second of delay. These differences are why the load adjustment algorithm parameters are designed to be configurable - so an operator can create the most efficient test for their network.

DISCUSS 3.4 In section 7.2, Status PDU: Please explain (and specify in the
text) what happens in the case of the first para of section 7.2 when there is
severe congestion on the return path - I assume this could result in
significant loss, and no Status PDU is sent?

   The watchdog timer at the Load PDU sender SHALL be reset each time a
   Status PDU is received.  See non-graceful test stop in Section 8 for
   handling the watchdog timeout expiration at each endpoint.

[Authors] See RFC9097, https://www.rfc-editor.org/rfc/rfc9097.html#name-load-rate-adjustment-pseudo, seqErr, seqErrThresh. A simple watchdog timer would shut down the test if status PDUs can't make it back to the sender. While none are making it back, no rate adjustments are made.

DISCUSS 3.5 In section 7.2, Status PDU. The present I-D says:
/ The watchdog timer at the Load PDU sender SHALL be reset each time a
   Status PDU is received.  See non-graceful test stop in Section 8 for
   handling the watchdog timeout expiration at each endpoint./
- Please explain (and specify in the text) why is it safe to reset the timer
even if the Status PDU does not indicate correct reception?

[Authors]: CHANGE .... be reset each time a valid PDU is received (i.e., if the checksum or authentication have been successfully checked). See....
...endpoint. Note that the watchdog timer is to 	detect a connection failure or a massive congestion condition only.

---------------------

DISCUSS Topics 4. Concurrent flows.

4.1 In section 9.0:
   /It is obviously counterproductive to run more than one independent
   and concurrent test/
- Is there a congestion control or DoS issue here, I think so, can we discuss
why the I-D says this is allowed without a mechanism to defend against this?

[Authors] One test may impact the results of another test by introducing traffic conditions that might be interpreted as congestion and result in a false maximum. However, the protocol does support multi-flow testing so environments that use multiple paths (MLPPP, LAGs, ECMP, etc.) can be properly tested for a maximum.

--------------------

DISCUSS Topics 5. Definition of an IANA Registry for alternative rate control
algorithms.

The present I-D says: "Test Activation PDU Rate Adjustment Algo.  Registry"
Why should the IETF publish a registry for this protocol with a private space
and no overarching congestion control framework?

In 11.2.8, the present I-D says:

   Value   Description             Reference
   (Numeric)
   ===================================================
    A(n/a)  Not used                <this RFC>

    B(0)    Rate algorithm Type B   <this RFC>

    C(1)    Rate algorithm Type C   <this RFC>

   Table 16: Initial Test Activation PDU Rate Adjustment Algo. values

DISCUSS 5.1: What is Value A in Table 16 (PDU Rate Adjustment)?
[Len] It has no value and does not exist in any formal document or publication. It was simply the initial algorithm that evolved into B as a result of extensive testing.

DISCUSS 5.2 : There is no specification in this I-D for these algorithms.

[Authors] Thanks, <this RFC> is certainly the wrong entry (we've argued that up and down above). Correct entries are:
B: Recommendation ITU-T Y.1540 (2019) Amd. 2 (03/2023)
CHANGE:   B(0)    Rate algorithm Type B   [Y.1540Amd. 2]  (& added normative reference)
C: BBF TR-471 Maximum IP-Layer Capacity Metric, Related Metrics, and Measurements Issue: 4 Date: September 2024
CHANGE:   C(1)    Rate algorithm Type C   [TR-471]

And related, please explain the testing for use of this algorithm:
   /An OPTIONAL load rate adjustment algorithm (designated C) has been
   defined in [TR-471].  Algorithm C operation and modes are similar to
   B, but C uses multiplicative increases in the fast mode to reach the
   Gigabit range quickly and adds the possibility to re-try the fast
   mode during a test (which improves the measurement accuracy in
   dynamic or error-prone access, such as radio access)./

DISCUSS 5.3 RFC 9473 has provided guidance on the IETF process if this is being
considered for use on Internet paths. Why is the IETF standardising use of a
rate adjustment algorithm defined elsewhere, without bounding its over-arching
congestion control properties? Please explain how these methods were evaluated,
and by whom. How in the future will this be tested for congestion control and
how this will be maintained.

[Authors] Standard conformant algorithms need to be conformant to standard track RFC 9097. IETF is free to update this work at any time. The current implementations were implemented and tested by the original authors of this draft (Al and Len, who edited/authored RFC 9097, TR417 and Y.1540, as well as the implementation). 
The algorithms were designed to be able to ramp-up from around 100 Kbps to many 10's of Gbps - and do it in about 10 seconds. As a side note, the rate adjustment algorithms were designed to be highly configurable so that an operator can create slower long duration (and much less aggressive) tests if they choose. Also, the algorithm is always controlled by the server so a client is limited in what and how fast it can send.
One of the shortcomings of B was that if the connection being tested was very error prone (poor DSL for example) it would potentially keep the algorithm from fully ramping up because of the inferred congestion. This would result in the test (which is a fixed duration) from reaching a true maximum. Algorithm C is an option in these environments because of faster more aggressive ramp-up. During testing of the algorithms, B (and its current default settings) was found to produce the most reliable results for a 10 second test. This was across DSL, cellular (3G, 4G, 5G), DOCSIS, PON, and satellite. Algorithm C came about because the common factor that could throw off a test was a high underlying (non-congestion induced) impairment rate.

DISCUSS 5.4  What is the scope of use for a rate adjustment algorithm defined
as experimental. RFC 9473 provided guidance on this topic. Why is this not
scoped only for use in limited domains?

[Authors] The idea of an experimental algorithm is that a new or uncommon technology (or a specifically desired test behavior) could be accommodated by an alternative rate adjustment algorithm in the future. For example, an algorithm that ignored delay and only considered loss (or ignored loss and only considered delay) for a particular technology or service. Until it was formalized and adequately tested, it would use an experimental designation.

-----------------------

DISCUSS Topics 6. How are the congestion control methods evaluated and
specified?

[Authors] That is a matter of the IETF, if there's a decision to maintain capacity measurement related congestion control algos. The algos mentioned above were created by the authors - any pre-existing implementations meeting the design constraints would have been appreciated, but there were none. We'd prefer to decouple the IETF decision to consent and install a capacity measurement related congestion control algo process from the progress of this draft.

DISCUSS 6.1 Please see this text:
 /On the other hand, the test configuration MAY use a fixed sending
   rate requested by the client, using the field srIndexConf./
- A fixed rate test brings additional concerns, and I didn't find this
analysis. Is there a need to scope this as a non-default mode? Should it be
restricted to certain domains.

[Authors] Fixed rate testing is intended for Test & Turn-up as well as port-repair verification of restored service. It is server controlled.

DISCUSS 6.2 Please add a note to the reaction to lack of feedback in Section
4.2: I agree that "the test session will prematurely terminate (no further load
traffic SHALL be transmitted)"

- To be clear about the importance, I'd recommend to add a note saying this is
necessary also to avoid potentially inducing congestion after there is a
overload/loss on the control path.

[Authors]: ADD This is necessary also to avoid potentially inducing congestion after there is an overload  or loss on the control path.

---
DISCUSS Topic 7: UDP Checksum Usage

- I have concerns around the definition of method that uses an optional
checksum. Section 3.4 of RFC 8085 provides the IETF current BCP on this topic
for both IPv4 and IPv6.

DISCUSS 7.1 In Section 4.5 on Optional Checksum.
- Why does this does not be enable the UDP checksum for IPv4, this would align
with RFC8085.

[Authors] Ideally, the UDP checksum would be enabled and this field would not be used. However, on low-end devices (with no NIC offload and the CPU doing all the work) it can make a big difference for the UDPST software to do a checksum calculation for the tiny UDPST PDU header (~32 bytes) than for the protocol stack to do it for the entire UDP datagram (up to ~9000 bytes) - especially when the payload is just dummy data.
If the comment is about "why don't we just recommend the UDPST checksum always be enabled" (i.e., what's the harm). It's just about trying to minimize processing overhead for devices that might already be struggling to accommodate potentially very high source/sink traffic rates.

DISCUSS 7.2 IPv6 requires a UDP checksum except for very specific usage, this
does not appear to be one of those cases (tell me if I am wrong), and so can
this require an IPv6 checksum?

[Authors] There are some gray areas where vendors try to be creative. So, although it is technically "required", some chipsets or device drivers (again, on low-end devices) may allow it to not be checked on reception to save CPU cycles. This is where the UDPST checksum is advantageous. Even when transmitting, if the payload is known to be all zeros the checksum can be calculated on the small UDPST header only without including the other ~8900 bytes. BTW, Broadcom was doing exactly this with their chipset SDK to allow UDPST to run in "accelerated" mode on the AT&T RGs to accommodate 5 and 10 Gbps.

---------------------------

DISCUSS Topic 8: ECN:

I can't understand if this I-D is intended to refer to ECN or not, but it does
mention the ECN code point in a variable definition. RFC 3819 specifies the
requirements for new transport methods to utilise an ECT() code point (see also
3.1.7 of RFC 8085). The transfer as defined does not support ECN, i.e. it does
not change its rate in response to ECN marks. TR471 is silent about the use of
ECN.

8.1  I think that this violates the conditions under which ECT() can be sent
and I assert that it needs to explain that the ECN field is always set to zero
(not-ECT) with this version of the specification.

[Authors] ADD dscpEcn:... The ECN bits MUST be set to 00 (not-ECT) for all UDPST Protocol-Version and Rate-Adjustment Algorithm combinations not supporting Explicit Congestion Notification.
Actually, fully supporting L4S in a future version seems not be difficult (this could potentially be mentioned). The sender could be configured to set ECT(1) in packets and the receiver could detect (be notified of) Congestion Experienced receptions and then indicate this congestion condition back via the existing Status PDU (resulting in a rate reduction). But this is certainly future work. In addition to the adapted feedback, the protocol version and suitable parameters need to be added (likely also an RFC 9097 matter).

---
DISCUSS Topic 9: Use of Addresses:
Specification (or non-specification) of support for Multicast, Broadcast and
Anycast addresses.

9.1 I think this looks like it has no multicast considerations and hence it
ought to be scoped to only unicast and not broadcast or multicast addresses.
Could this be specified for unicast address only ?

[Authors]: 5 Test Setup Request and Response
The clients origin IP address and the server destination address MUST NOT be broadcast or multicast addresses. Any Test Setup Request or Test Setup Response packet containing a multicast or broadcast origin or destination IP address MUST be silently dropped and ignored.

9.2 Could this method be used with an Anycast address?

[Authors] Yes, this is expected to work. 

---
DISCUSS Topic 10: Some parameters appear underspecified in this I-D, please
check. e.g.:

seqErrLoss/seqErrOoo/seqErrDup

[Authors] We've added the text explaining seqErrOoo and seqErrDup as part of the review and got an ok afterwards. Please be more specific (also about seqErrLoss).

===


----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------

I have a number of non-blocking comments that I expect could be addressed in
the next revision of this I-D:

In section 3:
Typos:/resepcitvely/respectively/
/the bottleneck is congested,/if the bottleneck is congested,/
In section 4.2:
- Is /Once the modes requiring or optionally adding
   authentication or encryption are implemented and configured locally,
   the unauthenticated mode must be disabled./ - Could this be a MUST?
In section 5,1: /A IANA is/IANA is/
In section 5.2.2: I expect the following text helps, but does this ensure or
assist in the firewall traversal? /To ensure that a server's local firewall
will successfully allow
   packets received for the new ephemeral port, the server SHALL
   immediately send a Null Request with the corresponding values
   including the source and destination IP addresses and port numbers./
In 6.1,  /continuosly/continuously/
In 6.2, Server Processes Test Activation Request and Generates Response:

In this I-D, the following keyword use seems odd to me and could be improved:
 /   After the server receives the Test Activation Request on the new
     connection, it MUST choose to accept, ignore or modify any of the
     test parameters. /
- is this an exclusive list of things that is required or a list of all that
are required? I do wonder if this is simply a list of what the protocol does
next?

In section 12: /comprehensability/comprehensibility/
Also in section 12: /sheperded/shepherded/