Re: [ippm] Magnus Westerlund's Discuss on draft-ietf-ippm-capacity-metric-method-06: (with DISCUSS)

["MORTON, ALFRED C (AL)" <[e-mail]>]

Hi Magnus, Thanks for your review.

Please see replies to your review and comments from Len and me, consolidated and marked [acm] below,

Al

> -----Original Message-----
> From: Magnus Westerlund via Datatracker [mailto:[e-mail]]
> Sent: Thursday, February 25, 2021 9:19 AM
> To: The IESG <[e-mail]>
> Cc: [e-mail]; [e-mail];
> [e-mail]; Ian Swett <[e-mail]>; [e-mail];
> [e-mail]
> Subject: Magnus Westerlund's Discuss on draft-ietf-ippm-capacity-metric-
> method-06: (with DISCUSS)
> 
> Magnus Westerlund has entered the following ballot position for
> draft-ietf-ippm-capacity-metric-method-06: Discuss
> 
...
> 
> ----------------------------------------------------------------------
> DISCUSS:
> ----------------------------------------------------------------------
> 
> A) Section 8. Method of Measurement
> 
> I think the metrics are fine, what makes me quite worried here is the
> measurement method. My concerns with it are the following.
> 
> 1. The application of this measurement method is not clearly scoped. Therefore
> I will assume that across the Internet measurements are possible. 
[acm]

Like all the ad hoc Internet Speed measurements in the world today, our primary interest is access measurement.  I added "Applicability" to the Scope and Goals section 2, as follows:

NEW paragraph
The primary application of this metric and method of measurement described here is the same as in Section 2 of RFC7479, where:

o The access portion of the network is the focus of this problem statement. The user typically subscribes to a service with bidirectional access partly described by rates in bits per second.

-=-=-=-=-=-=-=-

[acm] Obviously we won't be able to stop application beyond access, just like we can't stop people from grabbing one of any number of utilities and anywhere and sending at any rate they can achieve, but we have made an applicability statement. 

> However in
> that context I think the definition and protection against severe congestion
> has significant short comings. The main reason is that the during a
> configurable time period (default 1 s) the sender will attempt to send at
> a specified rate by a table independently on what happens during that
> second.
[acm] 
Not quite, 1 second is the default measurement interval for Capacity, but sender rate adjustments occur much faster (and we add a default at 50ms). This is a an important point (and one that Ben also noticed, regarding variable F in section 8.1). So, I have added FT as a parameter in section 4:

o FT, the feedback time interval between status feedback messages communicating measurement results, sent from the receiver to control the sender. The results are evaluated to determine how to adjust the current offered load rate at the sender (default 50ms)

-=-=-=-=-=-=-=-
Note that variable F in section 8.1 is redundant with parameter F in Section 4, 
the number of flows (in-06). So we changed the section 8.1 variable F to FT 
in the working text.


> 
> 2. The algorithm for adjusting rate is table driven but give no guidance on how
> to construct the table and limitations on value changes in the table. In
> addition the algorithm discusses larger steps in the table without any
> reflection of what theses steps sides may represent in offered load.
[acm] 
We can add (Len suggested the following text addition):
OLD
8.1. Load Rate Adjustment Algorithm

A table SHALL be pre-built defining all the offered load rates that
will be supported (R1 through Rn, in ascending order, corresponding
to indexed rows in the table). Each rate is defined as datagrams of...

NEW
8.1. Load Rate Adjustment Algorithm

A table SHALL be pre-built defining all the offered load rates that
will be supported (R1 through Rn, in ascending order, corresponding
to indexed rows in the table). It is RECOMMENDED that rates begin with
0.5 Mbps at index zero, use 1 Mbps at index one, and then continue in
1 Mbps increments to 1 Gbps. Above 1 Gbps, and up to 10 Gbps, it is
RECOMMENDED that 100 Mbps increments be used. Above 10 Gbps,
increments of 1 Gbps are RECOMMENDED. Each rate is defined as...

-=-=-=-=-=-=-=-

> 
> 3. Third the algorithms reaction to any sequence number gaps is dependent on
> delay and how it is related to unspecified delay thresholds. Also no text
> discussion how these thresholds should be configured for safe operation.
[acm] 
We can add some details in the paragraph below:
OLD
If the feedback indicates that sequence number anomalies were detected OR 
the delay range was above the upper threshold, the offered load rate is decreased. 
Also, if congestion is now ...
NEW
If the feedback indicates that sequence number anomalies were detected OR 
the delay range was above the upper threshold, the offered load rate is decreased. 
The RECOMMENDED values are 0 for sequence number gaps and 30-90 ms for lower 
and upper delay thresholds. Also, if congestion is now ...

-=-=-=-=-=-=-=-

Please also note many requirements for safe operation in Section 10, 
Security Considerations.

> 
> B) Section 8. Method of Measurement
> 
> There are no specification of the measurement protocol here that provides
> sequence numbers, and the feedback channel as well as the control channel.
[acm] 
That is correct. The Scope does not include protocol development.

> Is this intended to use TWAMP?
[acm] 
Maybe, but a lot of extensions would be involved.

> 
> From my perspective this document defines the metrics on standards track
> level. However, the method for actually running the measurements are not
> specified on a standards track level. 
[acm]
In IPPM work, the methods of measurement are described more broadly than 
the metrics, as actions and operations the Src and Dst hosts perform to 
send and receive, and calculate the results. 

IPPM Methods of Measurement have not included protocol requirements in 
the past, in any of our Standards Track Metrics RFCs.  In fact, we developed 
a measurement-specific criteria for moving our RFCs along the standards track 
that has nothing to do with protocols or interoperability. 
See BCP 176 aka RFC 6576:  https://tools.ietf.org/html/rfc6576
IP Performance Metrics (IPPM) Standard Advancement Testing 

> No one can build implementation. 
[acm] 
I'm sorry, but that is not correct.  Please see Section 8.4.

> And if the section is
> intended to provide requirements on a protocol that performs these measurements
> I think several aspects are missing. There appear several ways forward here to
> resolve this; one is to split out the method of measurement and define it
> separately to standard tracks level using a particular protocol, another
> is to write it purely as requirements on a measurement protocols.
[acm] 
As stated above, connecting a method with a single protocol is not IPPM's way.

This seems to be the type of document you are looking for, completely separate 
from the IPPM Metric the Method RFCs:

RFC 7497 https://tools.ietf.org/html/rfc7497
    Rate Measurement Test Protocol Problem Statement and Requirements

Abstract

   This memo presents a problem statement for access rate measurement
   for test protocols to measure IP Performance Metrics (IPPM).  Key
   rate measurement test protocol aspects include the ability to control
   packet characteristics on the tested path, such as asymmetric rate
   and asymmetric packet size.

RFC7497 doesn't anticipate the degree of rate control with feedback 
that we have described in the Capacity draft, but it has some useful aspects 
(access scope). It also provides evidence that the IPPM WG separates 
protocol requirements and protocol specification work (such as TWAMP and STAMP) 
from the RFCs on Metrics and Methods of Measurement.

This memo replaces ad hoc, inaccurate TCP-based measurements of "Internet Speed"
with a standardized metric and method. It allows the protocol development to begin; 
that's exactly what we said in the scope!

I have asked for agenda time at the IPPM session to briefly present and get 
feedback on the protocol used in the running code, the next step.

> 
> 
> 
>