Re: [ippm] Lars Eggert's Discuss on draft-ietf-ippm-capacity-metric-method-10: (with DISCUSS and COMMENT)

["MORTON JR., AL" <[e-mail]>]

Hi Lars,

Let's resolve your comments and answer your questions. 
Rüdiger [RG] and I [acm] will do our best.  

I have implemented all the resolutions offered below in the working text.

I'm pulling one point out for extra attention (from the DISCUSS):
@@@@ Is the DOWNREF registry an action item for chairs or AD/Martin?

Al

> -----Original Message-----
> From: Lars Eggert via Datatracker <[e-mail]>
> Sent: Tuesday, May 11, 2021 4:27 AM
> To: The IESG <[e-mail]>
> Cc: [e-mail]; [e-mail];
> [e-mail]; Ian Swett <[e-mail]>; [e-mail];
> [e-mail]
> Subject: Lars Eggert's Discuss on draft-ietf-ippm-capacity-metric-method-
> 10: (with DISCUSS and COMMENT)
> 
> Lars Eggert has entered the following ballot position for
> draft-ietf-ippm-capacity-metric-method-10: Discuss
> 
...
> 
> 
> ----------------------------------------------------------------------
> DISCUSS:
> ----------------------------------------------------------------------
> 
> DOWNREF from this Standards Track doc to Informational RFC7497.
> I didn't see this called out in the Last Call, and it's also not a
> document we have in the DOWNREF registry already.
[acm] 
Ok, I think this means another IETF Last Call?  It's not the first time
we've been tripped-up by keeping IPPM's Framework+updates and other 
requirements docs off the Standards-Track.

I also think that https://datatracker.ietf.org/doc/html/rfc7497 will figure
prominently in future work on the Capacity topic, such as a test protocol, so we
should add it to the DOWNREF registry.

@@@@ Is the DOWNREF registry an action item for chairs or AD/Martin?

> 
> 
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
> 
> Section 1, paragraph 3, comment:
> >    Here, the main use case is to assess the maximum capacity of the
> >    access network, with specific performance criteria used in the
> >    measurement.
> 
> If the main use case is to measure access network paths, that applicability
> should be made very explicit, i.e., in the title and abstract.

Two things:

1. The term "access" doesn't have a universally agreed definition regarding the portion of the network it covers, and that's partly because there are many architectures in use.

2. If we add access to the title or abstract, we are unnecessarily limiting the scope that we define later. We refer to section of RFC 7497 where the figure includes the subscriber's device, multiple private networks, and various devices and gateways ending at a transit gateway.

So, I don't want to emphasize the term "access" as you propose.
 
Perhaps we can clarify this further in the Intro, because the Scope says:

   The primary application of the metric and method of measurement
   described here is the same as in Section 2 of [RFC7497]...

How about:
Here, the main use case is to assess the maximum capacity of one or more networks
where the subscriber receives specific performance assurances, sometimes 
referred to as the Internet access.

> 
> Section 2, paragraph 2, comment:
> >    The scope of this memo is to define a metric and corresponding method
> >    to unambiguously perform Active measurements of Maximum IP-Layer
> >    Capacity, along with related metrics and methods.
> 
> I'm not sure what "unambiguously perform" is meant to express?

[RG] is it unambiguously measure...? If yes roughly:
"The scope of this memo is to define Active Measurement metrics and corresponding methods
to unambiguously determine Maximum IP-Layer Capacity and useful secondary evaluations."

[acm] With s/evaluations/metrics/ this WFM.

> 
> Section 2, paragraph 3, comment:
> >    A local goal is to aid efficient test procedures where possible, and
> >    to recommend reporting with additional interpretation of the results.
> 
> What is this goal "local" to - the IETF? 
[acm] Yes IETF, but that word choice seems unclear now.

[RG] further s/aid/add??

> Also, I'm unclear what "reporting
> with additional interpretation of the results" is supposed to express.

[acm] Let's try:

Secondary goals are to add considerations for test procedures, and
to provide interpretation of the Maximum IP-Layer Capacity results (to
identify cases where more testing is warranted, possibly with alternate
configurations).


> 
> Section 2, paragraph 10, comment:
> >    - If a network operator is certain of the access capacity to be
> >    validated, then testing MAY start with a fixed rate test at the
> >    access capacity and avoid activating the load adjustment algorithm.
> >    However, the stimulus for a diagnostic test (such as a subscriber
> >    request) strongly implies that there is no certainty and the load
> >    adjustment algorithm will be needed.
> 
> Since the first sentence of this paragraph uses a RFC2119 MAY, I'd suggest
> rephrasing the second sentence to end with "there is no certainty and use
> of the load adjustment algorithm is RECOMMENDED."

[RG] sounds good to me.
[acm] WFM
> 
> Section 3, paragraph 4, comment:
> >    1.  Internet access is no longer the bottleneck for many users.
> 
> What is the bottleneck then, and if it's not the access bandwidth, why is
> a new metric for it helpful?

[RG]+[acm] 1.  Internet access is no longer the bottleneck for many users, but subscribers expect network providers to honor contracted access performance.

[acm] IOW, when you subscribe to a 1 Gbps service, then your ISP, you, and possibly other parties want to assure that performance level is delivered. If you test and confirm the subscribed performance level, then you can seek the location of the bottleneck elsewhere.

> 
> Section 4, paragraph 4, comment:
> >    o  Src, the address of a host (such as the globally routable IP
> >       address).
> >
> >    o  Dst, the address of a host (such as the globally routable IP
> >       address).
> 
> For many hosts, their IP address is not globally routable, and hasn't been
> for decades. Or did you mean CPE?

[acm] I think *host* is general-enough to include CPE that is the source or destination of test packets. Also, "globally routable" is part of an example.

> 
> Section 8.1, paragraph 1, comment:
> > 8.1.  Load Rate Adjustment Algorithm
> 
> I wonder why this section is trying to define a crude new AIMD scheme, when we
> have lots of good experience with TCP's AIMD approach, which converges on
> the available bandwidth relatively well?

[acm] "crude"?? Our algorithm has the advantage of receiver measurements of rate,
loss, and delay. So, we aren't making inferences based on ACKs, we communicate
measurement feedback instead.

This is not a new AIMD. I'm sure you haven't had time to review
hundreds of test results where we compare our load adjustment algorithm to 
TCP's AIMDs when seeking to measure the maximum IP-layer capacity.

Please see section 4 of https://datatracker.ietf.org/doc/html/rfc8337
particularly the list of section 4.1 where Matt Mathis wrote:

   o  TCP is a control system with circular dependencies -- everything
      affects performance, including components that are explicitly not
      part of the test (for example, the host processing power is not
      in-scope of path performance tests).

and this is similar to the statements I have heard for >20 years in IETF,
where the conclusion is that TCP was not designed as a measurement tool.

So, it was a goal to avoid the limitations of TCP (windows, ACK feedback, 
retransmissions, and the CCA that were available, including BBR and BBRv2).
The algorithm we defined is intended to be fast, adjustable in straightforward 
ways, and (most important to Magnus) not to be deployed as a general-purpose
CCA in TCP. Our scope is infrequent, diagnostic measurements.

> 
> Also, the description of the algorithm itself is far from clear.

[acm] 
We added the pseudocode in case some readers weren't satisfied with the text,
and let's see what we can clarify with respect to your specific comments below.

> 
> Section 8.1, paragraph 7, comment:
> >    If the feedback indicates that no sequence number anomalies were
> >    detected AND the delay range was below the lower threshold, the
> >    offered load rate is increased.  If congestion has not been confirmed
> >    up to this point, the offered load rate is increased by more than one
> >    rate (e.g., Rx+10).  This allows the offered load to quickly reach a
> >    near-maximum rate.  Conversely, if congestion has been previously
> >    confirmed, the offered load rate is only increased by one (Rx+1).
> 
> The first sentence talks about sequence number anomalies and delay ranges.
> The rest of the paragraph then talks about whether congestion and whether it
> was confirmed or not. Does this mean that (some amount of) sequence number
> anomalies and/or delay indicates congestion? Is that defined somewhere?

[acm] Yes, in a paragraph that begins:
     Lastly, the method for inferring congestion is that there were...

I'll add a forward reference:
     ... If congestion has not been confirmed up to this point (see below 
     for the method to declare congestion), the offered load rate is increased ...
> 
> Section 8.1, paragraph 7, comment:
> >    If the feedback indicates that sequence number anomalies were
> >    detected OR the delay range was above the upper threshold, the
> >    offered load rate is decreased.  The RECOMMENDED values are 0 for
> 
> This doesn't agree with the previous paragraph, which says "Conversely, if
> congestion has been previously confirmed, the offered load rate is only
> increased by one (Rx+1)." (Or I don't understand the difference between
> congestion and sequence number anomalies/delay ranges; see previous
> comment.)

[acm]  Your "(Or ...)" is correct, we deferred the description of congestion
declaration to a later paragraph, where we require 2 feedback messages at least:

    Lastly, the method for inferring congestion is that there were sequence 
    number anomalies AND/OR the delay range was above the upper threshold 
    for two consecutive feedback intervals. ...

> 
> Section 8.4, paragraph 2, comment:
> >    This section is for the benefit of the Document Shepherd's form, and
> >    will be deleted prior to final review.
> 
> Should this section have been deleted before the IESG review then? Would
> you like the IESG to ignore it? 

[acm] I think that during the February review, that's exactly what the IESG did:
ignore the section. We would have happily supported any AD who wanted to review
or try the code by allowing access to our server.

> You probably also need to instruct the RFC
> Editor to remove it before publication?

[acm] OK
RFC Editor: This section is for the benefit of the Document Shepherd's form, 
and will be deleted prior to publication.

>>> I don't think this material from 8.4 made it into the Doc Shepherd's form,
which is one reason it's still hanging around...

> 
> This document uses RFC2119 keywords, but does not contain the recommended
> RFC8174 boilerplate. (It contains some text with a similar beginning.)
[acm] I think this is covered, section 1.1.
> 
> --------------------------------------------------------------------------
> -----
> All comments below are about very minor potential issues that you may choose to
> address in some way - or ignore - as you see fit. Some were flagged by
> automated tools, so there will likely be some false positives. There is no need
> to let me know what you did with these suggestions.

Ok, thanks.  I would like to know which tool(s) you used (and if free, 
even on a trial basis). Seemed to do a decent job.


> 
> Section 1, paragraph 5, nit:
> -    authors found that a high percentage of paths tested support UDP
> +    authors found that a high percentage of paths tested support the UDP
> +    
no                                                             ++++
> 
> Section 3, paragraph 8, nit:
> -        to less traffic crossing ISP gateways in future.
> +        to less traffic crossing ISP gateways in the future.
> +                                                 ++++
yes
> 
> Section 8.1, paragraph 2, nit:
> -    agressiveness (speed of ramp-up and down to the Maximum IP-Layer
> +    aggressiveness (speed of ramp-up and down to the Maximum IP-Layer
> +     
yes
> 
> Section 8.2, paragraph 5, nit:
> -    Another approach comes from Section 24 of RFC 2544[RFC2544] and its
> -                                              --------
> +    Another approach comes from Section 24 of [RFC2544] and its
yes
> 
> Section 8.3, paragraph 4, nit:
> -    The path measured may be state-full based on many factors, and the
> -                                  -  -
> +    The path measured may be stateful based on many factors, and the
yes
> 
> Section 14.2, paragraph 5, nit:
> -    measurements with a clear and concise rate reduction when warrented,
> -                                                                  ^
> +    measurements with a clear and concise rate reduction when warranted,
> +
yes

                                                                  ^
> 
> "I", paragraph 2, nit:
> >  to the specifications of other Standards Development Organizations
> (SDO) (t
> >                                 ^^^^^^^^^
> An apostrophe may be missing.
no
> 
> "S", paragraph 2, nit:
> > t Maximum IP-Layer Capacity is unknown (which is sometimes the case;
> service
> >                                        ^
> Unpaired symbol: ')' seems to be missing
yes
> 
> Section 6.3, paragraph 14, nit:
> > urement systems MUST be higher than than the smallest of the links on
> the pa
> >                                ^^^^^^^^^
> Possible typo: you repeated a word
yes
> 
> Section 6.3, paragraph 15, nit:
> > PM list over all dt-length intervals in I. Measurements according to
> these de
> >                                      ^^^^
> Did you mean "in me", "in her", "in him", "in us", or "in them"?
no
> 
> Section 8, paragraph 3, nit:
> > work, since this is an active test method and it will likely cause
> congestion
> >                                    ^^^^^^^^^^
> Use a comma before 'and' if it connects two independent clauses (unless
> they
> are closely connected and short).
no
> 
> Section 8.2, paragraph 4, nit:
> > etons (dt = 1 second) has proven to a be of practical value during tests
> of
> >                                     ^^^^
> After 'a', do not use a verb. Make sure that the spelling of 'be' is
> correct.
> If 'be' is the first word in a compound adjective, use a hyphen between
> the two
> words. Note: This error message can occur if you use a verb as a noun, and
> the
> word is not a noun in standard English.
yes
> 
> Section 8.3, paragraph 4, nit:
> > nts based on those packets. Many different traffic shapers and on-demand
> acce
> >                             ^^^^^^^^^^^^^^
> Consider using "Many".
alt solution, "different types of" 
> 
> Section 8.3, paragraph 6, nit:
> > , where packet losses may occur independently from the measurement
> sending ra
> >                                 ^^^^^^^^^^^^^^^^^^
> The usual collocation for "independently" is "of", not "from". Did you
> mean
> "independently of"?
yes
> 
> Section 10, paragraph 10, nit:
> > ffic (for example, the range of I duration values SHOULD be limited).
> The exa
> >                                 ^^^^^^^^^^
> Do not use a noun immediately after the pronoun 'I'. Use a verb or an
> adverb,
> or possibly some other part of speech.
yes
> 
> "R", paragraph 1, nit:
> > ble) seqErr = 0 # Measured count of any of Loss or Reordering
> impairments de
> >                                  ^^^^^^^^^
> Consider simply using "of" instead.
no
> 
> Section 14.1, paragraph 10, nit:
> > g a test when connectivity is lost for an longer interval (Feedback
> message o
> >                                        ^^
> Use "a" instead of 'an' if the following word doesn't start with a vowel
> sound,
> e.g. 'a sentence', 'a university'.
yes
> 
> Section 14.2, paragraph 10, nit:
> > easurements. A brief review of some of the other SHOULD-level
> requirements fo
> >                                ^^^^^^^^^^^
> If the text is a generality, 'of the' is not necessary.
yes
> 
> "Authors' Addresses", paragraph 2, nit:
> >  200 Laurel Avenue South Middletown,, NJ 07748 USA Phone: [phone]
> >                                    ^^
> Two consecutive commas
yes
> 
> "Authors' Addresses", paragraph 5, nit:
> >  200 Laurel Avenue South Middletown,, NJ 07748 USA Email: lencia@att
> >                                    ^^
> Two consecutive commas
yes
> 
>