Re: [ippm] Lars Eggert's Discuss on draft-ietf-ippm-capacity-metric-method-10: (with DISCUSS and COMMENT)

[Martin Duke <[e-mail]>]

Got it. According to RFC 8067, the IESG can just waive it when the AD
review misses it rather than go through another last call. I'll open that
subject at tomorrow's telechat -- unless the smart people in IPPM think
that changes warrant a new IETF Last Call in their own right.

On Wed, May 19, 2021 at 3:48 PM MORTON JR., AL <[e-mail]> wrote:

> in-line,
>
> Al
>
>
>
> *From:* Martin Duke <[e-mail]>
> *Sent:* Wednesday, May 19, 2021 6:30 PM
> *To:* Lars Eggert <[e-mail]>
> *Cc:* MORTON JR., AL <[e-mail]>; [e-mail];
> [e-mail]; [e-mail]; Ian Swett
> <[e-mail]>; The IESG <[e-mail]>; [e-mail]
> *Subject:* Re: Lars Eggert's Discuss on
> draft-ietf-ippm-capacity-metric-method-10: (with DISCUSS and COMMENT)
>
>
>
> OK, I've had a look at this, and I don't see why this is a normative
> reference at all!
>
>
>
> The primary application of the metric and method of measurement
>
>    described here is the same as in Section 2 of [RFC7497] <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc7497*section-2__;Iw!!BhdT!xlfpewVBdgR2F4flazNgF3Ypkw-6ovu-odnTKamNrSa250ONVpnJdOv2Pnhu$> where:
>
>
>
>    o  The access portion of the network is the focus of this problem
>
>       statement.  The user typically subscribes to a service with
>
>       bidirectional access partly described by rates in bits per second.
>
>
>
> How is this necessary to implement the draft?
>
> *[acm] *
>
> *For Implementation, Not at all! But we use it to set the Scope and
> Applicability, which is a Normative section by its very nature, IMO. The
> Figure in RFC 7497 limits the applicability to a small number of
> administrative domains at the edge of the Internet, including what most
> people think of when they hear “Internet access”.*
>
>
>
> *So, you can’t implement this RFC *and use it properly* without this
> reference.*
>
>
>
> *Adding the reference to RFC 7497 was a big step in resolving Magnus’
> comments. He didn’t want the Capacity test used end-to-end on the Internet,
> and RFC 7497 was the perfect solution.  But like all our framework RFCs,
> it’s Informational.*
>
>
>
> Happy to be corrected on this!
>
> *[acm] *
>
> *I made my case, it’s your call.  We have a working version going for
> Lars’ comments, so it’s an easy fix.*
>
>
>
> [Also, appropriately ashamed to have missed it in my AD review]
>
> *[acm] *
>
> *You didn’t miss it!  It wasn’t in the Scope/applicability section at that
> time!  This Ref was added for Magnus!*
>
>
>
> Martni
>
>
>
> On Tue, May 18, 2021 at 12:05 AM Lars Eggert <[e-mail]> wrote:
>
> Hi Al,
>
> On 2021-5-17, at 21:00, MORTON JR., AL <[e-mail]> wrote:
> >> DOWNREF from this Standards Track doc to Informational RFC7497.
> >> I didn't see this called out in the Last Call, and it's also not a
> >> document we have in the DOWNREF registry already.
> > [acm]
> > Ok, I think this means another IETF Last Call?  It's not the first time
> > we've been tripped-up by keeping IPPM's Framework+updates and other
> > requirements docs off the Standards-Track.
> >
> > I also think that https://datatracker.ietf.org/doc/html/rfc7497
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc7497__;!!BhdT!xlfpewVBdgR2F4flazNgF3Ypkw-6ovu-odnTKamNrSa250ONVpnJdPOyZrAO$>
> will figure
> > prominently in future work on the Capacity topic, such as a test
> protocol, so we
> > should add it to the DOWNREF registry.
> >
> > @@@@ Is the DOWNREF registry an action item for chairs or AD/Martin?
>
> yes, another LC will unfortunately be needed. It's an action item for
> Martin.
>
> Note that everything below was comments, which are not blocking the
> document:
>
> >> Section 1, paragraph 3, comment:
> >>>   Here, the main use case is to assess the maximum capacity of the
> >>>   access network, with specific performance criteria used in the
> >>>   measurement.
> >>
> >> If the main use case is to measure access network paths, that
> applicability
> >> should be made very explicit, i.e., in the title and abstract.
> >
> > Two things:
> >
> > 1. The term "access" doesn't have a universally agreed definition
> regarding the portion of the network it covers, and that's partly because
> there are many architectures in use.
> >
> > 2. If we add access to the title or abstract, we are unnecessarily
> limiting the scope that we define later. We refer to section of RFC 7497
> where the figure includes the subscriber's device, multiple private
> networks, and various devices and gateways ending at a transit gateway.
> >
> > So, I don't want to emphasize the term "access" as you propose.
>
> That's fine, too. When reading the document, I came away with the
> impression that it did want to focus on measuring access networks, because
> it actually says that in several places (e.g., the quote above.) If the
> intent is for this to be a general metric for IP paths, maybe review the
> places where the document talks about "access" to make sure they are not
> giving the wrong impression.
>
> > Perhaps we can clarify this further in the Intro, because the Scope says:
> >
> >   The primary application of the metric and method of measurement
> >   described here is the same as in Section 2 of [RFC7497]...
> >
> > How about:
> > Here, the main use case is to assess the maximum capacity of one or more
> networks
> > where the subscriber receives specific performance assurances, sometimes
> > referred to as the Internet access.
>
> It's not only the intro, though. There is text talking about "access"
> throughout the document.
>
> >> Section 2, paragraph 2, comment:
> >>>   The scope of this memo is to define a metric and corresponding method
> >>>   to unambiguously perform Active measurements of Maximum IP-Layer
> >>>   Capacity, along with related metrics and methods.
> >>
> >> I'm not sure what "unambiguously perform" is meant to express?
> >
> > [RG] is it unambiguously measure...? If yes roughly:
> > "The scope of this memo is to define Active Measurement metrics and
> corresponding methods
> > to unambiguously determine Maximum IP-Layer Capacity and useful
> secondary evaluations."
> >
> > [acm] With s/evaluations/metrics/ this WFM.
>
> WFM
>
> >> Section 2, paragraph 3, comment:
> >>>   A local goal is to aid efficient test procedures where possible, and
> >>>   to recommend reporting with additional interpretation of the results.
> >>
> >> What is this goal "local" to - the IETF?
> > [acm] Yes IETF, but that word choice seems unclear now.
> >
> > [RG] further s/aid/add??
> >
> >> Also, I'm unclear what "reporting
> >> with additional interpretation of the results" is supposed to express.
> >
> > [acm] Let's try:
> >
> > Secondary goals are to add considerations for test procedures, and
> > to provide interpretation of the Maximum IP-Layer Capacity results (to
> > identify cases where more testing is warranted, possibly with alternate
> > configurations).
>
> WFM
>
> >> Section 2, paragraph 10, comment:
> >>>   - If a network operator is certain of the access capacity to be
> >>>   validated, then testing MAY start with a fixed rate test at the
> >>>   access capacity and avoid activating the load adjustment algorithm.
> >>>   However, the stimulus for a diagnostic test (such as a subscriber
> >>>   request) strongly implies that there is no certainty and the load
> >>>   adjustment algorithm will be needed.
> >>
> >> Since the first sentence of this paragraph uses a RFC2119 MAY, I'd
> suggest
> >> rephrasing the second sentence to end with "there is no certainty and
> use
> >> of the load adjustment algorithm is RECOMMENDED."
> >
> > [RG] sounds good to me.
> > [acm] WFM
>
> WFM
>
> >> Section 3, paragraph 4, comment:
> >>>   1.  Internet access is no longer the bottleneck for many users.
> >>
> >> What is the bottleneck then, and if it's not the access bandwidth, why
> is
> >> a new metric for it helpful?
> >
> > [RG]+[acm] 1.  Internet access is no longer the bottleneck for many
> users, but subscribers expect network providers to honor contracted access
> performance.
> >
> > [acm] IOW, when you subscribe to a 1 Gbps service, then your ISP, you,
> and possibly other parties want to assure that performance level is
> delivered. If you test and confirm the subscribed performance level, then
> you can seek the location of the bottleneck elsewhere.
>
> That context would be helpful to have in the document in some form, IMO.
>
> > Section 4, paragraph 4, comment:
> >>>   o  Src, the address of a host (such as the globally routable IP
> >>>      address).
> >>>
> >>>   o  Dst, the address of a host (such as the globally routable IP
> >>>      address).
> >>
> >> For many hosts, their IP address is not globally routable, and hasn't
> been
> >> for decades. Or did you mean CPE?
> >
> > [acm] I think *host* is general-enough to include CPE that is the source
> or destination of test packets. Also, "globally routable" is part of an
> example.
>
> I still wonder if removing "globally routable" would reduce confusion
> here, but OK.
>
> > Section 8.1, paragraph 1, comment:
> >>> 8.1.  Load Rate Adjustment Algorithm
> >>
> >> I wonder why this section is trying to define a crude new AIMD scheme,
> when we
> >> have lots of good experience with TCP's AIMD approach, which converges
> on
> >> the available bandwidth relatively well?
> >
> > [acm] "crude"?? Our algorithm has the advantage of receiver measurements
> of rate,
> > loss, and delay. So, we aren't making inferences based on ACKs, we
> communicate
> > measurement feedback instead.
>
> Maybe I should have said "complex" :-)
>
> > This is not a new AIMD. I'm sure you haven't had time to review
> > hundreds of test results where we compare our load adjustment algorithm
> to
> > TCP's AIMDs when seeking to measure the maximum IP-layer capacity.
> >
> > Please see section 4 of https://datatracker.ietf.org/doc/html/rfc8337
> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc8337__;!!BhdT!xlfpewVBdgR2F4flazNgF3Ypkw-6ovu-odnTKamNrSa250ONVpnJdMQ2_xii$>
> > particularly the list of section 4.1 where Matt Mathis wrote:
> >
> >   o  TCP is a control system with circular dependencies -- everything
> >      affects performance, including components that are explicitly not
> >      part of the test (for example, the host processing power is not
> >      in-scope of path performance tests).
> >
> > and this is similar to the statements I have heard for >20 years in IETF,
> > where the conclusion is that TCP was not designed as a measurement tool.
> >
> > So, it was a goal to avoid the limitations of TCP (windows, ACK feedback,
> > retransmissions, and the CCA that were available, including BBR and
> BBRv2).
> > The algorithm we defined is intended to be fast, adjustable in
> straightforward
> > ways, and (most important to Magnus) not to be deployed as a
> general-purpose
> > CCA in TCP. Our scope is infrequent, diagnostic measurements.
>
> Point taken.
>
> >> Section 8.1, paragraph 7, comment:
> >>>   If the feedback indicates that no sequence number anomalies were
> >>>   detected AND the delay range was below the lower threshold, the
> >>>   offered load rate is increased.  If congestion has not been confirmed
> >>>   up to this point, the offered load rate is increased by more than one
> >>>   rate (e.g., Rx+10).  This allows the offered load to quickly reach a
> >>>   near-maximum rate.  Conversely, if congestion has been previously
> >>>   confirmed, the offered load rate is only increased by one (Rx+1).
> >>
> >> The first sentence talks about sequence number anomalies and delay
> ranges.
> >> The rest of the paragraph then talks about whether congestion and
> whether it
> >> was confirmed or not. Does this mean that (some amount of) sequence
> number
> >> anomalies and/or delay indicates congestion? Is that defined somewhere?
> >
> > [acm] Yes, in a paragraph that begins:
> >     Lastly, the method for inferring congestion is that there were...
> >
> > I'll add a forward reference:
> >     ... If congestion has not been confirmed up to this point (see below
> >     for the method to declare congestion), the offered load rate is
> increased ...
>
> WFM
>
> >> Section 8.1, paragraph 7, comment:
> >>>   If the feedback indicates that sequence number anomalies were
> >>>   detected OR the delay range was above the upper threshold, the
> >>>   offered load rate is decreased.  The RECOMMENDED values are 0 for
> >>
> >> This doesn't agree with the previous paragraph, which says "Conversely,
> if
> >> congestion has been previously confirmed, the offered load rate is only
> >> increased by one (Rx+1)." (Or I don't understand the difference between
> >> congestion and sequence number anomalies/delay ranges; see previous
> >> comment.)
> >
> > [acm]  Your "(Or ...)" is correct, we deferred the description of
> congestion
> > declaration to a later paragraph, where we require 2 feedback messages
> at least:
> >
> >    Lastly, the method for inferring congestion is that there were
> sequence
> >    number anomalies AND/OR the delay range was above the upper threshold
> >    for two consecutive feedback intervals. ...
>
> WFM
>
> > Section 8.4, paragraph 2, comment:
> >>>   This section is for the benefit of the Document Shepherd's form, and
> >>>   will be deleted prior to final review.
> >>
> >> Should this section have been deleted before the IESG review then? Would
> >> you like the IESG to ignore it?
> >
> > [acm] I think that during the February review, that's exactly what the
> IESG did:
> > ignore the section. We would have happily supported any AD who wanted to
> review
> > or try the code by allowing access to our server.
> >
> >> You probably also need to instruct the RFC
> >> Editor to remove it before publication?
> >
> > [acm] OK
> > RFC Editor: This section is for the benefit of the Document Shepherd's
> form,
> > and will be deleted prior to publication.
> >
> >>>> I don't think this material from 8.4 made it into the Doc Shepherd's
> form,
> > which is one reason it's still hanging around...
>
> WFM
>
> > This document uses RFC2119 keywords, but does not contain the recommended
> >> RFC8174 boilerplate. (It contains some text with a similar beginning.)
> > [acm] I think this is covered, section 1.1.
>
> WFM
>
> >>
> >>
> --------------------------------------------------------------------------
> >> -----
> >> All comments below are about very minor potential issues that you may
> choose to
> >> address in some way - or ignore - as you see fit. Some were flagged by
> >> automated tools, so there will likely be some false positives. There is
> no need
> >> to let me know what you did with these suggestions.
> >
> > Ok, thanks.  I would like to know which tool(s) you used (and if free,
> > even on a trial basis). Seemed to do a decent job.
>
> https://github.com/larseggert/ietf-reviewtool
> <https://urldefense.com/v3/__https:/github.com/larseggert/ietf-reviewtool__;!!BhdT!xlfpewVBdgR2F4flazNgF3Ypkw-6ovu-odnTKamNrSa250ONVpnJdGUm8Z7Q$>
>
> Thanks,
> Lars
>
>