Re: [ippm] Lars Eggert's Discuss on draft-ietf-ippm-capacity-metric-method-10: (with DISCUSS and COMMENT)

Martin Duke <martin.h.duke@gmail.com> Wed, 19 May 2021 22:30 UTC

Return-Path: <martin.h.duke@gmail.com>
X-Original-To: ippm@ietfa.amsl.com
Delivered-To: ippm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 23D963A21A4; Wed, 19 May 2021 15:30:03 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.096
X-Spam-Level:
X-Spam-Status: No, score=-2.096 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id brFMRkDWN8mw; Wed, 19 May 2021 15:29:58 -0700 (PDT)
Received: from mail-il1-x135.google.com (mail-il1-x135.google.com [IPv6:2607:f8b0:4864:20::135]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id A31503A219D; Wed, 19 May 2021 15:29:57 -0700 (PDT)
Received: by mail-il1-x135.google.com with SMTP id l15so8411519iln.8; Wed, 19 May 2021 15:29:57 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=3T7/yHtClo5RXtB+oPwlfIMbVc8NHWV7eRreu3NKeY4=; b=q1AN+GWckzeGhH9DiDGHOEhYA/eK3ekgRSuI0JU9cQfyOrWDS59mEFrCjNVtC8yU5v qBi04Q6n0xbizjSoptZFINNrNbpHhmVfZbwOFb8IrXjYD23Ybv7gVUwsujsw9ZoMIZQ3 TjFYGfURZ2Z2SjlWKC1abwjYxom8GYrkmYhdSMv46teFZilE9M8co0m4CB5agMViipNJ SHKpzNSA9jwWFOkdWBOSDd8zRFZlOkXeqjERSQZmzL6+B5UcV55xJd7mNyhv6vJ09WYN 7eSzaxm1OIAUlYaSHDU0mYG53ynVt9PkT1v99UCMEE0mdKaRpg3TNk6VviBwUACucOs9 Eh7w==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=3T7/yHtClo5RXtB+oPwlfIMbVc8NHWV7eRreu3NKeY4=; b=NPHnAxhhiYxAPlyQtnLA4JbqeUcl4aECL9IlMSL+hpoJW5MqPUHm4aTzs940fu1uF0 vLR+c5y6QztFbqggvHmV100AusolLhMxaxxPwFpkMa4j45d0ofb1QadpP7QyAFSUXGB8 /RLlaOQeHq6zVS2WJ0NwElw11m6WeGj3JS8NBAT7xqDiE7Q/RjfDTdyPXq7crI8j/Uqc tpynsPYYkzQgUwW+Du/iplAx6R3OgY1SiwSKqAblWKjkdWkvtF7iCEmzxsZekCIpkTca F8bY5eIeGPAcOWc7f54KWwMxP/67AXyUGIqGoHePsl77hb5wYuW8TlVkCqTLQe0kM7/L EdTQ==
X-Gm-Message-State: AOAM530dBWgW2GjQFYBswQ3uLY2vyH434agt5drCs/kyxgq3YJrxNHm6 p2MU7fM9rRZN2BZG937Oj4e6aZUDU+ASYH2pI4s=
X-Google-Smtp-Source: ABdhPJxUzh1eFeCjRxMQlrxXTFKJ5a13TXY+232M8KXpe2bz0Pegx9nGKRRK4klbUPNy30CwNB9E/+VpeX8rRsJnmcU=
X-Received: by 2002:a92:c24b:: with SMTP id k11mr1289990ilo.303.1621463395258; Wed, 19 May 2021 15:29:55 -0700 (PDT)
MIME-Version: 1.0
References: <162072161364.15626.11472410331094167599@ietfa.amsl.com> <1764ec14e5394838b2a2802c45561aab@att.com> <6B3E6E27-AE72-4ABB-AE28-0A74B3EE93D0@eggert.org>
In-Reply-To: <6B3E6E27-AE72-4ABB-AE28-0A74B3EE93D0@eggert.org>
From: Martin Duke <martin.h.duke@gmail.com>
Date: Wed, 19 May 2021 15:29:52 -0700
Message-ID: <CAM4esxQKCooYRGf_+=fxgXNHpAGr69JdavqWX3o0CHiGda+nMw@mail.gmail.com>
To: Lars Eggert <lars@eggert.org>
Cc: "MORTON JR., AL" <acmorton@att.com>, "ippm-chairs@ietf.org" <ippm-chairs@ietf.org>, "draft-ietf-ippm-capacity-metric-method@ietf.org" <draft-ietf-ippm-capacity-metric-method@ietf.org>, "ippm@ietf.org" <ippm@ietf.org>, Ian Swett <ianswett@google.com>, The IESG <iesg@ietf.org>, "tpauly@apple.com" <tpauly@apple.com>
Content-Type: multipart/alternative; boundary="000000000000b0d5de05c2b65d17"
Archived-At: <https://mailarchive.ietf.org/arch/msg/ippm/ObfVOkRXlTwav1cQPz9I7eptw3I>
Subject: Re: [ippm] Lars Eggert's Discuss on draft-ietf-ippm-capacity-metric-method-10: (with DISCUSS and COMMENT)
X-BeenThere: ippm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF IP Performance Metrics Working Group <ippm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ippm>, <mailto:ippm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ippm/>
List-Post: <mailto:ippm@ietf.org>
List-Help: <mailto:ippm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ippm>, <mailto:ippm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 19 May 2021 22:30:10 -0000

OK, I've had a look at this, and I don't see why this is a normative
reference at all!

The primary application of the metric and method of measurement
   described here is the same as in Section 2 of [RFC7497]
<https://datatracker.ietf.org/doc/html/rfc7497#section-2> where:

   o  The access portion of the network is the focus of this problem
      statement.  The user typically subscribes to a service with
      bidirectional access partly described by rates in bits per second.


How is this necessary to implement the draft?

Happy to be corrected on this!

[Also, appropriately ashamed to have missed it in my AD review]

Martni

On Tue, May 18, 2021 at 12:05 AM Lars Eggert <lars@eggert.org> wrote:

> Hi Al,
>
> On 2021-5-17, at 21:00, MORTON JR., AL <acmorton@att.com> wrote:
> >> DOWNREF from this Standards Track doc to Informational RFC7497.
> >> I didn't see this called out in the Last Call, and it's also not a
> >> document we have in the DOWNREF registry already.
> > [acm]
> > Ok, I think this means another IETF Last Call?  It's not the first time
> > we've been tripped-up by keeping IPPM's Framework+updates and other
> > requirements docs off the Standards-Track.
> >
> > I also think that https://datatracker.ietf.org/doc/html/rfc7497 will
> figure
> > prominently in future work on the Capacity topic, such as a test
> protocol, so we
> > should add it to the DOWNREF registry.
> >
> > @@@@ Is the DOWNREF registry an action item for chairs or AD/Martin?
>
> yes, another LC will unfortunately be needed. It's an action item for
> Martin.
>
> Note that everything below was comments, which are not blocking the
> document:
>
> >> Section 1, paragraph 3, comment:
> >>>   Here, the main use case is to assess the maximum capacity of the
> >>>   access network, with specific performance criteria used in the
> >>>   measurement.
> >>
> >> If the main use case is to measure access network paths, that
> applicability
> >> should be made very explicit, i.e., in the title and abstract.
> >
> > Two things:
> >
> > 1. The term "access" doesn't have a universally agreed definition
> regarding the portion of the network it covers, and that's partly because
> there are many architectures in use.
> >
> > 2. If we add access to the title or abstract, we are unnecessarily
> limiting the scope that we define later. We refer to section of RFC 7497
> where the figure includes the subscriber's device, multiple private
> networks, and various devices and gateways ending at a transit gateway.
> >
> > So, I don't want to emphasize the term "access" as you propose.
>
> That's fine, too. When reading the document, I came away with the
> impression that it did want to focus on measuring access networks, because
> it actually says that in several places (e.g., the quote above.) If the
> intent is for this to be a general metric for IP paths, maybe review the
> places where the document talks about "access" to make sure they are not
> giving the wrong impression.
>
> > Perhaps we can clarify this further in the Intro, because the Scope says:
> >
> >   The primary application of the metric and method of measurement
> >   described here is the same as in Section 2 of [RFC7497]...
> >
> > How about:
> > Here, the main use case is to assess the maximum capacity of one or more
> networks
> > where the subscriber receives specific performance assurances, sometimes
> > referred to as the Internet access.
>
> It's not only the intro, though. There is text talking about "access"
> throughout the document.
>
> >> Section 2, paragraph 2, comment:
> >>>   The scope of this memo is to define a metric and corresponding method
> >>>   to unambiguously perform Active measurements of Maximum IP-Layer
> >>>   Capacity, along with related metrics and methods.
> >>
> >> I'm not sure what "unambiguously perform" is meant to express?
> >
> > [RG] is it unambiguously measure...? If yes roughly:
> > "The scope of this memo is to define Active Measurement metrics and
> corresponding methods
> > to unambiguously determine Maximum IP-Layer Capacity and useful
> secondary evaluations."
> >
> > [acm] With s/evaluations/metrics/ this WFM.
>
> WFM
>
> >> Section 2, paragraph 3, comment:
> >>>   A local goal is to aid efficient test procedures where possible, and
> >>>   to recommend reporting with additional interpretation of the results.
> >>
> >> What is this goal "local" to - the IETF?
> > [acm] Yes IETF, but that word choice seems unclear now.
> >
> > [RG] further s/aid/add??
> >
> >> Also, I'm unclear what "reporting
> >> with additional interpretation of the results" is supposed to express.
> >
> > [acm] Let's try:
> >
> > Secondary goals are to add considerations for test procedures, and
> > to provide interpretation of the Maximum IP-Layer Capacity results (to
> > identify cases where more testing is warranted, possibly with alternate
> > configurations).
>
> WFM
>
> >> Section 2, paragraph 10, comment:
> >>>   - If a network operator is certain of the access capacity to be
> >>>   validated, then testing MAY start with a fixed rate test at the
> >>>   access capacity and avoid activating the load adjustment algorithm.
> >>>   However, the stimulus for a diagnostic test (such as a subscriber
> >>>   request) strongly implies that there is no certainty and the load
> >>>   adjustment algorithm will be needed.
> >>
> >> Since the first sentence of this paragraph uses a RFC2119 MAY, I'd
> suggest
> >> rephrasing the second sentence to end with "there is no certainty and
> use
> >> of the load adjustment algorithm is RECOMMENDED."
> >
> > [RG] sounds good to me.
> > [acm] WFM
>
> WFM
>
> >> Section 3, paragraph 4, comment:
> >>>   1.  Internet access is no longer the bottleneck for many users.
> >>
> >> What is the bottleneck then, and if it's not the access bandwidth, why
> is
> >> a new metric for it helpful?
> >
> > [RG]+[acm] 1.  Internet access is no longer the bottleneck for many
> users, but subscribers expect network providers to honor contracted access
> performance.
> >
> > [acm] IOW, when you subscribe to a 1 Gbps service, then your ISP, you,
> and possibly other parties want to assure that performance level is
> delivered. If you test and confirm the subscribed performance level, then
> you can seek the location of the bottleneck elsewhere.
>
> That context would be helpful to have in the document in some form, IMO.
>
> > Section 4, paragraph 4, comment:
> >>>   o  Src, the address of a host (such as the globally routable IP
> >>>      address).
> >>>
> >>>   o  Dst, the address of a host (such as the globally routable IP
> >>>      address).
> >>
> >> For many hosts, their IP address is not globally routable, and hasn't
> been
> >> for decades. Or did you mean CPE?
> >
> > [acm] I think *host* is general-enough to include CPE that is the source
> or destination of test packets. Also, "globally routable" is part of an
> example.
>
> I still wonder if removing "globally routable" would reduce confusion
> here, but OK.
>
> > Section 8.1, paragraph 1, comment:
> >>> 8.1.  Load Rate Adjustment Algorithm
> >>
> >> I wonder why this section is trying to define a crude new AIMD scheme,
> when we
> >> have lots of good experience with TCP's AIMD approach, which converges
> on
> >> the available bandwidth relatively well?
> >
> > [acm] "crude"?? Our algorithm has the advantage of receiver measurements
> of rate,
> > loss, and delay. So, we aren't making inferences based on ACKs, we
> communicate
> > measurement feedback instead.
>
> Maybe I should have said "complex" :-)
>
> > This is not a new AIMD. I'm sure you haven't had time to review
> > hundreds of test results where we compare our load adjustment algorithm
> to
> > TCP's AIMDs when seeking to measure the maximum IP-layer capacity.
> >
> > Please see section 4 of https://datatracker.ietf.org/doc/html/rfc8337
> > particularly the list of section 4.1 where Matt Mathis wrote:
> >
> >   o  TCP is a control system with circular dependencies -- everything
> >      affects performance, including components that are explicitly not
> >      part of the test (for example, the host processing power is not
> >      in-scope of path performance tests).
> >
> > and this is similar to the statements I have heard for >20 years in IETF,
> > where the conclusion is that TCP was not designed as a measurement tool.
> >
> > So, it was a goal to avoid the limitations of TCP (windows, ACK feedback,
> > retransmissions, and the CCA that were available, including BBR and
> BBRv2).
> > The algorithm we defined is intended to be fast, adjustable in
> straightforward
> > ways, and (most important to Magnus) not to be deployed as a
> general-purpose
> > CCA in TCP. Our scope is infrequent, diagnostic measurements.
>
> Point taken.
>
> >> Section 8.1, paragraph 7, comment:
> >>>   If the feedback indicates that no sequence number anomalies were
> >>>   detected AND the delay range was below the lower threshold, the
> >>>   offered load rate is increased.  If congestion has not been confirmed
> >>>   up to this point, the offered load rate is increased by more than one
> >>>   rate (e.g., Rx+10).  This allows the offered load to quickly reach a
> >>>   near-maximum rate.  Conversely, if congestion has been previously
> >>>   confirmed, the offered load rate is only increased by one (Rx+1).
> >>
> >> The first sentence talks about sequence number anomalies and delay
> ranges.
> >> The rest of the paragraph then talks about whether congestion and
> whether it
> >> was confirmed or not. Does this mean that (some amount of) sequence
> number
> >> anomalies and/or delay indicates congestion? Is that defined somewhere?
> >
> > [acm] Yes, in a paragraph that begins:
> >     Lastly, the method for inferring congestion is that there were...
> >
> > I'll add a forward reference:
> >     ... If congestion has not been confirmed up to this point (see below
> >     for the method to declare congestion), the offered load rate is
> increased ...
>
> WFM
>
> >> Section 8.1, paragraph 7, comment:
> >>>   If the feedback indicates that sequence number anomalies were
> >>>   detected OR the delay range was above the upper threshold, the
> >>>   offered load rate is decreased.  The RECOMMENDED values are 0 for
> >>
> >> This doesn't agree with the previous paragraph, which says "Conversely,
> if
> >> congestion has been previously confirmed, the offered load rate is only
> >> increased by one (Rx+1)." (Or I don't understand the difference between
> >> congestion and sequence number anomalies/delay ranges; see previous
> >> comment.)
> >
> > [acm]  Your "(Or ...)" is correct, we deferred the description of
> congestion
> > declaration to a later paragraph, where we require 2 feedback messages
> at least:
> >
> >    Lastly, the method for inferring congestion is that there were
> sequence
> >    number anomalies AND/OR the delay range was above the upper threshold
> >    for two consecutive feedback intervals. ...
>
> WFM
>
> > Section 8.4, paragraph 2, comment:
> >>>   This section is for the benefit of the Document Shepherd's form, and
> >>>   will be deleted prior to final review.
> >>
> >> Should this section have been deleted before the IESG review then? Would
> >> you like the IESG to ignore it?
> >
> > [acm] I think that during the February review, that's exactly what the
> IESG did:
> > ignore the section. We would have happily supported any AD who wanted to
> review
> > or try the code by allowing access to our server.
> >
> >> You probably also need to instruct the RFC
> >> Editor to remove it before publication?
> >
> > [acm] OK
> > RFC Editor: This section is for the benefit of the Document Shepherd's
> form,
> > and will be deleted prior to publication.
> >
> >>>> I don't think this material from 8.4 made it into the Doc Shepherd's
> form,
> > which is one reason it's still hanging around...
>
> WFM
>
> > This document uses RFC2119 keywords, but does not contain the recommended
> >> RFC8174 boilerplate. (It contains some text with a similar beginning.)
> > [acm] I think this is covered, section 1.1.
>
> WFM
>
> >>
> >>
> --------------------------------------------------------------------------
> >> -----
> >> All comments below are about very minor potential issues that you may
> choose to
> >> address in some way - or ignore - as you see fit. Some were flagged by
> >> automated tools, so there will likely be some false positives. There is
> no need
> >> to let me know what you did with these suggestions.
> >
> > Ok, thanks.  I would like to know which tool(s) you used (and if free,
> > even on a trial basis). Seemed to do a decent job.
>
> https://github.com/larseggert/ietf-reviewtool
>
> Thanks,
> Lars
>
>