Re: [ippm] Lars Eggert's Discuss on draft-ietf-ippm-capacity-metric-method-10: (with DISCUSS and COMMENT)

Martin Duke <martin.h.duke@gmail.com> Thu, 20 May 2021 19:03 UTC

Return-Path: <martin.h.duke@gmail.com>
X-Original-To: ippm@ietfa.amsl.com
Delivered-To: ippm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 08B453A225A; Thu, 20 May 2021 12:03:51 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.997
X-Spam-Level:
X-Spam-Status: No, score=-1.997 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3p9mhgttxcP4; Thu, 20 May 2021 12:03:46 -0700 (PDT)
Received: from mail-il1-x129.google.com (mail-il1-x129.google.com [IPv6:2607:f8b0:4864:20::129]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6DA153A2258; Thu, 20 May 2021 12:03:45 -0700 (PDT)
Received: by mail-il1-x129.google.com with SMTP id l15so11002632iln.8; Thu, 20 May 2021 12:03:45 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=g80UNaXocW5FxDEAEa6B8VMFit6rWahecS3N0dwrghA=; b=kDmX5V5DmVI4TGlpJpInbtO9mccFZFW+KyQoFbrJNwhqbL88RRBnhZFl/sFYQkJuWM KHHGhBZ1jvK4K871yWoegfvV56gBsnt5rWq4K07igOLhVXh+johWkSPRM+nSK7v9sZgS XQ6452dBN2niWCh0g3BtMOHwtXN/4zVZioFlhGLKAvDOKBDUC8Wiq75dnM8DKn6i7+F3 l/5gI/SaY2sPw9a0rjtjZ5bOsjwn0al0c1GotuOURFBv61evKyGtblk4DRvfs7/3PeXf 6J2v+S7eyYi6D404hqUQNqJJ7Sw3yvyyZXsS9uSh14xcHKZPh6ZwOxqlhOAKv2mXFJRg m62Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=g80UNaXocW5FxDEAEa6B8VMFit6rWahecS3N0dwrghA=; b=B5zhy4LOuPkdA7SEoF0tP5ZQ+FPXYmQKK//ngtW5WAQuu7SNStdphQOpQH2bjlvlE8 De10gAmy5RNVmrRsvRr2cRyqPOS20PEKW4ed2eb6395BBoHWJZ94oDTSW++nj1kafcNR 2rZGXoIHRPubIE+/Fh1GlTGCDDz/664I86okfnisyzvNMujWopr0lPpVqG2IAbRX1gQ6 i3gATnLORBLE8BSLZSZeM0P6IcpuAVMg1SyML2SwoGB+BdYbU7a1PR/Uxc5TzSR1Vzpm tGuXWnXsKGep4edyiXOt4sv7gNQhuLhyAziUQUJsoEJ7sqk6psdIJoWfynk8zIWFRfXZ TDEw==
X-Gm-Message-State: AOAM532PcNQcBbg5zTFL33SV3IVe4o08CrXjVT7VoDK0Wsiv76UgqdnD rydq0pp7XKw/vMsSP8LQDn1AytV1jlmyiPS0K94=
X-Google-Smtp-Source: ABdhPJwcjmNQSv9iX2QEw2uw1lCwirB3YdRNA4yKsWF2bvKfoHtB6+2iKmRe3bYGMJGEPoGItjaRkihTz+N982uy8qU=
X-Received: by 2002:a05:6e02:1bce:: with SMTP id x14mr5637316ilv.287.1621537422675; Thu, 20 May 2021 12:03:42 -0700 (PDT)
MIME-Version: 1.0
References: <162072161364.15626.11472410331094167599@ietfa.amsl.com> <1764ec14e5394838b2a2802c45561aab@att.com> <6B3E6E27-AE72-4ABB-AE28-0A74B3EE93D0@eggert.org> <CAM4esxQKCooYRGf_+=fxgXNHpAGr69JdavqWX3o0CHiGda+nMw@mail.gmail.com> <98764a1876b047038bc02ae37d5faabd@att.com> <CAM4esxQv3upjAToW-Ph1537VGsb7SbDmSY7FAq2O2JnOqnivWA@mail.gmail.com>
In-Reply-To: <CAM4esxQv3upjAToW-Ph1537VGsb7SbDmSY7FAq2O2JnOqnivWA@mail.gmail.com>
From: Martin Duke <martin.h.duke@gmail.com>
Date: Thu, 20 May 2021 12:03:31 -0700
Message-ID: <CAM4esxQDYNQfzJSdD4_kc2HzACVt2kG8bWN-_3sTSBkbtKMCjw@mail.gmail.com>
To: "MORTON JR., AL" <acmorton@att.com>
Cc: Lars Eggert <lars@eggert.org>, "ippm-chairs@ietf.org" <ippm-chairs@ietf.org>, "draft-ietf-ippm-capacity-metric-method@ietf.org" <draft-ietf-ippm-capacity-metric-method@ietf.org>, "ippm@ietf.org" <ippm@ietf.org>, Ian Swett <ianswett@google.com>, The IESG <iesg@ietf.org>, "tpauly@apple.com" <tpauly@apple.com>
Content-Type: multipart/alternative; boundary="00000000000011a1d105c2c79a3c"
Archived-At: <https://mailarchive.ietf.org/arch/msg/ippm/Di7KLtQtj2wqIh59UwrekFRvMxY>
Subject: Re: [ippm] Lars Eggert's Discuss on draft-ietf-ippm-capacity-metric-method-10: (with DISCUSS and COMMENT)
X-BeenThere: ippm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF IP Performance Metrics Working Group <ippm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ippm>, <mailto:ippm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ippm/>
List-Post: <mailto:ippm@ietf.org>
List-Help: <mailto:ippm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ippm>, <mailto:ippm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 20 May 2021 19:03:51 -0000

OK, we waived the downref.

If 7967 is going to be a perennial problem, I don't see that it meets any
of the downref criteria in 3967, and I would encourage the group to
consider updating its status to proposed standard.

On Wed, May 19, 2021 at 4:17 PM Martin Duke <martin.h.duke@gmail.com> wrote:

> Got it. According to RFC 8067, the IESG can just waive it when the AD
> review misses it rather than go through another last call. I'll open that
> subject at tomorrow's telechat -- unless the smart people in IPPM think
> that changes warrant a new IETF Last Call in their own right.
>
> On Wed, May 19, 2021 at 3:48 PM MORTON JR., AL <acmorton@att.com> wrote:
>
>> in-line,
>>
>> Al
>>
>>
>>
>> *From:* Martin Duke <martin.h.duke@gmail.com>
>> *Sent:* Wednesday, May 19, 2021 6:30 PM
>> *To:* Lars Eggert <lars@eggert.org>
>> *Cc:* MORTON JR., AL <acmorton@att.com>om>; ippm-chairs@ietf.org;
>> draft-ietf-ippm-capacity-metric-method@ietf.org; ippm@ietf.org; Ian
>> Swett <ianswett@google.com>om>; The IESG <iesg@ietf.org>rg>; tpauly@apple.com
>> *Subject:* Re: Lars Eggert's Discuss on
>> draft-ietf-ippm-capacity-metric-method-10: (with DISCUSS and COMMENT)
>>
>>
>>
>> OK, I've had a look at this, and I don't see why this is a normative
>> reference at all!
>>
>>
>>
>> The primary application of the metric and method of measurement
>>
>>    described here is the same as in Section 2 of [RFC7497] <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc7497*section-2__;Iw!!BhdT!xlfpewVBdgR2F4flazNgF3Ypkw-6ovu-odnTKamNrSa250ONVpnJdOv2Pnhu$> where:
>>
>>
>>
>>    o  The access portion of the network is the focus of this problem
>>
>>       statement.  The user typically subscribes to a service with
>>
>>       bidirectional access partly described by rates in bits per second.
>>
>>
>>
>> How is this necessary to implement the draft?
>>
>> *[acm] *
>>
>> *For Implementation, Not at all! But we use it to set the Scope and
>> Applicability, which is a Normative section by its very nature, IMO. The
>> Figure in RFC 7497 limits the applicability to a small number of
>> administrative domains at the edge of the Internet, including what most
>> people think of when they hear “Internet access”.*
>>
>>
>>
>> *So, you can’t implement this RFC *and use it properly* without this
>> reference.*
>>
>>
>>
>> *Adding the reference to RFC 7497 was a big step in resolving Magnus’
>> comments. He didn’t want the Capacity test used end-to-end on the Internet,
>> and RFC 7497 was the perfect solution.  But like all our framework RFCs,
>> it’s Informational.*
>>
>>
>>
>> Happy to be corrected on this!
>>
>> *[acm] *
>>
>> *I made my case, it’s your call.  We have a working version going for
>> Lars’ comments, so it’s an easy fix.*
>>
>>
>>
>> [Also, appropriately ashamed to have missed it in my AD review]
>>
>> *[acm] *
>>
>> *You didn’t miss it!  It wasn’t in the Scope/applicability section at
>> that time!  This Ref was added for Magnus!*
>>
>>
>>
>> Martni
>>
>>
>>
>> On Tue, May 18, 2021 at 12:05 AM Lars Eggert <lars@eggert.org> wrote:
>>
>> Hi Al,
>>
>> On 2021-5-17, at 21:00, MORTON JR., AL <acmorton@att.com> wrote:
>> >> DOWNREF from this Standards Track doc to Informational RFC7497.
>> >> I didn't see this called out in the Last Call, and it's also not a
>> >> document we have in the DOWNREF registry already.
>> > [acm]
>> > Ok, I think this means another IETF Last Call?  It's not the first time
>> > we've been tripped-up by keeping IPPM's Framework+updates and other
>> > requirements docs off the Standards-Track.
>> >
>> > I also think that https://datatracker.ietf.org/doc/html/rfc7497
>> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc7497__;!!BhdT!xlfpewVBdgR2F4flazNgF3Ypkw-6ovu-odnTKamNrSa250ONVpnJdPOyZrAO$>
>> will figure
>> > prominently in future work on the Capacity topic, such as a test
>> protocol, so we
>> > should add it to the DOWNREF registry.
>> >
>> > @@@@ Is the DOWNREF registry an action item for chairs or AD/Martin?
>>
>> yes, another LC will unfortunately be needed. It's an action item for
>> Martin.
>>
>> Note that everything below was comments, which are not blocking the
>> document:
>>
>> >> Section 1, paragraph 3, comment:
>> >>>   Here, the main use case is to assess the maximum capacity of the
>> >>>   access network, with specific performance criteria used in the
>> >>>   measurement.
>> >>
>> >> If the main use case is to measure access network paths, that
>> applicability
>> >> should be made very explicit, i.e., in the title and abstract.
>> >
>> > Two things:
>> >
>> > 1. The term "access" doesn't have a universally agreed definition
>> regarding the portion of the network it covers, and that's partly because
>> there are many architectures in use.
>> >
>> > 2. If we add access to the title or abstract, we are unnecessarily
>> limiting the scope that we define later. We refer to section of RFC 7497
>> where the figure includes the subscriber's device, multiple private
>> networks, and various devices and gateways ending at a transit gateway.
>> >
>> > So, I don't want to emphasize the term "access" as you propose.
>>
>> That's fine, too. When reading the document, I came away with the
>> impression that it did want to focus on measuring access networks, because
>> it actually says that in several places (e.g., the quote above.) If the
>> intent is for this to be a general metric for IP paths, maybe review the
>> places where the document talks about "access" to make sure they are not
>> giving the wrong impression.
>>
>> > Perhaps we can clarify this further in the Intro, because the Scope
>> says:
>> >
>> >   The primary application of the metric and method of measurement
>> >   described here is the same as in Section 2 of [RFC7497]...
>> >
>> > How about:
>> > Here, the main use case is to assess the maximum capacity of one or
>> more networks
>> > where the subscriber receives specific performance assurances, sometimes
>> > referred to as the Internet access.
>>
>> It's not only the intro, though. There is text talking about "access"
>> throughout the document.
>>
>> >> Section 2, paragraph 2, comment:
>> >>>   The scope of this memo is to define a metric and corresponding
>> method
>> >>>   to unambiguously perform Active measurements of Maximum IP-Layer
>> >>>   Capacity, along with related metrics and methods.
>> >>
>> >> I'm not sure what "unambiguously perform" is meant to express?
>> >
>> > [RG] is it unambiguously measure...? If yes roughly:
>> > "The scope of this memo is to define Active Measurement metrics and
>> corresponding methods
>> > to unambiguously determine Maximum IP-Layer Capacity and useful
>> secondary evaluations."
>> >
>> > [acm] With s/evaluations/metrics/ this WFM.
>>
>> WFM
>>
>> >> Section 2, paragraph 3, comment:
>> >>>   A local goal is to aid efficient test procedures where possible, and
>> >>>   to recommend reporting with additional interpretation of the
>> results.
>> >>
>> >> What is this goal "local" to - the IETF?
>> > [acm] Yes IETF, but that word choice seems unclear now.
>> >
>> > [RG] further s/aid/add??
>> >
>> >> Also, I'm unclear what "reporting
>> >> with additional interpretation of the results" is supposed to express.
>> >
>> > [acm] Let's try:
>> >
>> > Secondary goals are to add considerations for test procedures, and
>> > to provide interpretation of the Maximum IP-Layer Capacity results (to
>> > identify cases where more testing is warranted, possibly with alternate
>> > configurations).
>>
>> WFM
>>
>> >> Section 2, paragraph 10, comment:
>> >>>   - If a network operator is certain of the access capacity to be
>> >>>   validated, then testing MAY start with a fixed rate test at the
>> >>>   access capacity and avoid activating the load adjustment algorithm.
>> >>>   However, the stimulus for a diagnostic test (such as a subscriber
>> >>>   request) strongly implies that there is no certainty and the load
>> >>>   adjustment algorithm will be needed.
>> >>
>> >> Since the first sentence of this paragraph uses a RFC2119 MAY, I'd
>> suggest
>> >> rephrasing the second sentence to end with "there is no certainty and
>> use
>> >> of the load adjustment algorithm is RECOMMENDED."
>> >
>> > [RG] sounds good to me.
>> > [acm] WFM
>>
>> WFM
>>
>> >> Section 3, paragraph 4, comment:
>> >>>   1.  Internet access is no longer the bottleneck for many users.
>> >>
>> >> What is the bottleneck then, and if it's not the access bandwidth, why
>> is
>> >> a new metric for it helpful?
>> >
>> > [RG]+[acm] 1.  Internet access is no longer the bottleneck for many
>> users, but subscribers expect network providers to honor contracted access
>> performance.
>> >
>> > [acm] IOW, when you subscribe to a 1 Gbps service, then your ISP, you,
>> and possibly other parties want to assure that performance level is
>> delivered. If you test and confirm the subscribed performance level, then
>> you can seek the location of the bottleneck elsewhere.
>>
>> That context would be helpful to have in the document in some form, IMO.
>>
>> > Section 4, paragraph 4, comment:
>> >>>   o  Src, the address of a host (such as the globally routable IP
>> >>>      address).
>> >>>
>> >>>   o  Dst, the address of a host (such as the globally routable IP
>> >>>      address).
>> >>
>> >> For many hosts, their IP address is not globally routable, and hasn't
>> been
>> >> for decades. Or did you mean CPE?
>> >
>> > [acm] I think *host* is general-enough to include CPE that is the
>> source or destination of test packets. Also, "globally routable" is part of
>> an example.
>>
>> I still wonder if removing "globally routable" would reduce confusion
>> here, but OK.
>>
>> > Section 8.1, paragraph 1, comment:
>> >>> 8.1.  Load Rate Adjustment Algorithm
>> >>
>> >> I wonder why this section is trying to define a crude new AIMD scheme,
>> when we
>> >> have lots of good experience with TCP's AIMD approach, which converges
>> on
>> >> the available bandwidth relatively well?
>> >
>> > [acm] "crude"?? Our algorithm has the advantage of receiver
>> measurements of rate,
>> > loss, and delay. So, we aren't making inferences based on ACKs, we
>> communicate
>> > measurement feedback instead.
>>
>> Maybe I should have said "complex" :-)
>>
>> > This is not a new AIMD. I'm sure you haven't had time to review
>> > hundreds of test results where we compare our load adjustment algorithm
>> to
>> > TCP's AIMDs when seeking to measure the maximum IP-layer capacity.
>> >
>> > Please see section 4 of https://datatracker.ietf.org/doc/html/rfc8337
>> <https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/rfc8337__;!!BhdT!xlfpewVBdgR2F4flazNgF3Ypkw-6ovu-odnTKamNrSa250ONVpnJdMQ2_xii$>
>> > particularly the list of section 4.1 where Matt Mathis wrote:
>> >
>> >   o  TCP is a control system with circular dependencies -- everything
>> >      affects performance, including components that are explicitly not
>> >      part of the test (for example, the host processing power is not
>> >      in-scope of path performance tests).
>> >
>> > and this is similar to the statements I have heard for >20 years in
>> IETF,
>> > where the conclusion is that TCP was not designed as a measurement tool.
>> >
>> > So, it was a goal to avoid the limitations of TCP (windows, ACK
>> feedback,
>> > retransmissions, and the CCA that were available, including BBR and
>> BBRv2).
>> > The algorithm we defined is intended to be fast, adjustable in
>> straightforward
>> > ways, and (most important to Magnus) not to be deployed as a
>> general-purpose
>> > CCA in TCP. Our scope is infrequent, diagnostic measurements.
>>
>> Point taken.
>>
>> >> Section 8.1, paragraph 7, comment:
>> >>>   If the feedback indicates that no sequence number anomalies were
>> >>>   detected AND the delay range was below the lower threshold, the
>> >>>   offered load rate is increased.  If congestion has not been
>> confirmed
>> >>>   up to this point, the offered load rate is increased by more than
>> one
>> >>>   rate (e.g., Rx+10).  This allows the offered load to quickly reach a
>> >>>   near-maximum rate.  Conversely, if congestion has been previously
>> >>>   confirmed, the offered load rate is only increased by one (Rx+1).
>> >>
>> >> The first sentence talks about sequence number anomalies and delay
>> ranges.
>> >> The rest of the paragraph then talks about whether congestion and
>> whether it
>> >> was confirmed or not. Does this mean that (some amount of) sequence
>> number
>> >> anomalies and/or delay indicates congestion? Is that defined somewhere?
>> >
>> > [acm] Yes, in a paragraph that begins:
>> >     Lastly, the method for inferring congestion is that there were...
>> >
>> > I'll add a forward reference:
>> >     ... If congestion has not been confirmed up to this point (see below
>> >     for the method to declare congestion), the offered load rate is
>> increased ...
>>
>> WFM
>>
>> >> Section 8.1, paragraph 7, comment:
>> >>>   If the feedback indicates that sequence number anomalies were
>> >>>   detected OR the delay range was above the upper threshold, the
>> >>>   offered load rate is decreased.  The RECOMMENDED values are 0 for
>> >>
>> >> This doesn't agree with the previous paragraph, which says
>> "Conversely, if
>> >> congestion has been previously confirmed, the offered load rate is only
>> >> increased by one (Rx+1)." (Or I don't understand the difference between
>> >> congestion and sequence number anomalies/delay ranges; see previous
>> >> comment.)
>> >
>> > [acm]  Your "(Or ...)" is correct, we deferred the description of
>> congestion
>> > declaration to a later paragraph, where we require 2 feedback messages
>> at least:
>> >
>> >    Lastly, the method for inferring congestion is that there were
>> sequence
>> >    number anomalies AND/OR the delay range was above the upper threshold
>> >    for two consecutive feedback intervals. ...
>>
>> WFM
>>
>> > Section 8.4, paragraph 2, comment:
>> >>>   This section is for the benefit of the Document Shepherd's form, and
>> >>>   will be deleted prior to final review.
>> >>
>> >> Should this section have been deleted before the IESG review then?
>> Would
>> >> you like the IESG to ignore it?
>> >
>> > [acm] I think that during the February review, that's exactly what the
>> IESG did:
>> > ignore the section. We would have happily supported any AD who wanted
>> to review
>> > or try the code by allowing access to our server.
>> >
>> >> You probably also need to instruct the RFC
>> >> Editor to remove it before publication?
>> >
>> > [acm] OK
>> > RFC Editor: This section is for the benefit of the Document Shepherd's
>> form,
>> > and will be deleted prior to publication.
>> >
>> >>>> I don't think this material from 8.4 made it into the Doc Shepherd's
>> form,
>> > which is one reason it's still hanging around...
>>
>> WFM
>>
>> > This document uses RFC2119 keywords, but does not contain the
>> recommended
>> >> RFC8174 boilerplate. (It contains some text with a similar beginning.)
>> > [acm] I think this is covered, section 1.1.
>>
>> WFM
>>
>> >>
>> >>
>> --------------------------------------------------------------------------
>> >> -----
>> >> All comments below are about very minor potential issues that you may
>> choose to
>> >> address in some way - or ignore - as you see fit. Some were flagged by
>> >> automated tools, so there will likely be some false positives. There
>> is no need
>> >> to let me know what you did with these suggestions.
>> >
>> > Ok, thanks.  I would like to know which tool(s) you used (and if free,
>> > even on a trial basis). Seemed to do a decent job.
>>
>> https://github.com/larseggert/ietf-reviewtool
>> <https://urldefense.com/v3/__https:/github.com/larseggert/ietf-reviewtool__;!!BhdT!xlfpewVBdgR2F4flazNgF3Ypkw-6ovu-odnTKamNrSa250ONVpnJdGUm8Z7Q$>
>>
>> Thanks,
>> Lars
>>
>>