Re: [ippm] [Bloat] Fwd: New Version Notification for draft-cpaasch-ippm-responsiveness-00.txt

Erik Auerswald <auerswal@unix-ag.uni-kl.de> Sun, 15 August 2021 13:39 UTC

Return-Path: <auerswal@unix-ag.uni-kl.de>
X-Original-To: ippm@ietfa.amsl.com
Delivered-To: ippm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2C9B83A1629; Sun, 15 Aug 2021 06:39:43 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.897
X-Spam-Level:
X-Spam-Status: No, score=-1.897 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qGr4k_0-H2GT; Sun, 15 Aug 2021 06:39:36 -0700 (PDT)
Received: from mailgw1.uni-kl.de (mailgw1.uni-kl.de [IPv6:2001:638:208:120::220]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D9E633A1626; Sun, 15 Aug 2021 06:39:31 -0700 (PDT)
Received: from sushi.unix-ag.uni-kl.de (sushi.unix-ag.uni-kl.de [IPv6:2001:638:208:ef34:0:ff:fe00:65]) by mailgw1.uni-kl.de (8.14.4/8.14.4/Debian-8+deb8u2) with ESMTP id 17FDdNcC089000 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 15 Aug 2021 15:39:23 +0200
Received: from sushi.unix-ag.uni-kl.de (ip6-localhost [IPv6:::1]) by sushi.unix-ag.uni-kl.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id 17FDdMA9018804 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 15 Aug 2021 15:39:22 +0200
Received: (from auerswal@localhost) by sushi.unix-ag.uni-kl.de (8.14.4/8.14.4/Submit) id 17FDdMxW018803; Sun, 15 Aug 2021 15:39:22 +0200
Date: Sun, 15 Aug 2021 15:39:22 +0200
From: Erik Auerswald <auerswal@unix-ag.uni-kl.de>
To: bloat@lists.bufferbloat.net
Cc: draft-cpaasch-ippm-responsiveness@ietf.org, ippm@ietf.org
Message-ID: <20210815133922.GA18118@unix-ag.uni-kl.de>
References: <YRbm8ZqLdi3xs3bl@MacBook-Pro-2.local>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="opJtzjQTFsWo+cga"
Content-Disposition: inline
In-Reply-To: <YRbm8ZqLdi3xs3bl@MacBook-Pro-2.local>
Author: Erik Auerswald <auerswal@unix-ag.uni-kl.de>
User-Agent: Mutt/1.5.21 (2010-09-15)
Archived-At: <https://mailarchive.ietf.org/arch/msg/ippm/WI172KJm81PswFnt9WptX7l-kHc>
Subject: Re: [ippm] [Bloat] Fwd: New Version Notification for draft-cpaasch-ippm-responsiveness-00.txt
X-BeenThere: ippm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF IP Performance Metrics Working Group <ippm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ippm>, <mailto:ippm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ippm/>
List-Post: <mailto:ippm@ietf.org>
List-Help: <mailto:ippm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ippm>, <mailto:ippm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 15 Aug 2021 13:39:43 -0000

Hi,

I'd like to thank you for working on a nice I-D describing an interesting
and IMHO useful network measurement metric.

Since feedback was asked for, I'd like to try and provide constructive
feedback.

In general, I like the idea of "Round-trips per Minute" (RPM) as a
metric used to characterize (one aspect of) a network.  I do think that
introducing this would improve the status quo.  Since this RPM definition
comprises a specific way of adding load to the network and measuring a
complex metric, I think it is useful to "standardize" it.

I do not think RPM can replace all other metrics.  This is, in a way,
mentioned in the introduction, where it is suggested to add RPM to
existing measurement platforms.  As such I just want to point this out
more explicitely, but do not intend to diminish the RPM idea by this.
In short, I'd say it's complicated.

Bandwidth matters for bulk data transfer, e.g., downloading a huge update
required for playing a multiplayer game online.

Minimum latency matters for the feasibility of interactive applications,
e.g., controlling a toy car in your room vs. a robotic arm on the ISS
from Earth vs. orbital insertion around Mars from Earth.  For a more
mundane use case consider a voice conference.  (A good decade ago I
experienced a voice conferencing system running over IP that introduced
over one second of (minimum) latency and therefore was awkward to use.)

Expressing 'bufferbloat as a measure of "Round-trips per Minute" (RPM)'
exhibits (at least) two problems:

1. A high RPM value is associated with little bufferbloat problems.

2. A low RPM value may be caused by high minimum delay instead of
   bufferbloat.

I think that RPM (i.e., under working conditions) measures a network's
usefulness for interactive applications, but not necessarily bufferbloat.
I do think that RPM is in itself more generally useful than minimum
latency or bandwidth.

A combination of low minimum latency with low RPM value strongly hints
at bufferbloat.  Other combinations are less easily characterized.

Bufferbloat can still lie in hiding, e.g., when a link with bufferbloat
is not yet the bottleneck, or if the communications end-points are not
yet able to saturate the network inbetween.  Thus high bandwidth can
result in high RPM values despite (hidden) bufferbloat.

The "Measuring is Hard" section mentions additional complications.

All in all, I do think that "measuring bufferbloat" and "measuring RPM"
should not be used synonymously.  The I-D title clearly shows this:
RPM is measuring "Responsiveness under Working Conditions" which may be
affected by bufferbloat, among other potential factors, but is not in
itself bufferbloat.

Under the assumption that only a single value (performance score) is
considered, I do think that RPM is more generally useful than bandwidth
or idle latency.

On a meta-level, I think that the word "bufferbloat" is not used according
to a single self-consistent definition in the I-D.

Additionally, I think that the I-D should reference DNS, HTTP/2, and
TLS 1.3, since these protocols are required for implementing the RPM
measurement.  The same for JSON, I think.  Possibly URL.

Using "rpm.example" instead of "example.apple.com" would result in shorter
lines for the example JSON.

"host123.cdn.example" instead of "hostname123.cdnprovider.com" might be
a more appropriate example DNS name.

Adding an informative reference to RFC 2606 / BCP 32 might raise awareness
of the existence of a BCP on example DNS names.

Please find both a unified diff against the text rendering of the I-D,
and a word diff produced from the unified diff, attached to this email
in order to suggest editorial changes that are intended to improve the
reading experience.  They are intended for reading and (possibly partial)
manual application, since the text rendering of an I-D is usually not
the preferred form of editing it.

Thanks,
Erik
-- 
Always use the right tool for the job.
                        -- Rob Pike


On Fri, Aug 13, 2021 at 02:41:05PM -0700, Christoph Paasch via Bloat wrote:
> I already posted this to the RPM-list, but the audience here on bloat should
> be interested as well.
> 
> 
> This is the specification of Apple's responsiveness/RPM test. We believe that it
> would be good for the bufferbloat-effort to have a specification of how to
> quantify the extend of bufferbloat from a user's perspective. Our
> Internet-draft is a first step in that direction and we hope that it will
> kick off some collaboration.
> 
> 
> Feedback is very welcome!
> 
> 
> Cheers,
> Christoph
> 
> 
> ----- Forwarded message from internet-drafts@ietf.org -----
> 
> From: internet-drafts@ietf.org
> To: Christoph Paasch <cpaasch@apple.com>om>, Omer Shapira <oesh@apple.com>om>, Randall Meyer <rrm@apple.com>om>, Stuart Cheshire
> 	<cheshire@apple.com>
> Date: Fri, 13 Aug 2021 09:43:40 -0700
> Subject: New Version Notification for draft-cpaasch-ippm-responsiveness-00.txt
> 
> 
> A new version of I-D, draft-cpaasch-ippm-responsiveness-00.txt
> has been successfully submitted by Christoph Paasch and posted to the
> IETF repository.
> 
> Name:		draft-cpaasch-ippm-responsiveness
> Revision:	00
> Title:		Responsiveness under Working Conditions
> Document date:	2021-08-13
> Group:		Individual Submission
> Pages:		12
> URL:            https://www.ietf.org/archive/id/draft-cpaasch-ippm-responsiveness-00.txt
> Status:         https://datatracker.ietf.org/doc/draft-cpaasch-ippm-responsiveness/
> Htmlized:       https://datatracker.ietf.org/doc/html/draft-cpaasch-ippm-responsiveness
> 
> 
> Abstract:
>    Bufferbloat has been a long-standing problem on the Internet with
>    more than a decade of work on standardizing technical solutions,
>    implementations and testing.  However, to this date, bufferbloat is
>    still a very common problem for the end-users.  Everyone "knows" that
>    it is "normal" for a video conference to have problems when somebody
>    else on the same home-network is watching a 4K movie.
> 
>    The reason for this problem is not the lack of technical solutions,
>    but rather a lack of awareness of the problem-space, and a lack of
>    tooling to accurately measure the problem.  We believe that exposing
>    the problem of bufferbloat to the end-user by measuring the end-
>    users' experience at a high level will help to create the necessary
>    awareness.
> 
>    This document is a first attempt at specifying a measurement
>    methodology to evaluate bufferbloat the way common users are
>    experiencing it today, using today's most frequently used protocols
>    and mechanisms to accurately measure the user-experience.  We also
>    provide a way to express the bufferbloat as a measure of "Round-trips
>    per minute" (RPM) to have a more intuitive way for the users to
>    understand the notion of bufferbloat.
> 
>                                                                                   
> 
> 
> The IETF Secretariat
> 
> 
> 
> ----- End forwarded message -----
> _______________________________________________
> Bloat mailing list
> Bloat@lists.bufferbloat.net
> https://lists.bufferbloat.net/listinfo/bloat