Re: [ippm] Adoption call for draft-olden-ippm-qoo-02

Bjørn Ivar Teigen <bjorn@domos.no> Fri, 02 February 2024 14:53 UTC

Return-Path: <bjorn@domos.no>
X-Original-To: ippm@ietfa.amsl.com
Delivered-To: ippm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 04606C1519A8 for <ippm@ietfa.amsl.com>; Fri, 2 Feb 2024 06:53:07 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.896
X-Spam-Level:
X-Spam-Status: No, score=-0.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001, URI_DOTEDU=1] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=domos-no.20230601.gappssmtp.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2oTZ8F53CS7O for <ippm@ietfa.amsl.com>; Fri, 2 Feb 2024 06:53:02 -0800 (PST)
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com [IPv6:2a00:1450:4864:20::12b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C8F4DC151524 for <ippm@ietf.org>; Fri, 2 Feb 2024 06:53:02 -0800 (PST)
Received: by mail-lf1-x12b.google.com with SMTP id 2adb3069b0e04-511206d1c89so2982661e87.1 for <ippm@ietf.org>; Fri, 02 Feb 2024 06:53:02 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=domos-no.20230601.gappssmtp.com; s=20230601; t=1706885580; x=1707490380; darn=ietf.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=q56ErDB5t85MAyTKsuO33y6ADqPqjcdyDoJfivhiP2M=; b=Af16K+bjdno2F6DAgAVw9T2aZJij5S5WuuzFiU+QSK9lVgQ3EweoM4SCqyZLivmUwN 9egXozJDfMk3wl81vBvH8LrhGiHHLy+igrqsfECanVlIDYlU+bdlG3YUFCIYp1nqz/dU HP8BLtdF3zqhjt/86vSWIRmTdv2TfqefWjjWvA641tKSBibxKD/UKVtWL+gpAuZGajxa bjZu+Qr3jYxH2sHdhGB/pBBqU6YaU1PBkvZbMITIZe4lk2uXHsfQ6DDij2j1FtZ4YEyS LVxiWmTTOYln+Rs+duL0kJ10puA8IR9dgHYJo1B/EnZZc/+SY/acEIBfZntnTEycTnsn JzDw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706885580; x=1707490380; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=q56ErDB5t85MAyTKsuO33y6ADqPqjcdyDoJfivhiP2M=; b=d2Mb7G6A6CRTUu00p0Y5Qw0e9AY4qpxMCXBLx6Pp1ejcsgQ2nXrF8cq9p3ivcy0Row juwylloYfxBP/1+c4JEb1+jxFJvkLuk5TYLF6Elhz6lGj36Pdt8yTQuAHymfcZatmbgA ReNWG+vOmx87S4q4BnT7MwZzfj4SmNmTjktNw/IFj4VHX9DO4OAJCNX1wQaE+4M73TwM CuV4aDBps25sSWU/k+bnI520cRxjKtmPpsClICjJRfjU30XaGXvur2B/FKcrXu8ygIiA F3dsNBJZ0CiPyonFT2X5XBLHWHZLUgXMwh7aRlipFSeNcLejJ7rAbi1LhW0G7f2sC9HX njKw==
X-Gm-Message-State: AOJu0YyPzpR0fJaJt6YTU8/AewxfKgF9N+Pj4gemDnyPvpFQ6t6EvAro H1BByZSg6exs3pYEHDOKOvjFWV03bdU0j1+0qlEJYxx1FMattRmv4rxxTdAswAcV5OtaL5zr52Q VgpBGIY0JG564JoENk+zoZCFhnBSFclVoJGN3gXyZOQhW+kDBQLubdg==
X-Google-Smtp-Source: AGHT+IHZO7m/Yl6szwqDM50K/YUVxgHWq9XzTpA/vchKgZYnIaEu2M4n8E6RxFVjtWWdZkquaqE1P10BC8BsDDPIDYE=
X-Received: by 2002:a05:6512:2828:b0:511:31b4:ac16 with SMTP id cf40-20020a056512282800b0051131b4ac16mr1797986lfb.47.1706885579482; Fri, 02 Feb 2024 06:52:59 -0800 (PST)
MIME-Version: 1.0
References: <B92594EC-B6FE-4160-94E3-AA65EE65EDBC@ifi.uio.no>
In-Reply-To: <B92594EC-B6FE-4160-94E3-AA65EE65EDBC@ifi.uio.no>
From: Bjørn Ivar Teigen <bjorn@domos.no>
Date: Fri, 02 Feb 2024 15:52:48 +0100
Message-ID: <CAKf5G6LL4h=Pg6Ysnw0N0oPGr-Vq+XO-FRkP_ewHcLpME=_eSw@mail.gmail.com>
To: Michael Welzl <michawe@ifi.uio.no>
Cc: ippm@ietf.org
Content-Type: multipart/alternative; boundary="000000000000a33cff06106744c5"
Archived-At: <https://mailarchive.ietf.org/arch/msg/ippm/E1Qx5L8BzZd-Ri3BOVIhYGm4W9k>
Subject: Re: [ippm] Adoption call for draft-olden-ippm-qoo-02
X-BeenThere: ippm@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: IETF IP Performance Metrics Working Group <ippm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ippm>, <mailto:ippm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ippm/>
List-Post: <mailto:ippm@ietf.org>
List-Help: <mailto:ippm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ippm>, <mailto:ippm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 02 Feb 2024 14:53:07 -0000

Hi Michael,

Thanks for the feedback. Much appreciated!

On Wed, 31 Jan 2024 at 11:39, Michael Welzl <michawe@ifi.uio.no> wrote:

> Dear all,
>
> I have just reviewed both draft-olden-ippm-qoo-02 and
> draft-teigen-ippm-app-quality-metric-reqs-02. I only wanted to read the qoo
> document but I also found that I lack key information unless I read both -
> and so I agree with the stated suggestion that it would be good to handle
> them in tandem.
>
> My general conclusion is that this is *very* useful - I applaud the
> approach of trying to be pragmatic and simple towards users here, and
> clarifying the different needs of app developers, users and net admins.
> Surely a latency distribution is great input for “quality" - this all seems
> "just right" to me.
> Of course, there will be some open questions… I got some, as I read the
> qoo-document (e.g., about the granularity of how packet loss is handled
> here, and the linearity of “better” or “worse”), and I found them all
> discussed in section 7 - so this section is much appreciated.
>
> All in all, I definitely support WG adoption.
>
> Please find detailed comments on both drafts below - mostly just nits.
>
> Cheers,
> Michael
>
>
> ========
>
>
> draft-teigen-ippm-app-quality-metric-reqs-02
>
> Section 3 heading:  stray “,” at the end
> "Compositionality is essential for fault detection and accountability.”
>  I think this should be: Composability
> Section 4.8: "It, therefore, measures something very close to the
> user-perceived application performance of HTTP-based applications.”
> I think that it’s quite possible to implement a relatively large one-way
> transfer over HTTP; perhaps instead you mean “of applications that involve
> many handshakes, e.g. typical HTTP-based web surfing.”
> Next sentence: I believe that "well-suited to detecting bufferbloat“
> should either be “for detecting” or “to detect”.
>
> Thanks. I agree with your comments and have made an issue to address them,
here: https://github.com/domoslabs/AppQualityMetricID/issues/4


>
> draft-olden-ippm-qoo-02
>
> As a general editorial comment, I think the word “useless” isn’t very
> suitable here. “Unusuable” would be more precise. E.g., you talk about
> “useless quality” at some point, which I think doesn’t exist - “poor
> quality” does. Similarly, an application doesn’t become “useless” because
> the network quality is poor - it becomes “unusable”.
> (e.g., you do write “a quality threshold beyond which the application is
> considered useless.”)  - Google gives me this pretty funny list of useless
> applications:
> https://universe.byu.edu/2015/03/19/15-of-the-most-useless-apps-available/
>   … but they may all be *usable*  (the Harmonica app is my favorite in this
> list, btw - I mean, who comes up with something like this???)
>

"Useless" is unusable. Got it! I agree unusable is a better word - will
make the proposed changes.


> Also, while it’s indeed enough to consider delay and throughput
> measurements, not packet loss, when packet loss is defined as infinite, I
> think the relationship to packet loss should again be stated just with a
> sentence in section 5. It’s possible for a reader to skip (or only skim)
> the rest of the document and quickly want to know how to calculate the
> measure - and then, when reading this section, the question “but what about
> packet loss?” might be raised.
>
> Good point.


> abstract: s/layed out/laid out
> intro par 2: I think it would be better to remove “the” before "most of
> the criteria”
> intro par 3: “makes” should be “make” in "Applications and the underlying
> networking protocols makes separate optimizations based on their perceived
> network quality over time”
> Intro par 4: suggest: s/We propose representing network quality as minimum
> required throughput and set of latency and loss percentiles./We propose
> representing network quality as a minimum required throughput and a set of
> latency and loss percentiles.
> "We propose a formula for a distance measure between perfect and useless
> quality.” - I just find the word combination of “useless quality” odd; I
> think the word “quality” could just be removed here.
> section 4: "Until now, network requirements and measurements are what is
> already standardized in BBF TR-452 (aka QED) framework [TR-452.1].”
> missing “the” before “BBF”.
>
> There are a few stylistic elements in this document that strike me as
> somewhat inappropriate for the spec that it’s meant to be. E.g. the use of
> irony in:
> "On the other hand, making the framework overly complex and difficult to
> adhere to using real-world equipment and applications is the best way to
> ensure that this framework goes unused.”
> and this: "A method for going from Network Requirements and Network
> Measurements to probabilities of outcomes, or Quality of Outcomes if you
> will.”
> section 5: “The QoO.” is not a sentence. suggest to connect it with the
> preceding sentence, e.g. with a dash.
> "For 3, we will now specify the calculation between to translate these
> distances to a 0 to 100 measure. “  - I think “between” should be removed
> here.
> Next par: “Essentially, where…” is not a complete sentence (misses a verb).
> Next sentence: “Where: …” - this is also not really a sentence.
> The next sentence is also broken “is the points” is wrong, and I’m not
> sure if the sentence is complete or what’s wrong here, somehow. Same for
> the next. This whole paragraph is just grammatically broken somehow.
>
> Last sentence of sec. 5.1. misses a full stop.
>
> The end of section 6 is a bit odd:
> The reason for making the QoO metric for a session or time-period is to
> make it understandable for an end-user, an end-user should not have to
> relate to the time period the metric is for.
> The first use of “time-period” makes this seems self-contradictory.
> Perhaps just say “for a session”, and maybe add something like “ that is
> considered typical for the application” ?
>
> sec 7 par 2 s/precition/precision
>

Thanks you very much for the detailed feedback! I've made an issue to
address these comments, here: https://github.com/domoslabs/QoOID/issues/10

Cheers,
Bjørn


>
> _______________________________________________
> ippm mailing list
> ippm@ietf.org
> https://www.ietf.org/mailman/listinfo/ippm
>


-- 
Bjørn Ivar Teigen, Ph.D.
Head of Research
+47 47335952 | bjorn@domos.ai | www.domos.ai
[image: https://www.understandinglatency.com/]
<https://www.understandinglatency.com/>