Re: [ippm] Adoption call for draft-olden-ippm-qoo-02
Bjørn Ivar Teigen <bjorn@domos.no> Fri, 02 February 2024 14:53 UTC
Return-Path: <bjorn@domos.no>
X-Original-To: ippm@ietfa.amsl.com
Delivered-To: ippm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 04606C1519A8 for <ippm@ietfa.amsl.com>; Fri, 2 Feb 2024 06:53:07 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.896
X-Spam-Level:
X-Spam-Status: No, score=-0.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_REMOTE_IMAGE=0.01, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001, URI_DOTEDU=1] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=domos-no.20230601.gappssmtp.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2oTZ8F53CS7O for <ippm@ietfa.amsl.com>; Fri, 2 Feb 2024 06:53:02 -0800 (PST)
Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com [IPv6:2a00:1450:4864:20::12b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C8F4DC151524 for <ippm@ietf.org>; Fri, 2 Feb 2024 06:53:02 -0800 (PST)
Received: by mail-lf1-x12b.google.com with SMTP id 2adb3069b0e04-511206d1c89so2982661e87.1 for <ippm@ietf.org>; Fri, 02 Feb 2024 06:53:02 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=domos-no.20230601.gappssmtp.com; s=20230601; t=1706885580; x=1707490380; darn=ietf.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=q56ErDB5t85MAyTKsuO33y6ADqPqjcdyDoJfivhiP2M=; b=Af16K+bjdno2F6DAgAVw9T2aZJij5S5WuuzFiU+QSK9lVgQ3EweoM4SCqyZLivmUwN 9egXozJDfMk3wl81vBvH8LrhGiHHLy+igrqsfECanVlIDYlU+bdlG3YUFCIYp1nqz/dU HP8BLtdF3zqhjt/86vSWIRmTdv2TfqefWjjWvA641tKSBibxKD/UKVtWL+gpAuZGajxa bjZu+Qr3jYxH2sHdhGB/pBBqU6YaU1PBkvZbMITIZe4lk2uXHsfQ6DDij2j1FtZ4YEyS LVxiWmTTOYln+Rs+duL0kJ10puA8IR9dgHYJo1B/EnZZc/+SY/acEIBfZntnTEycTnsn JzDw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706885580; x=1707490380; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=q56ErDB5t85MAyTKsuO33y6ADqPqjcdyDoJfivhiP2M=; b=d2Mb7G6A6CRTUu00p0Y5Qw0e9AY4qpxMCXBLx6Pp1ejcsgQ2nXrF8cq9p3ivcy0Row juwylloYfxBP/1+c4JEb1+jxFJvkLuk5TYLF6Elhz6lGj36Pdt8yTQuAHymfcZatmbgA ReNWG+vOmx87S4q4BnT7MwZzfj4SmNmTjktNw/IFj4VHX9DO4OAJCNX1wQaE+4M73TwM CuV4aDBps25sSWU/k+bnI520cRxjKtmPpsClICjJRfjU30XaGXvur2B/FKcrXu8ygIiA F3dsNBJZ0CiPyonFT2X5XBLHWHZLUgXMwh7aRlipFSeNcLejJ7rAbi1LhW0G7f2sC9HX njKw==
X-Gm-Message-State: AOJu0YyPzpR0fJaJt6YTU8/AewxfKgF9N+Pj4gemDnyPvpFQ6t6EvAro H1BByZSg6exs3pYEHDOKOvjFWV03bdU0j1+0qlEJYxx1FMattRmv4rxxTdAswAcV5OtaL5zr52Q VgpBGIY0JG564JoENk+zoZCFhnBSFclVoJGN3gXyZOQhW+kDBQLubdg==
X-Google-Smtp-Source: AGHT+IHZO7m/Yl6szwqDM50K/YUVxgHWq9XzTpA/vchKgZYnIaEu2M4n8E6RxFVjtWWdZkquaqE1P10BC8BsDDPIDYE=
X-Received: by 2002:a05:6512:2828:b0:511:31b4:ac16 with SMTP id cf40-20020a056512282800b0051131b4ac16mr1797986lfb.47.1706885579482; Fri, 02 Feb 2024 06:52:59 -0800 (PST)
MIME-Version: 1.0
References: <B92594EC-B6FE-4160-94E3-AA65EE65EDBC@ifi.uio.no>
In-Reply-To: <B92594EC-B6FE-4160-94E3-AA65EE65EDBC@ifi.uio.no>
From: Bjørn Ivar Teigen <bjorn@domos.no>
Date: Fri, 02 Feb 2024 15:52:48 +0100
Message-ID: <CAKf5G6LL4h=Pg6Ysnw0N0oPGr-Vq+XO-FRkP_ewHcLpME=_eSw@mail.gmail.com>
To: Michael Welzl <michawe@ifi.uio.no>
Cc: ippm@ietf.org
Content-Type: multipart/alternative; boundary="000000000000a33cff06106744c5"
Archived-At: <https://mailarchive.ietf.org/arch/msg/ippm/E1Qx5L8BzZd-Ri3BOVIhYGm4W9k>
Subject: Re: [ippm] Adoption call for draft-olden-ippm-qoo-02
X-BeenThere: ippm@ietf.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: IETF IP Performance Metrics Working Group <ippm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ippm>, <mailto:ippm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ippm/>
List-Post: <mailto:ippm@ietf.org>
List-Help: <mailto:ippm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ippm>, <mailto:ippm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 02 Feb 2024 14:53:07 -0000
Hi Michael, Thanks for the feedback. Much appreciated! On Wed, 31 Jan 2024 at 11:39, Michael Welzl <michawe@ifi.uio.no> wrote: > Dear all, > > I have just reviewed both draft-olden-ippm-qoo-02 and > draft-teigen-ippm-app-quality-metric-reqs-02. I only wanted to read the qoo > document but I also found that I lack key information unless I read both - > and so I agree with the stated suggestion that it would be good to handle > them in tandem. > > My general conclusion is that this is *very* useful - I applaud the > approach of trying to be pragmatic and simple towards users here, and > clarifying the different needs of app developers, users and net admins. > Surely a latency distribution is great input for “quality" - this all seems > "just right" to me. > Of course, there will be some open questions… I got some, as I read the > qoo-document (e.g., about the granularity of how packet loss is handled > here, and the linearity of “better” or “worse”), and I found them all > discussed in section 7 - so this section is much appreciated. > > All in all, I definitely support WG adoption. > > Please find detailed comments on both drafts below - mostly just nits. > > Cheers, > Michael > > > ======== > > > draft-teigen-ippm-app-quality-metric-reqs-02 > > Section 3 heading: stray “,” at the end > "Compositionality is essential for fault detection and accountability.” > I think this should be: Composability > Section 4.8: "It, therefore, measures something very close to the > user-perceived application performance of HTTP-based applications.” > I think that it’s quite possible to implement a relatively large one-way > transfer over HTTP; perhaps instead you mean “of applications that involve > many handshakes, e.g. typical HTTP-based web surfing.” > Next sentence: I believe that "well-suited to detecting bufferbloat“ > should either be “for detecting” or “to detect”. > > Thanks. I agree with your comments and have made an issue to address them, here: https://github.com/domoslabs/AppQualityMetricID/issues/4 > > draft-olden-ippm-qoo-02 > > As a general editorial comment, I think the word “useless” isn’t very > suitable here. “Unusuable” would be more precise. E.g., you talk about > “useless quality” at some point, which I think doesn’t exist - “poor > quality” does. Similarly, an application doesn’t become “useless” because > the network quality is poor - it becomes “unusable”. > (e.g., you do write “a quality threshold beyond which the application is > considered useless.”) - Google gives me this pretty funny list of useless > applications: > https://universe.byu.edu/2015/03/19/15-of-the-most-useless-apps-available/ > … but they may all be *usable* (the Harmonica app is my favorite in this > list, btw - I mean, who comes up with something like this???) > "Useless" is unusable. Got it! I agree unusable is a better word - will make the proposed changes. > Also, while it’s indeed enough to consider delay and throughput > measurements, not packet loss, when packet loss is defined as infinite, I > think the relationship to packet loss should again be stated just with a > sentence in section 5. It’s possible for a reader to skip (or only skim) > the rest of the document and quickly want to know how to calculate the > measure - and then, when reading this section, the question “but what about > packet loss?” might be raised. > > Good point. > abstract: s/layed out/laid out > intro par 2: I think it would be better to remove “the” before "most of > the criteria” > intro par 3: “makes” should be “make” in "Applications and the underlying > networking protocols makes separate optimizations based on their perceived > network quality over time” > Intro par 4: suggest: s/We propose representing network quality as minimum > required throughput and set of latency and loss percentiles./We propose > representing network quality as a minimum required throughput and a set of > latency and loss percentiles. > "We propose a formula for a distance measure between perfect and useless > quality.” - I just find the word combination of “useless quality” odd; I > think the word “quality” could just be removed here. > section 4: "Until now, network requirements and measurements are what is > already standardized in BBF TR-452 (aka QED) framework [TR-452.1].” > missing “the” before “BBF”. > > There are a few stylistic elements in this document that strike me as > somewhat inappropriate for the spec that it’s meant to be. E.g. the use of > irony in: > "On the other hand, making the framework overly complex and difficult to > adhere to using real-world equipment and applications is the best way to > ensure that this framework goes unused.” > and this: "A method for going from Network Requirements and Network > Measurements to probabilities of outcomes, or Quality of Outcomes if you > will.” > section 5: “The QoO.” is not a sentence. suggest to connect it with the > preceding sentence, e.g. with a dash. > "For 3, we will now specify the calculation between to translate these > distances to a 0 to 100 measure. “ - I think “between” should be removed > here. > Next par: “Essentially, where…” is not a complete sentence (misses a verb). > Next sentence: “Where: …” - this is also not really a sentence. > The next sentence is also broken “is the points” is wrong, and I’m not > sure if the sentence is complete or what’s wrong here, somehow. Same for > the next. This whole paragraph is just grammatically broken somehow. > > Last sentence of sec. 5.1. misses a full stop. > > The end of section 6 is a bit odd: > The reason for making the QoO metric for a session or time-period is to > make it understandable for an end-user, an end-user should not have to > relate to the time period the metric is for. > The first use of “time-period” makes this seems self-contradictory. > Perhaps just say “for a session”, and maybe add something like “ that is > considered typical for the application” ? > > sec 7 par 2 s/precition/precision > Thanks you very much for the detailed feedback! I've made an issue to address these comments, here: https://github.com/domoslabs/QoOID/issues/10 Cheers, Bjørn > > _______________________________________________ > ippm mailing list > ippm@ietf.org > https://www.ietf.org/mailman/listinfo/ippm > -- Bjørn Ivar Teigen, Ph.D. Head of Research +47 47335952 | bjorn@domos.ai | www.domos.ai [image: https://www.understandinglatency.com/] <https://www.understandinglatency.com/>
- [ippm] Adoption call for draft-olden-ippm-qoo-02 Tommy Pauly
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Kevin Smith, Vodafone
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Mehmet Sukru Kuran
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Bjørn Ivar Teigen
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Bjørn Ivar Teigen
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Dave Taht
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Michael Welzl
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Bjørn Ivar Teigen
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Greg Mirsky
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Bjørn Ivar Teigen
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Greg Mirsky
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Bjørn Ivar Teigen
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Bjørn Ivar Teigen
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Tommy Pauly
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Bjørn Ivar Teigen
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… LUIS MIGUEL CONTRERAS MURILLO
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Bjørn Ivar Teigen
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Bjørn Ivar Teigen
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Will Hawkins
- Re: [ippm] Adoption call for draft-olden-ippm-qoo… Bjørn Ivar Teigen