[ippm] QoO simulation results

Bjørn Ivar Teigen <bjorn@domos.no> Thu, 23 January 2025 13:30 UTC

Return-Path: <bjorn@domos.no>
X-Original-To: ippm@ietfa.amsl.com
Delivered-To: ippm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3606FC15198B for <ippm@ietfa.amsl.com>; Thu, 23 Jan 2025 05:30:08 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.904
X-Spam-Level:
X-Spam-Status: No, score=-1.904 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_BLOCKED=0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=domos-no.20230601.gappssmtp.com
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3baNomyVPMRU for <ippm@ietfa.amsl.com>; Thu, 23 Jan 2025 05:30:03 -0800 (PST)
Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com [IPv6:2a00:1450:4864:20::134]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D8360C15155F for <ippm@ietf.org>; Thu, 23 Jan 2025 05:30:02 -0800 (PST)
Received: by mail-lf1-x134.google.com with SMTP id 2adb3069b0e04-540215984f0so995387e87.1 for <ippm@ietf.org>; Thu, 23 Jan 2025 05:30:02 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=domos-no.20230601.gappssmtp.com; s=20230601; t=1737639001; x=1738243801; darn=ietf.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=doBp6uHsgsg10gMFKCuq8dZJvr4jQEXzdF/i7g7Ybdc=; b=genUm4mMLqjD7ekIAbaVqdNP7BsggVe3PunW8h3l66ex3IanV4qk1uiHIRmN/xMiqv CsAx5LYUYgwkr9+KnWt6hEc+UxRCyWxO4Nt98lwUSQINTVY4mwrIVzRcmiQTj/7qCWUl xrWFUip0ojQn9fnON+dtg3hjkUr9nDLGQ894I1A2sc5eqeEo/IyQZl5474XpEYTCzQC6 kiy5By+D510rz2MrKsFwltpiHBVST/8GMl6Qay02UxaxrE171Q4a9RVPinf1dEx3JzxI 8Ucd8VN67NB6pdwRVriAmKlCvTQOyBKmMPZxtezv9jzupw7CAtRus5A9LWspO0xxH5UZ mGjw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737639001; x=1738243801; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=doBp6uHsgsg10gMFKCuq8dZJvr4jQEXzdF/i7g7Ybdc=; b=hFa78dQMRdS3C8Nt8X9Yz/x6Nlc0K12FWIQ0oxjAfeTP1206M/ThybH0xJ+LogV7lX P8OZaNoD9OjiWvYdLObi52zY6Gawi3zSLuBGiwWZo2fo/RKQi45spONMFs1Ob/GB0meD 8/r+Fe0d4D7AW3NFiU4TGlTKhMxQTmK8AGudm6J0dA0l1p08c+yPcpkCJ+q7wf1GZEtY eaQOBoZsMu06/k6jXdz7VP1T0PHI2KyjuuER+3/EckKiVJ0pAh0q8cxN/juLZmF4Urkx TcG38dPvcWflY82ZIPeZHrWrUYvNYXvuDr753gxuAtRusi3I+WwN1HK/QK4SM5ZaDfNF oiGQ==
X-Gm-Message-State: AOJu0YxIQ5Eonft7a/4yzAtEeS9SoXR6gDWMGGs8hAvqcnprTSg0IO3m +zAgitZ7f03jKR5ioqivNAG6Z0W5mDw0XDD1DUOCZoBYj8CjbJM8O45RfvQPvdMOxbAZFYCgdIV T5UDTuAg01BvFM2rif+cJNVI1ZnVkZTPyPkX4lgueXwc2qsdOoZI=
X-Gm-Gg: ASbGncuFMvolufNidGpzqaPLpNM5s08r4gthRKXei37oNwAJOIa02UHSKqBewAL3wcD QACBTpk5D9V1C7GXGiBFO/i7TqtQcPf5haM8m6XcvWauIccfhA8YOngr/iAv6BnTcqecLwOojJ1 2KlqnKtMSoslMluRuhV+8=
X-Google-Smtp-Source: AGHT+IHJ+e+/0qir7Td72hjvyP+q2N+S2ujDZLMscHQ7W7b/egOpY9PpNjIGOJjGU+iGWrJumEOqq99/zHot7ae5+Is=
X-Received: by 2002:ac2:5e9d:0:b0:542:2166:44cb with SMTP id 2adb3069b0e04-5439c282b25mr8188509e87.35.1737639000080; Thu, 23 Jan 2025 05:30:00 -0800 (PST)
MIME-Version: 1.0
From: Bjørn Ivar Teigen <bjorn@domos.no>
Date: Thu, 23 Jan 2025 14:29:49 +0100
X-Gm-Features: AWEUYZk--fHLNDEiUf0CIQnOOtnrJONwMEcOy93rTFCc2vtJferWkfx5g_Jatpg
Message-ID: <CAKf5G6JA1aAkteuRETpKkvoHnQ2wq1qiNQfXpbY0M1PTuxFw5w@mail.gmail.com>
To: "IETF IPPM WG (ippm@ietf.org)" <ippm@ietf.org>
Content-Type: multipart/alternative; boundary="00000000000059187a062c5f9ba4"
Message-ID-Hash: I4BEJHQWWXYAETREIF46G7OMWI7U66NU
X-Message-ID-Hash: I4BEJHQWWXYAETREIF46G7OMWI7U66NU
X-MailFrom: bjorn@domos.no
X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-ippm.ietf.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header
X-Mailman-Version: 3.3.9rc6
Precedence: list
Subject: [ippm] QoO simulation results
List-Id: IETF IP Performance Metrics Working Group <ippm.ietf.org>
Archived-At: <https://mailarchive.ietf.org/arch/msg/ippm/lX9Db_r1kudaZ5-luejdrs5_LFs>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ippm>
List-Help: <mailto:ippm-request@ietf.org?subject=help>
List-Owner: <mailto:ippm-owner@ietf.org>
List-Post: <mailto:ippm@ietf.org>
List-Subscribe: <mailto:ippm-join@ietf.org>
List-Unsubscribe: <mailto:ippm-leave@ietf.org>

Dear IETF IPPM Working Group,

I am writing to share our latest simulation study, "Evaluating Quality of
Outcome (QoO) in Challenging Network Conditions,
<https://assets.ycodeapp.com/assets/app24919/Documents/published/evaluating%20qoo%20in%20challenging%20network%20conditions-5gtulrheli.pdf>"
which provides new insights into the QoO metric as detailed in the QoO draft
<https://datatracker.ietf.org/doc/draft-ietf-ippm-qoo/>.

Our study rigorously tests the robustness of QoO in a variety of
challenging simulated network scenarios. We focused on QoO's sensitivity to
sampling frequency, measurement accuracy, application-specific
requirements, and sampling duration.

Key results from our research include:

   1. Demonstrating the performance of QoO in varying sampling scenarios,
   measuring sensitivity to sampling rate and measurement duration.
   2. Quantifying the influence of measurement accuracy on QoO scores.
   3. Clarifying the impact of network requirement specifications on QoO
   scores.

The complete simulation methods, detailed scenario results, and code for
reproducing our findings are accessible here: QoO Simulation Code
<https://github.com/domoslabs/qoosim>.

We look forward to your feedback and are eager to engage in further
discussions to refine and finalize the QoO metric.

Thank you for your attention and consideration of our work. I am
enthusiastic about the potential improvements our findings may contribute
to the IPPM community.

The full report of the study can be found here
<https://assets.ycodeapp.com/assets/app24919/Documents/published/evaluating%20qoo%20in%20challenging%20network%20conditions-5gtulrheli.pdf>.
Code here <https://github.com/domoslabs/qoosim>. A new version of the draft
<https://datatracker.ietf.org/doc/draft-ietf-ippm-qoo/> referencing these
results will be published soon.

Warm regards,
-- 
Bjørn Ivar Teigen, Ph.D.
Head of Research
+47 47335952 | bjorn@domos.ai | www.domos.ai