Re: [ippm] Adoption call for draft-cpaasch-ippm-responsiveness

"MORTON JR., AL" <acmorton@att.com> Sun, 20 February 2022 17:38 UTC

Return-Path: <acmorton@att.com>
X-Original-To: ippm@ietfa.amsl.com
Delivered-To: ippm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 9BC5E3A07E2 for <ippm@ietfa.amsl.com>; Sun, 20 Feb 2022 09:38:07 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.896
X-Spam-Level:
X-Spam-Status: No, score=-1.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=att.onmicrosoft.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VsQ5gTQR5qmt for <ippm@ietfa.amsl.com>; Sun, 20 Feb 2022 09:38:02 -0800 (PST)
Received: from mx0b-00191d01.pphosted.com (mx0b-00191d01.pphosted.com [67.231.157.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id ECA883A0125 for <ippm@ietf.org>; Sun, 20 Feb 2022 09:38:01 -0800 (PST)
Received: from pps.filterd (m0288869.ppops.net [127.0.0.1]) by m0288869.ppops.net-00191d01. (8.17.1.5/8.17.1.5) with ESMTP id 21KCLrrW004296; Sun, 20 Feb 2022 12:37:55 -0500
Received: from alpi154.enaf.aldc.att.com (sbcsmtp6.sbc.com [144.160.229.23]) by m0288869.ppops.net-00191d01. (PPS) with ESMTPS id 3eb1f4w907-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 20 Feb 2022 12:37:54 -0500
Received: from enaf.aldc.att.com (localhost [127.0.0.1]) by alpi154.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id 21KHbsTm009774; Sun, 20 Feb 2022 12:37:54 -0500
Received: from zlp30487.vci.att.com (zlp30487.vci.att.com [135.47.91.176]) by alpi154.enaf.aldc.att.com (8.14.5/8.14.5) with ESMTP id 21KHbqCX009762 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 20 Feb 2022 12:37:52 -0500
Received: from zlp30487.vci.att.com (zlp30487.vci.att.com [127.0.0.1]) by zlp30487.vci.att.com (Service) with ESMTP id 57F6E400579E; Sun, 20 Feb 2022 17:37:52 +0000 (GMT)
Received: from GAALPA1MSGED2DC.ITServices.sbc.com (unknown [135.50.89.140]) by zlp30487.vci.att.com (Service) with ESMTP id CC1E14005792; Sun, 20 Feb 2022 17:37:51 +0000 (GMT)
Received: from GAALPA1MSGEX1BB.ITServices.sbc.com (135.50.89.103) by GAALPA1MSGED2DC.ITServices.sbc.com (135.50.89.140) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18; Sun, 20 Feb 2022 12:37:07 -0500
Received: from GAALPA1MSGETA02.tmg.ad.att.com (144.160.249.124) by GAALPA1MSGEX1BB.ITServices.sbc.com (135.50.89.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.18 via Frontend Transport; Sun, 20 Feb 2022 12:37:07 -0500
Received: from NAM02-SN1-obe.outbound.protection.outlook.com (104.47.57.45) by edgeal2.exch.att.com (144.160.249.124) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2375.18; Sun, 20 Feb 2022 12:37:07 -0500
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dGUwd+BBsKJ2OpB8ZOC9DfgQUPD6IghOWNwb5urTaK4NqulTO6vyhwfz0knL7fAjXAD+9xVvUutyyzsiH8S/cqOA4VHzQuiI5J/1luB3UjWOvgZToWiRulwqfQ6wAvEUQKtLpOurFvO6j14p49p//rHlLf3aBUEImVzgRpO3DS75d292iiGuZmtdSbPWkx4vCx3kk/Y9rM8ccMxzoTTsP8BBC5fhTZgHtsdwZaWpCJ2JHje4dZvpB15Ju07j9k42YueVgpPj6ZeiI0bD5JOT/tGSH6y+PVcu9iea8myC/fI9V6CSPa82e7HkHiESQQa9Y4nW+9zbHUNW/HZckMg6zQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LWMMKZ4jbXTxEEfZGVMhZvoDD7eRAy7+Csngu/hID84=; b=Txfk36TG16sU400sXzRRZ4dFHtMgCfe53EI8obFH6TPbV/VlJ0UfC55Dwg9vWiFkD8zV60sYLg8+4jtpKtciqVBUa538mJlVSl752osnCb/JBff3wPU4bUSvOq5MADR/bslpigBd5+IlBEOtaJjNC4qh8Ma6zi9WeaSKBoWAmSWR1hbTlcA0xS3VkjuO/ho3F8Dvp7wh+LKrFy+QbtFDj3k16jaR3Y2irPUu2e3fGqUNevMVMMO44V9b+ALnxSKoBK/8qRx/DoAfapk8FzBEbI52DNjYXcD4uI9PuDYy66JhNf8ztF5Aa+amUgvkWPSMLNeXp7TYQizCbECZitjuXQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=att.com; dmarc=pass action=none header.from=att.com; dkim=pass header.d=att.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=att.onmicrosoft.com; s=selector2-att-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LWMMKZ4jbXTxEEfZGVMhZvoDD7eRAy7+Csngu/hID84=; b=vUsBuPt9ohtctglgTbRSIRbwP2G+L9oI5rZtqYpLYJJo+Tr53xnTw/otPSvuyXcn02/uZAX83TFMk7TEG/Z8ImVkeA87/L5V2TxVLAeFwRwFvaTl80ynV5/iMDFKtlxdcoDHJpNsV1veIpdUTr56zONWuYj7pNFROUquhosfKKI=
Received: from CH0PR02MB7980.namprd02.prod.outlook.com (2603:10b6:610:105::17) by BYAPR02MB5127.namprd02.prod.outlook.com (2603:10b6:a03:6c::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4995.24; Sun, 20 Feb 2022 17:37:04 +0000
Received: from CH0PR02MB7980.namprd02.prod.outlook.com ([fe80::cd34:2582:613:215b]) by CH0PR02MB7980.namprd02.prod.outlook.com ([fe80::cd34:2582:613:215b%4]) with mapi id 15.20.4995.026; Sun, 20 Feb 2022 17:37:04 +0000
From: "MORTON JR., AL" <acmorton@att.com>
To: Christoph Paasch <cpaasch@apple.com>
CC: Dave Taht <dave.taht@gmail.com>, Marcus Ihlar <marcus.ihlar@ericsson.com>, IETF IPPM WG <ippm@ietf.org>
Thread-Topic: [ippm] Adoption call for draft-cpaasch-ippm-responsiveness
Thread-Index: AdfquNUJKsdYHHTaTOi3VLkEWROGkgNF/40AAqUhp4AAOvLbAAADkD8ABnQ5BIABZlmoAAABuRMAAB7QLrAARH7+AACHceEA
Date: Sun, 20 Feb 2022 17:37:04 +0000
Message-ID: <CH0PR02MB798026C5430398B37E91EA95D3399@CH0PR02MB7980.namprd02.prod.outlook.com>
References: <AM0PR07MB4131542BCD0A6DE3F82F1E19E26D9@AM0PR07MB4131.eurprd07.prod.outlook.com> <CA+RyBmU_j9-vR+BnjvhKCDuaWYPZ_Ym96yUJPX0LhGihfsp1ng@mail.gmail.com> <3DC3F6B6-229E-46C6-BD84-2A6A7FE6DD48@apple.com> <CA+RyBmV_+yysquiZ=2PwB=oaqeJmfKV39c3=GE9sxWkb4qTM=Q@mail.gmail.com> <9340CFDA-079C-4490-A01C-EB863D365F8F@apple.com> <CA+RyBmW=xMmj70GymYwbsG0XcDNS64UNSWxGdwy10+KMjuVWww@mail.gmail.com> <A39D7366-201F-4B96-9667-C53582A79E17@apple.com> <CAA93jw45K4VscaMQPnF4wQD_D-nc=gJRi9X5wMTEeXws5KX_xA@mail.gmail.com> <CH0PR02MB798067D922BD755D52A54986D3359@CH0PR02MB7980.namprd02.prod.outlook.com> <74A71CF0-16BA-4403-B2DD-D2DAA264E2CA@apple.com>
In-Reply-To: <74A71CF0-16BA-4403-B2DD-D2DAA264E2CA@apple.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: e306c614-ab42-4a22-a016-08d9f4979c28
x-ms-traffictypediagnostic: BYAPR02MB5127:EE_
x-microsoft-antispam-prvs: <BYAPR02MB51271FB60C0109C0452A7754D3399@BYAPR02MB5127.namprd02.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: VOSX7U/i0KIpiFL/eINHhllcLeRH9B0nvp5SAyxQVgpcqMBZ5HQFw8FIUYFTMBuMK1RrrwByUIliuqu9wLbNdRbjSh2goY3DBnvXlqhyEEw5inll4aLdWcZmSucKVK8oofB1btV2xXaM+XII7ubpGD3ncypq174TuPpgvoyUQM4txr+XEUHIHuyjsIjuZzaeNUAJnaAh1PeKhlYLpliQR6grWSRYBSjbIxD7FRRGAr+3fLub+w+MR4b2rIfjPezwOPpeBXlvTTu8qXKV/O1v47OsJendHFlzYkHgPhLrk/5Qlc+rhuUmOgpufQjqZzvTvgnU7O1YidSMfNITjicehnzgADSTlCsKyQSU6IC1zz+ZxlavLScypX9ndy35Y2sUyXPGeaoMQT4KGpvhvVl+JVDpkTqPqthZas35MEYTJnTik/5tyLpVR7eBtnh2taJfsnS6afyVSWnB4tpuWmnLHvPsOg6Z/JXQmHGdWAP6pR3BTWr9cRbRCSBO8XYZBVfQfh8DTnUjnDjB0dRor1fWDL6obR+s8ukVdbpncoY99rLV96L3S6P6zcigXB/ZHcKQOzPgysz/9ucizzpawjcwEULaIAnUXApKpXFrkETZ9kdSw3vArdf1fJlIRholZJi4pNoQvKUpxQpZm2E/PY3RfgvNzzeKgWzfpC2YSrjPnX4Xq146ktjwV8yiXlOR5hw65rP4b29EFVfmYGSFD5+glA7ETWX71zD8rTId82AMnSYOcQC/90Uv5koTmLl/NketukA/3LwhVGC5YJakJB4YP6cHfow5lTc2NHN0MWCkg3mEGTzXbksaFwdfC1bbx9yA
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH0PR02MB7980.namprd02.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(38100700002)(122000001)(86362001)(38070700005)(66446008)(82960400001)(166002)(66946007)(4326008)(66556008)(66476007)(64756008)(8676002)(76116006)(6916009)(54906003)(55016003)(316002)(2906002)(8936002)(52536014)(9326002)(30864003)(66574015)(186003)(82202003)(26005)(83380400001)(508600001)(966005)(5660300002)(71200400001)(53546011)(9686003)(7696005)(6506007)(33656002)(559001)(579004); DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 3UmcjECD74kldkV1b+z+OQMhS7vlkKkhh0LTIi2hOv8YvEB3WJtOt12gr7PzYFpj3H7udhqLT3uVmzrzYuT6kJdhOJU79cQF+YHM9wWXKwjks6o8RzJgClq59JGB74L0h6AqjSORqFUFKyti1SuiwVLMZ4BbWaXHiNWBnBH+mw/gkdtbuZm23b8HN3gyaHVI4tREqLF3ktUru8JhMGTeDtoor3C3WssukIXtSVQVV5or1Hwmn8vGyjbZbn7U3mhafUfsVq2H2rjHe0lHZfloD29VlhJHY9EhCXQvamIuL6tMJadGszESxu1mmlPUjuk0A7BnJVZxUBaVkLYAT/kdGc9Xm1ui8tePgvmCJ6nWBRolOhWVeXVWnXvpuPrpPKcx6k97j/kih/R8oOC4YoOjjmx958fanHfUuVxRv+PzR0h8Yz1HPLjnXdl9/JBNyloM7wsdu2PCcrCcMh73dBeZ1Oi7nrUu1v/XTwLqKDUzwNTNC28eS9yMDYx9TQjkwfLXgwQw5iUIxFmGv6ZF+53GdEsPpu8hL7Sqr5j/oUPL2Ol16egdVVXpnoonPTRz05UJjfedXWRp2/+nhYQEff6Ja2rVzdIr/cg9liFAr4rlZ5gacCxAtvn7HbhdfRkrPl1hGRiEuGJtdZ9ckgLFA4qJZ/Wr9K7onOsGAPrYwbe1nFdoGxi4qy18o9EpvO1kcnzaPApIz1MXwwJKe6orx5/RFb9I7KBKjKdkR5McMEN8P4OE45fK9jiVEvYJB9VKpWF8ry4KP4yiq/wq5sEYS/1o7VdOBKDOvX9N5kQmX4tUZD3CyOJTJWYEJTs9nakGWoJ8HTU3yLTnarbcHWnycwgn3Q8bVsMEJ1z3cHltnG4jLoD/6hjdM39LDQfOu8Lz8lq1jUNeFKNre7I6Cp5ULXEy6FE2ZnDYAlHjScHbqIdI/dKVha+xTHtI58ZPk40RedkBrn6gkXtJxEajTKe5LAE7z9k0Jaq1gDcCpGXdFYCf3i4pIhI80gdk1DgjgmZBnltWB4iBv7XyuvWmFr8zUpU37OC2X2NFLP7fLzNFh3pmccXduVgEgKS1ronxdKjsr6ovaYhYlMjykhlJgfLURh2MIEDEEi/BYOTEytKIDB7iMuuQXuAEAj4Dmfpe526sbfJCad/q4QWak+gAkjIemAOh0PcQ23TvBZ7f8ovW+RABJxWx+qWmBwF78znEOkYD8ANw2kLQ/Yb+39B4hwNkIAFy25dKclVP7yF2W3KSc4evS+PwPlePnWxEzEFr2QQUMEe9rya3lbh/IUa/bA/rkaf4C6hGk/BKeOnl33YUJcWK1y2/AtxTXMH5RIxy9rtqHg48UI2RLMzFYZAN0zKKxdFp44UnX0mH6gLOvXtYYePTPv/42vBW9baFZtHeNejFSggNgA/JnQypScSi7aQoNhyxv0vB2S3R3nM9imd07rvy5G1VstmUGuvmLc21l3Xi1sLKoyjlkPfM6kxwdpu/KV8+nvXqUjkv5bRN6J+E+QY0PL5bIMKcKkw5VRygyRAoeAsYaU/qKnv8Y5sU5ZY/jQhbCg==
Content-Type: multipart/alternative; boundary="_000_CH0PR02MB798026C5430398B37E91EA95D3399CH0PR02MB7980namp_"
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: CH0PR02MB7980.namprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e306c614-ab42-4a22-a016-08d9f4979c28
X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Feb 2022 17:37:04.6314 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: e741d71c-c6b6-47b0-803c-0f3b32b07556
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: jVlJ7BEm8FvR6fNdcdflLvVjIFcgmZFR3BltSmXVkO6jrj87LMrE/Py5WQBco/Gk
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR02MB5127
X-OriginatorOrg: att.com
X-TM-SNTS-SMTP: 6558DE4DABB415D424E91C8085C26984C955B03386C28C046ACEBFD18502022B2
X-Proofpoint-ORIG-GUID: 6RscVZSPLJ0CWTRgNYWSmIAACIH4SDEy
X-Proofpoint-GUID: 6RscVZSPLJ0CWTRgNYWSmIAACIH4SDEy
X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.816,Hydra:6.0.425,FMLib:17.11.62.513 definitions=2022-02-20_07,2022-02-18_01,2021-12-02_01
X-Proofpoint-Spam-Details: rule=outbound_policy_notspam policy=outbound_policy score=0 spamscore=0 priorityscore=1501 malwarescore=0 phishscore=0 suspectscore=0 mlxlogscore=999 bulkscore=0 lowpriorityscore=0 impostorscore=0 mlxscore=0 clxscore=1011 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000 definitions=main-2202200114
Archived-At: <https://mailarchive.ietf.org/arch/msg/ippm/BlqaIFZv2rTH_36cQUHoCvr1wHI>
Subject: Re: [ippm] Adoption call for draft-cpaasch-ippm-responsiveness
X-BeenThere: ippm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF IP Performance Metrics Working Group <ippm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ippm>, <mailto:ippm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ippm/>
List-Post: <mailto:ippm@ietf.org>
List-Help: <mailto:ippm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ippm>, <mailto:ippm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 20 Feb 2022 17:38:08 -0000

Hi Christoph,
Thanks for continuing this discussion. Allow me to reply and add my view below,
Al

From: Christoph Paasch <cpaasch@apple.com>
Sent: Thursday, February 17, 2022 7:24 PM
To: MORTON JR., AL <acmorton@att.com>
Cc: Dave Taht <dave.taht@gmail.com>; Marcus Ihlar <marcus.ihlar@ericsson.com>; IETF IPPM WG <ippm@ietf.org>
Subject: Re: [ippm] Adoption call for draft-cpaasch-ippm-responsiveness

Hello Al,


On Feb 16, 2022, at 8:07 AM, MORTON JR., AL <acmorton@att.com<mailto:acmorton@att.com>> wrote:

Hi Christoph,

you wrote:

I think what would be helpful would be a section in the draft that explains the different sources of latency (network, server, client) and how they affect the final RPM-number and how one can separate out these two components. It is also important to understand that the results are highly implementation-dependent. And explaining that in this section should help, I believe.

when you say, "...the results are highly implementation-dependent.",
which implementations are you referring to:

Network path? Server? Client? "Working Load" conditions?

Network path, Server and Client. Because the layer at which things are being measured is right above the HTTP layer. Thus, implementation-details all the way from the PHY up to HTTP matter and can influence the results.

All of the above?

Network implementation should influence the results, the others not so much (with a standardized metric, we should have equivalent results across implementations, at least that's what we said in RFC6576/BCP176). And it can be rather difficult to separate the (I count at least 4) results-influencing factors.

I think it is impossible to only have network implementation influence the results. For example, if UDP is used to measure latency the entire UDP-stack, IP-stack,... of the sender and receiver will influence the results.
[acm]
Agreed, although the one-to-one correspondence of packets on the wire and UDP datagrams is an advantage with UDP measurements. Many measurement systems use UDP today, and it has been straightforward to calibrate the delay introduced by the host stack and remove that component when necessary (within a tolerance of variation appropriate for network measurements, it may be nearly deterministic).

In some way, sender and receive UDP/IP/Driver/... stack is part of the "network implementation" here.
[acm]
We have performed the calibrations of the stack so that we can continue to use the notion of “wire-time” (that separate stack from network).

For the responsiveness it is exactly the same. We use HTTP/2, and thus this is part of the "network implementation".
[acm]
I would like to see more on this, because:
I haven’t measured or attempted to calibrate delays this high in the stack myself. But I suspect there may be less determinism and more host processing time/variation compared to UDP delay measurements. This is one place that new forms of implementation dependence might creep-in, in addition to the aspect of Working-Load generation (another possible implementation-dependency mentioned above).


Does that make sense?


Thanks,
Christoph



more stuff to consider,
Al



-----Original Message-----
From: ippm <ippm-bounces@ietf.org<mailto:ippm-bounces@ietf.org>> On Behalf Of Dave Taht
Sent: Tuesday, February 15, 2022 8:00 PM
To: Christoph Paasch <cpaasch=40apple.com@dmarc.ietf.org<mailto:cpaasch=40apple.com@dmarc.ietf.org>>
Cc: Marcus Ihlar <marcus.ihlar=40ericsson.com@dmarc.ietf.org<mailto:marcus.ihlar=40ericsson.com@dmarc.ietf.org>>; IETF IPPM WG
<ippm@ietf.org<mailto:ippm@ietf.org>>
Subject: Re: [ippm] Adoption call for draft-cpaasch-ippm-responsiveness

On Tue, Feb 15, 2022 at 4:11 PM Christoph Paasch
<cpaasch=40apple.com@dmarc.ietf.org<mailto:cpaasch=40apple.com@dmarc.ietf.org>> wrote:


Hello Greg,

On Feb 8, 2022, at 1:10 PM, Greg Mirsky <gregimirsky@gmail.com<mailto:gregimirsky@gmail.com>> wrote:

Hi Christoph,
apologies for the belated response and thank you for sharing interesting
details of you using the measurement method. I think that if the measurement
method can not only provide the Round-trip Per Minute (RPM) metric but expose
the network propagation and residential components of the round-trip delay,
then it seems to me, the scope of the draft to be aligned with the charter of
the IPPM WG and I'll be in favor of the WG adoption of the work.

What do you think? What is the opinion of the authors and the WG?


I am assuming that with "residential components" you mean the server/client-
side contribution to the measured latency, right?


In that case, yes the method does allow to separate these, as latency-probes
are sent on both the load-generating connections and on separate connections.
The difference between the two represents the "server-side contribution" to
the latency.

I remember vividly how "neat" it seemed that QUIC had adopted the idea
of an inband "ping" in version Q018 in 2012 (or so).

https://urldefense.com/v3/__https://docs.google.com/document/d/1WJvyZflAO2pq77<https://urldefense.com/v3/__https:/docs.google.com/document/d/1WJvyZflAO2pq77>
yOLbp9NsGjC1CHetAXV8I0fQe-B_U/edit__;!!BhdT!yrleDfru-45Lar4Qe5-
4bWSRTYtJSdvp8oh643W10p69kHM1mzsvPeUiZuj5$
Q018: Added a PING frame

What I don't remember or understand at the moment is I'm under the
impression that's in the SSL layer as of http2.0? There a good ref?





I think what would be helpful would be a section in the draft that explains
the different sources of latency (network, server, client) and how they affect
the final RPM-number and how one can separate out these two components. It is
also important to understand that the results are highly implementation-
dependent. And explaining that in this section should help, I believe.


Would that be in line with what you are looking for?


Thanks,
Christoph



Regards,
Greg

On Thu, Jan 6, 2022 at 4:42 PM Christoph Paasch <cpaasch@apple.com<mailto:cpaasch@apple.com>> wrote:


Hello Greg,

On Jan 6, 2022, at 3:00 PM, Greg Mirsky <gregimirsky@gmail.com<mailto:gregimirsky@gmail.com>> wrote:

Hi Christoph,
a happy and healthy New Year to you and All!


Happy New Year to you as well!

Thank you for your kind consideration of my notes and detailed responses.
Please find my follow-up notes in-line below under the GIM>> tag.



Thanks for your replies. Please see inline:

On Wed, Jan 5, 2022 at 10:52 AM Christoph Paasch <cpaasch@apple.com<mailto:cpaasch@apple.com>> wrote:


Hello Greg,

thanks for your comments. Please see inline:

On Dec 22, 2021, at 11:43 PM, Greg Mirsky <gregimirsky@gmail.com<mailto:gregimirsky@gmail.com>> wrote:

Dear Marcus, Authors, et al,
apologies for the belated response.
I've read the draft and have some comments to share with you:

as I understand it, the proposed new responsiveness metric is viewed as
the single indicator of a bufferbloat condition in a network. As I recall, the
discussion at Measuring Network Quality for End-Users workshop and on the
mailing list indicated, that there’s no consensus on what behaviors, symptoms
can reliably signal the bufferbloat.


We are not trying for this responsiveness metric to be "the single
indicator of bufferbloat". Bufferbloat can be measured in many different
number of ways. And each of these will produce a correct, but a different
result. Thus, "bufferbloat" is whatever the methodology tries to detect.


Let me give an example of two methodologies that are both correct but both
will produce entirely different numbers :


If we would decide to generate the load by flooding the network with UDP
traffic from a specific 4-tuple and measure latency with parallel ICMP pings.
Then, on a over-buffered FIFO queue we would measure huge latencies (thus
correctly expose bufferbloat), while on a FQ-codel queue we would not measure
any bufferbloat.


If on the other hand, the load-generating traffic is changing the source-
port for every single UDP-packet, then in both the FIFO-queue and the FQ-codel
queue we will measure huge amounts of bufferbloat.


Thus, these two methods both produced correct results but with hugely
different numbers in the FQ-codel case. [1]


Now, while both methods measure some variant of bufferbloat, they both
don't measure a realistic usage of the network.


GIM>> Thank you for the insights. It seems to me that what the method can
demonstrate is rather the level of efficiency of the AQM in the network for a
particular class of applications.



Yes, that is a good description. It is for a "particular class of
applications" and we are trying to make this class of applications
representative of a "typical user-scenario". (admittedly, we can debate
forever on what kind of applications are representative and I would love to
have that debate :-)).


On the point of "efficiency of the AQM". I would go even further that it's
not only AQM but also the client- and server-side implementations of these
applications (as noted further below).






That is why the "Responsiveness under working conditions" tries to clearly
specify how the load is generated and how the latency is being measured. And
it does not measure "bufferbloat" but it measures "responsiveness under
working conditions" based on the methodology that is being used (using HTTP/2
or HTTP/3, multiple flows, ...). It does expose bufferbloat which can happen
in the network. It also exposes certain server-side behaviors that can cause
(huge amounts of) additional latency - those behaviors are typically not
called "bufferbloat".


GIM>> Thank you for pointing out that the result of the RTT measurement has
two contributing factors - network and server.



Yes, servers contribute as do the client-side implementations. It's all
three (client, network, server) that need to work "correctly" to achieve good
responsiveness. Btw., as we are now gathering more experience with our
methodology in different environments we find that the biggest portions of
latency actually come from the server-side. We see several seconds of latency
introduced by the HTTP/2 and TCP implementations.


It seems worth enhancing the method to localize each contribution and
measure them separately.



With the latency measuring probes being sent on load-bearing connections
and separate connections and with the separate connections serving to measure
DNS/TCP/... individually, the different data-points actually allow to localize
to some extend.


However, I would be reluctant to dive too deep into localization/trouble-
shooting/debugging of networks as part of this I-D. As this opens a whole new
can of worms. We could then start thinking about sending latency-probes while
playing with the IP TTL to find which router is introducing the latency,...
It's an entirely different research-topic IMO :-) Dave Taht was thinking of
starting something along these lines
(https://urldefense.com/v3/__https://github.com/dtaht/wtbb__;!!BhdT!yrleDfru-<https://urldefense.com/v3/__https:/github.com/dtaht/wtbb__;!!BhdT!yrleDfru->
45Lar4Qe5-4bWSRTYtJSdvp8oh643W10p69kHM1mzsvPRghwEjg$ ).



It seems that it would be reasonable to first define what is being
measured, characterized by the responsiveness metric. Having a document that
discusses and defines the bufferbloat would be great.


I agree that there is a lack of definition for what "bufferbloat" really
is.


The way we look at "responsiveness under working conditions" is that it
measures the latency in conditions that may realistically happen in worst-case
scenarios with end-users/implementations that are non-malicious (non-malicious
to exclude the UDP-flooding scenario).


Thus, I assume we should make a better job at explaining this. The lack of
a formal definition of "bufferbloat" doesn't help and thus we are indeed using
this term a bit freely in the current draft. We will improve the Introduction
to better set the stage
(https://urldefense.com/v3/__https://github.com/network-quality/draft-cpaasch-<https://urldefense.com/v3/__https:/github.com/network-quality/draft-cpaasch->
ippm-responsiveness/issues/31__;!!BhdT!yrleDfru-45Lar4Qe5-
4bWSRTYtJSdvp8oh643W10p69kHM1mzsvPU0pYBOv$ ).


It seems like in the foundation of the methodology described in the draft
lies the assumption that without adding new flows the available bandwidth is
constant, does not change. While that is mostly the case, there are
technologies that behave differently and may change bandwidth because of the
outside conditions. Some of these behaviors of links with variable discrete
bandwidth are discussed in, for example, RFC 8330 and RFC 8625.


I'm not sure I entirely understand your comment. But let me explain why we
are gradually adding new flows:


1. TCP-implementations have usually a fixed limit for the upper bound of
the receive window. In some networks that upper bound is lower than the BDP of
the network. Thus, the only way to reach full capacity is by having multiple
flows.

2. Having multiple connections allows to quicker achieve full capacity in
high-RTT networks and thus speeds up the test-duration.

3. In some networks with "random" packet-loss, congestion-control may come
in the way of achieving full capacity. Again, multiple flows will work around
that.


GIM>> I might have asked several questions at once. Let me clarify what I
am looking for:


As I understand the method of creating the "working conditions in a
network" is based on certain assumptions. First, seems is that the bandwidth
is symmetrical between the measurement points. Second, that the bandwidth
doesn't change for the duration of the measurement session. AFAIK, in the
access networks, both are not necessarily always the case.


We don't have the assumption that bandwidth is symmetrical (assuming, you
mean uplink/downlink symmetry - please clarify otherwise).


The load-generating algorithm runs independently for uplink and downlink
traffic. And it is perfectly fine when both have huge asymmetry.



Regarding the stability of the bandwidth:
You are making a good point indeed that we assume that the bandwidth is to
some extend stable while ramping up the flows to "working conditions".
Admittedly that assumption does not always hold, and that is one of the
reasons why we try hard for the test to not take too long.

I'm not sure how we could adjust the algorithm for varying bandwidth
without introducing too much complexity. I'm open for suggestions :-)


On the other hand, I might have missed how the method of creating the
"working conditions" guarantees a symmetrical load between the measurement
points.


As mentioned above, we don't assume a symmetrical load. Can you show us
where in the draft we give that impression, so we can fix that?




Then, I find the motivation not to use time units to express the
responsiveness metric not convincing:


  "Latency" is a poor measure of responsiveness, since it can be hard
  for the general public to understand.  The units are unfamiliar
  ("what is a millisecond?") and counterintuitive ("100 msec - that
  sounds good - it's only a tenth of a second!").


Can you expand on what exactly is not convincing to you? Do you think that
people will mis-understand the metric or that milli-seconds is the right way
to communicate responsiveness to the general public?


GIM>> Let me try. We know packet delay requirements for AR, VR
applications. I believe that gamers are familiar with these numbers too. The
same is likely the case for the industrial automation use cases served, for
example, by Deterministic Networking.



I can understand that for a technical audience, milli-seconds is easy and
familiar. A non-technical audience might be more open to accepting a new
"higher-is-better" metric. Responsiveness is something new and abstract so,
it's kind of natural that it comes with a new unit.


But I fully recognize that that's a controversial topic and can be
discussed at length :)



Cheers,
Christoph




Thanks a lot,
Christoph

[1] And there are many networks that prioritize ICMP pings, thus we could
observe even more different results based on what protocol is used to measure
the latency.



On Mon, Dec 6, 2021 at 7:53 AM Marcus Ihlar
<marcus.ihlar=40ericsson.com@dmarc.ietf.org<mailto:marcus.ihlar=40ericsson.com@dmarc.ietf.org>> wrote:


Hi IPPM,



This email starts an adoption call for draft-cpaasch-ippm-responsiveness,
"Responsiveness under Working Conditions”. This document specifies the “RPM
Test” for measuring user experience when the network is fully loaded. The
intended status of the document is Experimental.




https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft->
cpaasch-ippm-responsiveness/__;!!BhdT!yrleDfru-45Lar4Qe5-
4bWSRTYtJSdvp8oh643W10p69kHM1mzsvPU3zpKI3$


https://urldefense.com/v3/__https://datatracker.ietf.org/doc/html/draft-<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/html/draft->
cpaasch-ippm-responsiveness-01__;!!BhdT!yrleDfru-45Lar4Qe5-
4bWSRTYtJSdvp8oh643W10p69kHM1mzsvPf9N-E8h$




This adoption call will last until Monday, December 20. Please review the
document, and reply to this email thread to indicate if you think IPPM should
adopt this document.




BR,

Marcus



_______________________________________________
ippm mailing list
ippm@ietf.org<mailto:ippm@ietf.org>
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ippm__;!!Bhd<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/ippm__;!!Bhd>
T!yrleDfru-45Lar4Qe5-4bWSRTYtJSdvp8oh643W10p69kHM1mzsvPTFWzNHi$


_______________________________________________
ippm mailing list
ippm@ietf.org<mailto:ippm@ietf.org>
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ippm__;!!Bhd<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/ippm__;!!Bhd>
T!yrleDfru-45Lar4Qe5-4bWSRTYtJSdvp8oh643W10p69kHM1mzsvPTFWzNHi$




_______________________________________________
ippm mailing list
ippm@ietf.org<mailto:ippm@ietf.org>
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ippm__;!!Bhd<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/ippm__;!!Bhd>
T!yrleDfru-45Lar4Qe5-4bWSRTYtJSdvp8oh643W10p69kHM1mzsvPTFWzNHi$



--
I tried to build a better future, a few times:
https://urldefense.com/v3/__https://wayforward.archive.org/?site=https*3A*2F*2<https://urldefense.com/v3/__https:/wayforward.archive.org/?site=https*3A*2F*2>
Fwww.icei.org__;JSUl!!BhdT!yrleDfru-45Lar4Qe5-
4bWSRTYtJSdvp8oh643W10p69kHM1mzsvPXLNE64R$

Dave Täht CEO, TekLibre, LLC

_______________________________________________
ippm mailing list
ippm@ietf.org<mailto:ippm@ietf.org>
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ippm__;!!Bhd<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/ippm__;!!Bhd>
T!yrleDfru-45Lar4Qe5-4bWSRTYtJSdvp8oh643W10p69kHM1mzsvPTFWzNHi$