Re: [alto] Zaheduzzaman Sarker's Discuss on draft-ietf-alto-performance-metrics-20: (with DISCUSS and COMMENT)

Zaheduzzaman Sarker <zaheduzzaman.sarker@ericsson.com> Wed, 02 March 2022 10:30 UTC

Return-Path: <zaheduzzaman.sarker@ericsson.com>
X-Original-To: alto@ietfa.amsl.com
Delivered-To: alto@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BB0643A103F; Wed, 2 Mar 2022 02:30:23 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.11
X-Spam-Level:
X-Spam-Status: No, score=-2.11 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ericsson.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id r4SLH8oM6tPL; Wed, 2 Mar 2022 02:30:20 -0800 (PST)
Received: from EUR04-DB3-obe.outbound.protection.outlook.com (mail-db3eur04on061d.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe0c::61d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 051BC3A1041; Wed, 2 Mar 2022 02:30:18 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FjLKUW/QBG8WX01+VtOz2bBBWRZJag2K7KK6MLGuuK4diu0RGnDgEIkHnsQ/UJrMO1JvAdvUT4hXu++h15MWEFaIwzoFTvTmGfnHI2rFc8GZJpvDGS/awaH3ntZKpKLirFFPvv8t3iWaqBbVqZ1MeRW+kaAhYNClVnyNNbQhW4GrJAVXEd+fZE+XvCFeVloUPCiQZxTIsL2W9VVP0tgHihFzhx3Y0SG4NKSnVCwFJDjiHwtae4hXJOxAgpsGnshJqdj+mKuVlx/FmijC+8Jw02Kl28yq6aWyPya4B7Rl/aYV0Tkr5l7SjKoxPuJt6aueifCv9iT4tu/QrKpu/EdV8A==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3TnXT6XphR6oyzJilcgJQvY66MoH4e+3yX7ZIqZUsCw=; b=IcAN3LloitUbDNZ8Y+9/+JTtyhoHoyor0RWsqILPWQLJt4BucWdzfPFCpZ3fNVcUW3aVMxGR+TAgCDeXplegS/Y3AkVWW8QDGxZnbMGiTq4iFYaajAyh6VOsIgcUBVBtv4MpyeBbNKmk6X45sxRMLIsggEB9H2l4PokZEy45Elz/hOwrW3nT9HP1lwRG5jEZKQTT2n9x4tPPAtlPkgLtVY2WNf6Dbhn3Q7N+5z4bHlAGLbTxEivvjTcTix1mt5BxbLIE3fQasUiwxieByjo5J+U4L2pL4m0JgV9lyyNFjK1Fsf4eC80+XJJiudTHieH5TaPdpV57qYvuhsoCTH40Hw==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=ericsson.com; dmarc=pass action=none header.from=ericsson.com; dkim=pass header.d=ericsson.com; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ericsson.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3TnXT6XphR6oyzJilcgJQvY66MoH4e+3yX7ZIqZUsCw=; b=gDneEtonIfNm/bB7JWtLSnydlSuzSjP2LH/m2AOhQroSqMK0qhQh9aeN5I/qd6r3CSjAZ2bMYeTeihHEYdQVgLTb1Lfd7CJQEBrBAozhfRsmUvzM1RioKd4TWZx+OybDNvalrKZvIMyP911a3Ju9pFuMHNkRifSjz2Q9mwqvfJU=
Received: from AM0PR07MB4178.eurprd07.prod.outlook.com (2603:10a6:208:b7::31) by AM9PR07MB7074.eurprd07.prod.outlook.com (2603:10a6:20b:2cd::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5038.9; Wed, 2 Mar 2022 10:30:15 +0000
Received: from AM0PR07MB4178.eurprd07.prod.outlook.com ([fe80::fcfb:44ee:db1c:521a]) by AM0PR07MB4178.eurprd07.prod.outlook.com ([fe80::fcfb:44ee:db1c:521a%3]) with mapi id 15.20.5038.014; Wed, 2 Mar 2022 10:30:13 +0000
From: Zaheduzzaman Sarker <zaheduzzaman.sarker@ericsson.com>
To: "Y. Richard Yang" <yry@cs.yale.edu>
CC: Qin Wu <bill.wu=40huawei.com@dmarc.ietf.org>, The IESG <iesg@ietf.org>, "alto-chairs@ietf.org" <alto-chairs@ietf.org>, "draft-ietf-alto-performance-metrics@ietf.org" <draft-ietf-alto-performance-metrics@ietf.org>, "alto@ietf.org" <alto@ietf.org>
Thread-Topic: [alto] Zaheduzzaman Sarker's Discuss on draft-ietf-alto-performance-metrics-20: (with DISCUSS and COMMENT)
Thread-Index: AdfngNK+EK/hF42enECKzU+gnPg+OBFZ1K4AAE4Wp4A=
Date: Wed, 02 Mar 2022 10:30:13 +0000
Message-ID: <704F5CEC-777D-4D0C-A688-DBB772C9B7CF@ericsson.com>
References: <d2c14e67364f480b902e532894930daa@huawei.com> <CANUuoLq=7Cr+WpNfVTMFTL=z8FhypTYQaakwbCW4mwjSXZNfGQ@mail.gmail.com>
In-Reply-To: <CANUuoLq=7Cr+WpNfVTMFTL=z8FhypTYQaakwbCW4mwjSXZNfGQ@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach: yes
X-MS-TNEF-Correlator:
x-mailer: Apple Mail (2.3654.120.0.1.13)
authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=ericsson.com;
x-ms-publictraffictype: Email
x-ms-office365-filtering-correlation-id: cd3f9668-dbcf-4216-005f-08d9fc37a2df
x-ms-traffictypediagnostic: AM9PR07MB7074:EE_
x-microsoft-antispam-prvs: <AM9PR07MB70740A7EFEC1D200ACDE2F979F039@AM9PR07MB7074.eurprd07.prod.outlook.com>
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: K53xgrV1GHycEaFaLU5hN+vBgzHhnx8S8wrfqj6+cAPiKvPyMZM2M+6GIQT83dfx/kII1/VhqmvVK6iUcF+KbBfQNyiwqE3U+crLO06GTsOEKFAKA4JG4bWwuWNnQDsPwRcxJQfWlmyfST9rM9h9VrjIBgeJEIdm+y9klUeiCyjEleVshnTFfb8kRGiOKxt04v4CJvku+jS4Cgsdnt5w72ogGSkKhXQhyTdfFysV/eQ8lzHMwIEILSI82ODhQAW3253ZkpJ4+zjlxsdVj5vVotF5k8Z7c63k1Lu2K/r4ol86mGzQBXP6voiEgpRzsZttgk7nNMZZ869jP5GVw8kJVSgf2AQmnalSBT9cldr0YTE2eT7lg6f2jrbkB/oL8rrS8E4v6uewpMxllF31zyy/GkSzB4iKYB+M0cN1oV/ulf6fnLYJ8zOZl7S8TkpFY0p0JJ64vKDRKk1XJwtR4tt9yjehDcZ8ct+2MGV2KcZCCW2dqnIPYosW5lp8H9wIe+CY9X9D33BghdAK03Yh7bIPDx5hJWevgie7YMhrKa3vCoA5qRg1ASbFnNF0hqGb0VxnweA98Qi6oabjE7UXxeyruk74NwG31SLtfzG643dxWrb7pRKZSSYo//D4mhppC/c7v6fP9SkVruFbrCGbxMkyH1Dz/L4MxhOZ/bQRM0PJh+wAsILkLM2B3g2txUxW7n25dei61Vsvknqz/ytmjF3bedp2Nmsf5FBsZiMhSg7si5dNfBDkhZ2RMZZ1I6YJF0DZjROw3D5eiNbmFQt0Y8VtgCbS+7TU2FPQ8wO49w+nsNFWa/Niebxln42qv9+M2Ngq3SKkMqmRGYQGK1wCyZlJpdcY7KxrTb5/Oh0KZDK1/9kTiJLea3pxeaYoph6PJYDSvHFHDMag72PcQZ/mXfmfBQ==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR07MB4178.eurprd07.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230001)(4636009)(366004)(5660300002)(44832011)(86362001)(2616005)(166002)(36756003)(99936003)(26005)(83380400001)(82960400001)(53546011)(8936002)(186003)(2906002)(66574015)(122000001)(6512007)(71200400001)(66556008)(76116006)(66946007)(6506007)(66476007)(66446008)(8676002)(38100700002)(38070700005)(4326008)(33656002)(64756008)(91956017)(316002)(296002)(54906003)(6916009)(508600001)(6486002)(966005)(45980500001); DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: BqySr3+FCAM2drkjZEBP9brIRi8RBhRd7qEuXGsKhVEER1mQIYQLnQsFMNgN7gMdIy/q4FoyhNwbeqkh5xsk4/XOjTVPPQ3MnNGVCrUUMp7qWzMU8qCnUf+IWU182EDdhowy2RGHExPbiOF/1K+ezhhkfU4sJjGnVpIRSo7OGKlD6lF85V4V+uq/7tLtAb5UWsycqFtc4fnb/XXlai0qTT6xsD8W1C4y5l3KtFLJbJQIc67k2j4FfYxSswFhXc7imXpc8m4MqOFx8K7dXbHYdfvHk3jL3P1bmZk06lQDgQtrCdhlyrlBVG9/WMcGOnl8miDUzmqYFrjv3FPvRQ80hwcPj7RMI9T0wI0/A8O1l+jPJPHFO/rJnRNIO3virLNDLDF63AfVkWMxGQXHxlBxK9/1REDgWNpFUdHMl+giRzGJHsV5ZoFvmhCeXiJ1nyK37TED6lEWFPL0C9wWlQUY7s3TjcLlcbmiYVtDd2DKnrLhYYKU79m+7yt6oqSUu1wxXvsOzev3KV5yKBAh2pzeFNvxkn0/nUK/pV7gMaa4u8ZEyp/h1DVigNW/yE22j+eRKoGCteRT/Ul8w6f3OJEpWapbkDZLGaDJaLZ6TMBkshLILzU1qToCRXLZ9jjFkFAOjH2cK06KFrzKWHwJfN51npUR5A1eCAhGmuIpeCmebekyY3yar823WhtMS6bKIpJ2RZdyMlR62Pj7DnkccqciYMmVzwmReM265MJpFKAQVoE6Y1yVe/Vw0T0fiYJ7Gw7xKEa5NYRT4kcpGwXW1QWb0bw90QaLSlq+SiGh5VVD7k1lcn6sSCnQbe9v3YaCuA/NQY3cys0l8ty0/gaj7fr0M5y4iMrbqCMCFpV53gJbjY7bzWkfQzY80Sv4LRMPnfWV3sbM7LK9jI1E83S91gFr/bh78XaCGrcpAWhq1VIvoVlAT4d9UOXm4nON7z1XleBcdn/1ydgZ/0xnoae3gDie0lrBLiRqENuojjeEKWNej4aCNlkAqgb9ed8p14BnxeZrL78XX/GElsqhp8eGXhEzt7Tipag6syxfiLDAHwgkFRcCm8u6OzZQI3NJ0UkIjSuPSC7NH8jEUIdmXFk5NWmaz3L+Lan9pOpce2IAJJJmPq6c5pB1OSHsSM6JObM3ixGCEMYJp421qNp+iVr77zDviWs2ze4I9vWYG14mjgRVsGkVO1eBedjt7hKC4otbNpL1lYQ1hp5D7AG0DZZxbBzo8PqK8Q/B1XXrP2eIiKt4bacpsjtFxgm6+k/q81WE0Gao8xJC/igOEM3xnlV6eKXHNQTyqGZvokLxOTizOrkXvWpDBy2dm4MVWMF+kFy70r+J+Rr9M1mLOKmr+orMpfkw5riEZ2SpVOYdTCJTYbD67wCeHrHyxbsx3ckdFzJa+Ae/Rm3DesC7NccOrDcr45lZBfoDwM7FmMeOccDly5NMOo+SY3mTTEJWPyxEdeFo7m9Fq4NdPyW9kt2m7kODOP4zNe+9otnuY19Otv3ttR+jHXSSpsASzwDUBeTZIQv/AbK1xWw6xayHXV+R1ZVE+XbCclRSJPOleMxcu4Tby3rHFbV5m7WoJO+8exkAir4BPWdcOnccuHtcjWwvo42xHK7GevqPGkGR8TRm4qOC5VlN+TmT3BDe3S682ttYblwF96douL4W7oEHzOD6tnRLfqQfcGvf/4vZ1rob13j/A/wpzE4E91Z0cJ9XaS8FF99agvNIxN3KrRMCr07pVNc05uAG1g==
Content-Type: multipart/signed; boundary="Apple-Mail=_2AFAF018-8A23-4BB2-B067-B7B37D52B620"; protocol="application/pkcs7-signature"; micalg="sha-256"
MIME-Version: 1.0
X-OriginatorOrg: ericsson.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: AM0PR07MB4178.eurprd07.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cd3f9668-dbcf-4216-005f-08d9fc37a2df
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Mar 2022 10:30:13.4935 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 92e84ceb-fbfd-47ab-be52-080c6b87953f
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ys66En1k3Ylr4HOBCXD/f18c3chY2C3ihe/ZxMKtYTkwsaa+uqxNLUrLdKMrv7wFYlk6mHdohzrVwCONlPXujRF6fWuYrdiNzAXxUUm/xtvFya734RP7C7iDNLy5cl4E
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR07MB7074
Archived-At: <https://mailarchive.ietf.org/arch/msg/alto/lGstM3y444ES3IYgAhvj0FojF4o>
Subject: Re: [alto] Zaheduzzaman Sarker's Discuss on draft-ietf-alto-performance-metrics-20: (with DISCUSS and COMMENT)
X-BeenThere: alto@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Application-Layer Traffic Optimization \(alto\) WG mailing list" <alto.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/alto>, <mailto:alto-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/alto/>
List-Post: <mailto:alto@ietf.org>
List-Help: <mailto:alto-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/alto>, <mailto:alto-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 02 Mar 2022 10:30:24 -0000

Thanks for the updates. 

I however, didn’t notice the change in section 2.2 egarding JSON number format, which previously was agreed to be changed, in the 25th of this document.

//Zahed



> On 28 Feb 2022, at 22:14, Y. Richard Yang <yry@cs.yale.edu> wrote:
> 
> Hi Zaheduzzaman,
> 
> We have posted the latest version of the document: https://datatracker.ietf.org/doc/html/draft-ietf-alto-performance-metrics-24 <https://datatracker.ietf.org/doc/html/draft-ietf-alto-performance-metrics-24>
> 
> Could you please take a look to see if all of your comments are addressed? In particular, 
> - We checked and made sure that the normative references are correct.
> - We updated the abstract to clarify the wording and added sentences in Sec. 1 on the uses.
> - We revised the final wording of 2.2 on the number format
> - We checked all json examples and fixed the issues.
> 
> Please take a look and let us know if there are remaining issues to be addressed.
> 
> Thank you so much!
> Richard
> 
> On Thu, Dec 2, 2021 at 8:32 AM Qin Wu <bill.wu=40huawei.com@dmarc.ietf.org <mailto:40huawei.com@dmarc.ietf.org>> wrote:
> Hi, Zaheduzzaman:
> -----邮件原件-----
> 发件人: Zaheduzzaman Sarker via Datatracker [mailto:noreply@ietf.org <mailto:noreply@ietf.org>] 
> 发送时间: 2021年12月2日 19:35
> 收件人: The IESG <iesg@ietf.org <mailto:iesg@ietf.org>>
> 抄送: draft-ietf-alto-performance-metrics@ietf.org <mailto:draft-ietf-alto-performance-metrics@ietf.org>; alto-chairs@ietf.org <mailto:alto-chairs@ietf.org>; alto@ietf.org <mailto:alto@ietf.org>; ietf@j-f-s.de <mailto:ietf@j-f-s.de>; ietf@j-f-s.de <mailto:ietf@j-f-s.de>
> 主题: Zaheduzzaman Sarker's Discuss on draft-ietf-alto-performance-metrics-20: (with DISCUSS and COMMENT)
> 
> Zaheduzzaman Sarker has entered the following ballot position for
> draft-ietf-alto-performance-metrics-20: Discuss
> 
> When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.)
> 
> 
> Please refer to https://www.ietf.org/blog/handling-iesg-ballot-positions/ <https://www.ietf.org/blog/handling-iesg-ballot-positions/>
> for more information about how to handle DISCUSS and COMMENT positions.
> 
> 
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-alto-performance-metrics/ <https://datatracker.ietf.org/doc/draft-ietf-alto-performance-metrics/>
> 
> 
> 
> ----------------------------------------------------------------------
> DISCUSS:
> ----------------------------------------------------------------------
> 
> I perhaps understand the intention of extending the ALTO protocol so that the ALTO client and server have defined way of exchanging values for already defined metrics. However, I need to agree with my fellow AD colleagues that this document need to describe why those metrics are needed and describe the relationship with other RFCs those defines those metrics mostly for other contexts. To that end all the RFCs in the Table 1 in section 1 need to be normative references.
> 
> [Qin Wu] I think the key use case is defined in RFC7752 section 2.2, i.e., export BGP-LS collected topology data to ALTO server and the ALTO server expose data to the client. RFC8571 provides additional performance metric related data which is part of topology data. Most of performance cost metrics derived from metrics defined in RFC8571.
> Another two relevant use cases are documented in section 3 of draft-xie-alto-lmap-00, one is targeted to network operators who need to understand the performance of their networks, the performance of the suppliers (downstream and upstream networks), the performance of Internet access services, and the impact that such performance has on the experience of their customers.
> The other is targeted to regulators who want to evaluate the performance of the network services offered by operators.
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
> 
> Thanks for the work on this document and thanks to Brian Trammell for his TSVART early review.
> 
> I have following comments which I believe will improve the document quality -
> 
> 1. In the abstract I read about "a better delay performance" and was hoping it will be clear to me what is "a better delay performance". Unfortunately, I was unable to get that. This comes to the point that this document needs to describe why purpose of using the defined metrics well.
> [Qin Wu] See clarification above.
> 
> 2. Section 2.2 says
> 
>     The number MUST be a non-negative JSON integer in the range [0, 100] (i.e.,
>     greater than or equal to 0 and less than or equal to 100), followed by an
>     optional decimal part, if a higher precision is needed.
> 
>   This should be a JSON number type not integer type.
> [Qin Wu] See clarification to Ben's comments. The format of percentile is integer number followed by optional decimal part starting with the '.' separator.
> 3. There are number of broken JSON examples. for example, in section 4.2.3
>     "ipv4:192.0.2.2" {
>       "ipv4:192.0.2.89" :    0,
>       "ipv4:198.51.100.34": 2000
>     }
>    missing ":" after  ipv4:192.0.2.2
> [Qin Wu] Agree to fix this.
> 4. Content-Length: TBA in the examples, I actually don't know how to interpret it.
> 
> [Qin Wu] Agree to fix this.
> 
> _______________________________________________
> alto mailing list
> alto@ietf.org <mailto:alto@ietf.org>
> https://www.ietf.org/mailman/listinfo/alto <https://www.ietf.org/mailman/listinfo/alto>
>