Re: [alto] Review for draft-ietf-alto-performance-metrics-12
Qin Wu <bill.wu@huawei.com> Tue, 12 January 2021 09:37 UTC
Return-Path: <bill.wu@huawei.com>
X-Original-To: alto@ietfa.amsl.com
Delivered-To: alto@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id CF2633A0E5C for <alto@ietfa.amsl.com>; Tue, 12 Jan 2021 01:37:12 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Level:
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YzxZ_vxiuSen for <alto@ietfa.amsl.com>; Tue, 12 Jan 2021 01:37:09 -0800 (PST)
Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CB5C53A0E4C for <alto@ietf.org>; Tue, 12 Jan 2021 01:37:08 -0800 (PST)
Received: from fraeml703-chm.china.huawei.com (unknown [172.18.147.207]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4DFQMK25Vzz67YGl; Tue, 12 Jan 2021 17:31:57 +0800 (CST)
Received: from fraeml703-chm.china.huawei.com (10.206.15.52) by fraeml703-chm.china.huawei.com (10.206.15.52) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2106.2; Tue, 12 Jan 2021 10:37:01 +0100
Received: from DGGEML421-HUB.china.huawei.com (10.1.199.38) by fraeml703-chm.china.huawei.com (10.206.15.52) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256) id 15.1.2106.2 via Frontend Transport; Tue, 12 Jan 2021 10:37:00 +0100
Received: from DGGEML531-MBS.china.huawei.com ([169.254.5.18]) by dggeml421-hub.china.huawei.com ([10.1.199.38]) with mapi id 14.03.0509.000; Tue, 12 Jan 2021 17:36:54 +0800
From: Qin Wu <bill.wu@huawei.com>
To: Jensen Zhang <jingxuan.n.zhang@gmail.com>
CC: IETF ALTO <alto@ietf.org>, "Y. Richard Yang" <yry@cs.yale.edu>, Kai Gao <kaigao@scu.edu.cn>
Thread-Topic: [alto] Review for draft-ietf-alto-performance-metrics-12
Thread-Index: AdboxnSWlhMFTOkvTGOkiEFFvPMS7Q==
Date: Tue, 12 Jan 2021 09:36:52 +0000
Message-ID: <B8F9A780D330094D99AF023C5877DABAADCCD890@dggeml531-mbs.china.huawei.com>
Accept-Language: zh-CN, en-US
Content-Language: zh-CN
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [10.136.101.103]
Content-Type: multipart/alternative; boundary="_000_B8F9A780D330094D99AF023C5877DABAADCCD890dggeml531mbschi_"
MIME-Version: 1.0
X-CFilter-Loop: Reflected
Archived-At: <https://mailarchive.ietf.org/arch/msg/alto/YoANE2gvjoDDotqBX6MbEOgThaM>
Subject: Re: [alto] Review for draft-ietf-alto-performance-metrics-12
X-BeenThere: alto@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Application-Layer Traffic Optimization \(alto\) WG mailing list" <alto.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/alto>, <mailto:alto-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/alto/>
List-Post: <mailto:alto@ietf.org>
List-Help: <mailto:alto-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/alto>, <mailto:alto-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Jan 2021 09:37:13 -0000
Jensen: My impression, <stat> is an optional item in the ANBF, therefore the metric identifier can be used without <stat> as suffix. I think your proposed change makes sense to me. -Qin 发件人: Jensen Zhang [mailto:jingxuan.n.zhang@gmail.com] 发送时间: 2021年1月12日 15:36 收件人: Qin Wu <bill.wu@huawei.com> 抄送: IETF ALTO <alto@ietf.org>; Y. Richard Yang <yry@cs.yale.edu>; Kai Gao <kaigao@scu.edu.cn> 主题: Re: [alto] Review for draft-ietf-alto-performance-metrics-12 Hi Qin, I agree with you that the constraints checking should rely on the implementation. I'm OK that it is not in the scope of this document. For other comments, I have checked v-13. I think most of them have been addressed, except for the following one. Section 2.2., paragraph 16: > If a metric has no <stat> (and hence no - as well), the metric MUST recommend adding " surrounding -, or using dash character instead; if possible, giving the precise BNF grammar will be better, as I see some metrics names also include the dash character ("-"). > be considered as the 50 percentile (median). Since this scheme is > common for all metrics defined in this document, below we only > specify the base identifier. Although I can understand this sentence, I still think it should be better clarified. I would suggest giving the BNF grammar at the beginning of this section, e.g., ... Hence, each performance metric's identifier should indicate the statistic (i.e., an aggregation operation), to become <metric-identifier> ::= <metric-base-identifier> [ '-' <stat> ] where <stat> MUST be one of the following: And changing the last paragraph of the section to: If '-' <stat> is not present in <metric-identifier>, the metric MUST be considered as the 50 percentile (median). Thanks, Jensen On Tue, Jan 12, 2021 at 2:22 PM Qin Wu <bill.wu@huawei.com<mailto:bill.wu@huawei.com>> wrote: Hi, Jensen: Speak as individual, My answer to your following question is false as well, even based on RFC7285, defining hopecount as float point value seem also weird. I think we can rely on implementation or some automation tools for constraints checking, but it is not scope of this document. For other comments, I think Richard have addressed in v-13. Please double check it. Thanks -Qin [alto] Review for draft-ietf-alto-performance-metrics-12 Jensen Zhang <jingxuan.n.zhang@gmail.com<mailto:jingxuan.n.zhang@gmail.com>> Tue, 13 October 2020 04:17 UTCShow header<https://mailarchive.ietf.org/arch/msg/alto/qZrkPza-vEUcIqQMR3OJfk8G-uw/> Dear ALTOers and authors of draft-ietf-alto-performance-metrics-12, Below is my review for draft-ietf-alto-performance-metrics-12. Best regards, Jensen ============================================== General issue: The document is well written. I only have one question about the design part: the base ALTO protocol only uses the cost-mode to infer the value format, e.g., "numerical" infers the cost value MUST be a floating-point value; but this document requires different value formats for different cost-metrics, e.g., "delay-ow" requires the non-negative floating-point value, and "hopcount" requires the non-negative integer value. But based on Sec 11.3.2.3 of RFC7285, in the "constraints" field, "ALTO servers SHOULD use at least IEEE 754 double-precision floating point [IEEE.754.2008] to store the cost value". I wonder if a test constraint expression like "eq 3.1" for the cost-metric "hopcount" is valid. Should the ALTO server reject such a request? According to RFC7285, it should be valid. But according to this document, it is clearly always false. ============================================== Nits and writing suggestions: Section 1., paragraph 5: > The purpose of this document is to ensure proper usage of the > performance metrics defined in Table 1; it does not claim novelty of > the metrics. The Origin column of Table 1 gives the RFC which > defines each metric. Origin -> Origin Example (to be consistent with the table) > We can rough classify the performance metrics into two categories: > those derived from the performance of individual packets (i.e., one- > way delay, round-trip delay, delay variation, hop count, and loss > rate), and those related with bandwidth (TCP throughput, residue > bandwidth and max reservable bandwidth). These two categories are > defined in Section 3 and Section 4 respectively. Note that all > metrics except round trip delay are unidirectional. Hence, a client > will need to query both directions if needed. Section 2., paragraph 1: > When defining the metrics in Table 1, this document considers the > guidelines specified in [RFC6390], which requires fine-grained > specification of (i) Metric Name, (ii) Metric Description, (iii) > Method of Measurement or Calculation, (iv) Units of Measurement, (v) > Measurement Points, and (vi) Measurement Timing. In particular, for > each metric, this document defines (i) Metric Name, (ii) Metric > Description, and (iv) Units of Measurement. The Measurement Points > are always specified by the specific ALTO services; for example, > endpoint cost service is between the two end points. end points -> endpoints Section 2.1., paragraph 11: > A particular type of "estimation is direct "import", which indicates > that the value of the metric is imported directly from a specific > existing protocol or system. Specifying "import" as source instead source -> the source > of the more generic "estimation" may allow better tracing of > information flow. For an "import" metric, it is RECOMMENDED that the > "parameters" field provides details to the system from which raw data > is imported. In particular, one may notice that the set of end-to- > end metrics defined in Table 1 has large overlap with the set defined > in [RFC8571], in the setting of IGP traffic engineering performance > metrics for each link (i.e., unidirectional link delay, min/max > unidirectional link delay, unidirectional delay variation, > unidirectional link loss, unidirectional residual bandwidth, > unidirectional available bandwidth, unidirectional utilized > bandwidth). Hence, an ALTO server may use "import" to indicate that > its end-to-end metrics are computed from link metrics imported from > [RFC8571]. Section 2.2., paragraph 2: > percentile, with letter p followed by a number p: a number p -> a number Section 2.2., paragraph 16: > If a metric has no <stat> (and hence no - as well), the metric MUST recommend adding " surrounding -, or using dash character instead; if possible, giving the precise BNF grammar will be better, as I see some metrics names also include the dash character ("-"). > be considered as the 50 percentile (median). Since this scheme is > common for all metrics defined in this document, below we only > specify the base identifier. Section 3., paragraph 1: > This section introduces ALTO network performance metrics including > one way delay, round trip delay, delay variation, hop count, and > packet loss rate. They measure the "quality of experience" of the > stream of packets sent from a resource provider to a resource > consumer. The measures of each individual packet (pkt) can include > the delay from the time that the packet enters the network to the > time that the packet leaves the network (pkt.delay); the number of the time that -> the time when > network hops that the packet traverses (pkt.hopcount); and whether > the packet is dropped before reaching destination (pkt.dropped). The destination -> the destination > semantics of the performance metrics defined in this section is that > they are statistics (percentiles) computed from these measures; for > example, the x-percentile of the one-way delay is the x-percentile of > the set of delays {pkt.delay} for the packets in the stream. Section 3.1.3., paragraph 1: > Intended Semantics: To specify spatial and temporal aggregated delay spatial -> the spatial > of a stream of packets from the specified source and the specified > destination. The spatial aggregation level is specified in the query > context (e.g., PID to PID, or endpoint to endpoint). Section 3.1.4., paragraph 2: > "sla": Many networks provide delay in their application-level service > level agreements. It is RECOMMENDED that the "parameters" field of > an "sla" one-way delay metric provides a link ("link") to the SLA I assume that the second link (the one surrounding with ") means a field called "link", and the first link (the one without ") means the value of this field is a URI. Please make it clear. Adding an example could be better. > definition. Section 5.3., paragraph 2: > To address this issue, the only defined "routingcost" metric can be > ONLY "estimation". "ONLY" is not an RFC 2119 key word, doesn't have to be uppercase. Section 7., paragraph 3: > Since he This document requests the creation of the "ALTO Cost Source > Registry" with the following currently defined values: This paragraph seems to be incomplete and repeated to the next one. _______________________________________________ alto mailing list alto@ietf.org<mailto:alto@ietf.org> https://www.ietf.org/mailman/listinfo/alto