Re: [alto] Review for draft-ietf-alto-performance-metrics-12
"Y. Richard Yang" <yry@cs.yale.edu> Wed, 13 January 2021 23:11 UTC
Return-Path: <yang.r.yang@gmail.com>
X-Original-To: alto@ietfa.amsl.com
Delivered-To: alto@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 02C873A14C2 for <alto@ietfa.amsl.com>; Wed, 13 Jan 2021 15:11:51 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.401
X-Spam-Level:
X-Spam-Status: No, score=-1.401 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.248, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TLKy_lshCner for <alto@ietfa.amsl.com>; Wed, 13 Jan 2021 15:11:48 -0800 (PST)
Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com [209.85.167.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B81A03A14BE for <alto@ietf.org>; Wed, 13 Jan 2021 15:11:47 -0800 (PST)
Received: by mail-lf1-f45.google.com with SMTP id m25so5241557lfc.11 for <alto@ietf.org>; Wed, 13 Jan 2021 15:11:47 -0800 (PST)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=1OBFJEhjYdZHHAsvURAuj0fVV0NlcYYpzQLOG7oZtNM=; b=YxSDTHONzp5D/KEymFNfT5FLK50J2+mEp145s/cGW3hZaJJI3phbMrq+ud2NdIRxUe cvbpQ7iYYCJ9LrKw70aHhOFa2cYtibKUhHt4tgu1b31vohHdHfPsKW+2GzRHj5mkQpaH dp8RBVQ7z7ogZUf7ToboMQaV4m1j1yR9q3zG5kPlCSRGEUOgXzTmBMu5A5vUQsAKwpj8 4ZmNbT4GkrDXTppOEhjNunTgE8dS4w44n/QV0+jJexi3Whfpt1XIGWxAdaidZYcn5/g2 hx7j3nU0CkOshbWvgQ+70OMt3I0+jBgI7Q2jrzV7puOiyMrUzsBpL+I5rFGGP6DU4+52 rR1Q==
X-Gm-Message-State: AOAM530qmnoNm+Yv3c7HrEVmdlyrKzW8G/gcl6zpKYzf7EaMqDvon/su Wz/cMdX8Lw4pcp+qumr1xCHIKxoWEO3QYVcA6+fkS5fMpCY=
X-Google-Smtp-Source: ABdhPJzXtT9Q0pDflCbFWx7Gh49dqiV+yLfAjprj1C1dFSKNMAIIRIEA0FWzXBIr471QDV7Fi9QXSLcDn1xGsSrqo2g=
X-Received: by 2002:a19:3f87:: with SMTP id m129mr1854663lfa.560.1610579505776; Wed, 13 Jan 2021 15:11:45 -0800 (PST)
MIME-Version: 1.0
References: <B8F9A780D330094D99AF023C5877DABAADCCD890@dggeml531-mbs.china.huawei.com>
In-Reply-To: <B8F9A780D330094D99AF023C5877DABAADCCD890@dggeml531-mbs.china.huawei.com>
From: "Y. Richard Yang" <yry@cs.yale.edu>
Date: Wed, 13 Jan 2021 18:11:34 -0500
Message-ID: <CANUuoLpQCk5cwfOfDT5z6QN0Y5ktb7HtGtgBEm=YuHSaG5pBsw@mail.gmail.com>
To: Qin Wu <bill.wu@huawei.com>
Cc: Jensen Zhang <jingxuan.n.zhang@gmail.com>, IETF ALTO <alto@ietf.org>, Kai Gao <kaigao@scu.edu.cn>
Content-Type: multipart/alternative; boundary="000000000000530e7c05b8d043d7"
Archived-At: <https://mailarchive.ietf.org/arch/msg/alto/qCz5749U3cc_2Aky8gAa29WbkN0>
Subject: Re: [alto] Review for draft-ietf-alto-performance-metrics-12
X-BeenThere: alto@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Application-Layer Traffic Optimization \(alto\) WG mailing list" <alto.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/alto>, <mailto:alto-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/alto/>
List-Post: <mailto:alto@ietf.org>
List-Help: <mailto:alto-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/alto>, <mailto:alto-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 13 Jan 2021 23:11:51 -0000
Hi Jensen, Qin, I thought a bit more on the suggestion by Jensen and agree with both of you that it is OK. I have updated the document in -14. Thanks a lot! Richard On Tue, Jan 12, 2021 at 4:37 AM Qin Wu <bill.wu@huawei.com> wrote: > Jensen: > > My impression, <stat> is an optional item in the ANBF, therefore the > metric identifier can be used without <stat> as suffix. > > I think your proposed change makes sense to me. > > > > -Qin > > *发件人:* Jensen Zhang [mailto:jingxuan.n.zhang@gmail.com] > *发送时间:* 2021年1月12日 15:36 > *收件人:* Qin Wu <bill.wu@huawei.com> > *抄送:* IETF ALTO <alto@ietf.org>; Y. Richard Yang <yry@cs.yale.edu>; Kai > Gao <kaigao@scu.edu.cn> > *主题:* Re: [alto] Review for draft-ietf-alto-performance-metrics-12 > > > > Hi Qin, > > > > I agree with you that the constraints checking should rely on the > implementation. I'm OK that it is not in the scope of this document. > > > > For other comments, I have checked v-13. I think most of them have been > addressed, except for the following one. > > > > Section 2.2., paragraph 16: > > > > > If a metric has no <stat> (and hence no - as well), the metric MUST > > > > recommend adding " surrounding -, or using dash character instead; > > if possible, giving the precise BNF grammar will be better, as I > > see some metrics names also include the dash character ("-"). > > > be considered as the 50 percentile (median). Since this scheme is > > > common for all metrics defined in this document, below we only > > > specify the base identifier. > > > > Although I can understand this sentence, I still think it should be better > clarified. > > I would suggest giving the BNF grammar at the beginning of this section, > e.g., > > > > ... Hence, each performance metric's identifier > > should indicate the statistic (i.e., an aggregation operation), to > > become > > > > <metric-identifier> ::= <metric-base-identifier> [ '-' <stat> ] > > > > where <stat> MUST be one of the following: > > > > And changing the last paragraph of the section to: > > > > If '-' <stat> is not present in <metric-identifier>, the metric MUST > > be considered as the 50 percentile (median). > > > > Thanks, > > Jensen > > > > > > On Tue, Jan 12, 2021 at 2:22 PM Qin Wu <bill.wu@huawei.com> wrote: > > Hi, Jensen: > > Speak as individual, My answer to your following question is false as > well, even based on RFC7285, defining hopecount as float point value seem > also weird. > > I think we can rely on implementation or some automation tools for > constraints checking, but it is not scope of this document. > > For other comments, I think Richard have addressed in v-13. Please double > check it. Thanks > > > > -Qin > > [alto] Review for draft-ietf-alto-performance-metrics-12 > > Jensen Zhang <jingxuan.n.zhang@gmail.com> Tue, 13 October 2020 04:17 UTCShow > header > <https://mailarchive.ietf.org/arch/msg/alto/qZrkPza-vEUcIqQMR3OJfk8G-uw/> > > Dear ALTOers and authors of draft-ietf-alto-performance-metrics-12, > > > > Below is my review for draft-ietf-alto-performance-metrics-12. > > > > Best regards, > > Jensen > > > > ============================================== > > General issue: > > > > The document is well written. I only have one question about the design > > part: > > > > the base ALTO protocol only uses the cost-mode to infer the value format, > > e.g., "numerical" infers the cost value MUST be a floating-point value; but > > this document requires different value formats for different cost-metrics, > > e.g., "delay-ow" requires the non-negative floating-point value, and > > "hopcount" requires the non-negative integer value. But based on > > Sec 11.3.2.3 of RFC7285, in the "constraints" field, "ALTO servers SHOULD > > use at least IEEE 754 double-precision floating point [IEEE.754.2008] to > > store the cost value". I wonder if a test constraint expression like "eq > > 3.1" for the cost-metric "hopcount" is valid. Should the ALTO server reject > > such a request? According to RFC7285, it should be valid. But according to > > this document, it is clearly always false. > > > > ============================================== > > > > Nits and writing suggestions: > > > > Section 1., paragraph 5: > > > > > The purpose of this document is to ensure proper usage of the > > > performance metrics defined in Table 1; it does not claim novelty of > > > the metrics. The Origin column of Table 1 gives the RFC which > > > defines each metric. > > > > Origin -> Origin Example (to be consistent with the table) > > > We can rough classify the performance metrics into two categories: > > > those derived from the performance of individual packets (i.e., one- > > > way delay, round-trip delay, delay variation, hop count, and loss > > > rate), and those related with bandwidth (TCP throughput, residue > > > bandwidth and max reservable bandwidth). These two categories are > > > defined in Section 3 and Section 4 respectively. Note that all > > > metrics except round trip delay are unidirectional. Hence, a client > > > will need to query both directions if needed. > > > > > > Section 2., paragraph 1: > > > > > When defining the metrics in Table 1, this document considers the > > > guidelines specified in [RFC6390], which requires fine-grained > > > specification of (i) Metric Name, (ii) Metric Description, (iii) > > > Method of Measurement or Calculation, (iv) Units of Measurement, (v) > > > Measurement Points, and (vi) Measurement Timing. In particular, for > > > each metric, this document defines (i) Metric Name, (ii) Metric > > > Description, and (iv) Units of Measurement. The Measurement Points > > > are always specified by the specific ALTO services; for example, > > > endpoint cost service is between the two end points. > > > > end points -> endpoints > > > > > > Section 2.1., paragraph 11: > > > > > A particular type of "estimation is direct "import", which indicates > > > that the value of the metric is imported directly from a specific > > > existing protocol or system. Specifying "import" as source instead > > > > source -> the source > > > of the more generic "estimation" may allow better tracing of > > > information flow. For an "import" metric, it is RECOMMENDED that the > > > "parameters" field provides details to the system from which raw data > > > is imported. In particular, one may notice that the set of end-to- > > > end metrics defined in Table 1 has large overlap with the set defined > > > in [RFC8571], in the setting of IGP traffic engineering performance > > > metrics for each link (i.e., unidirectional link delay, min/max > > > unidirectional link delay, unidirectional delay variation, > > > unidirectional link loss, unidirectional residual bandwidth, > > > unidirectional available bandwidth, unidirectional utilized > > > bandwidth). Hence, an ALTO server may use "import" to indicate that > > > its end-to-end metrics are computed from link metrics imported from > > > [RFC8571]. > > > > > > Section 2.2., paragraph 2: > > > > > percentile, with letter p followed by a number p: > > > > a number p -> a number > > > > > > Section 2.2., paragraph 16: > > > > > If a metric has no <stat> (and hence no - as well), the metric MUST > > > > recommend adding " surrounding -, or using dash character instead; > > if possible, giving the precise BNF grammar will be better, as I > > see some metrics names also include the dash character ("-"). > > > be considered as the 50 percentile (median). Since this scheme is > > > common for all metrics defined in this document, below we only > > > specify the base identifier. > > > > > > Section 3., paragraph 1: > > > > > This section introduces ALTO network performance metrics including > > > one way delay, round trip delay, delay variation, hop count, and > > > packet loss rate. They measure the "quality of experience" of the > > > stream of packets sent from a resource provider to a resource > > > consumer. The measures of each individual packet (pkt) can include > > > the delay from the time that the packet enters the network to the > > > time that the packet leaves the network (pkt.delay); the number of > > > > the time that -> the time when > > > network hops that the packet traverses (pkt.hopcount); and whether > > > the packet is dropped before reaching destination (pkt.dropped). The > > > > destination -> the destination > > > semantics of the performance metrics defined in this section is that > > > they are statistics (percentiles) computed from these measures; for > > > example, the x-percentile of the one-way delay is the x-percentile of > > > the set of delays {pkt.delay} for the packets in the stream. > > > > > > Section 3.1.3., paragraph 1: > > > > > Intended Semantics: To specify spatial and temporal aggregated delay > > > > spatial -> the spatial > > > of a stream of packets from the specified source and the specified > > > destination. The spatial aggregation level is specified in the query > > > context (e.g., PID to PID, or endpoint to endpoint). > > > > > > Section 3.1.4., paragraph 2: > > > > > "sla": Many networks provide delay in their application-level service > > > level agreements. It is RECOMMENDED that the "parameters" field of > > > an "sla" one-way delay metric provides a link ("link") to the SLA > > > > I assume that the second link (the one surrounding with ") means a > > field called "link", and the first link (the one without ") means > > the value of this field is a URI. Please make it clear. Adding an > > example could be better. > > > definition. > > > > > > Section 5.3., paragraph 2: > > > > > To address this issue, the only defined "routingcost" metric can be > > > ONLY "estimation". > > > > "ONLY" is not an RFC 2119 key word, doesn't have to be uppercase. > > > > > > Section 7., paragraph 3: > > > > > Since he This document requests the creation of the "ALTO Cost Source > > > Registry" with the following currently defined values: > > > > This paragraph seems to be incomplete and repeated to the next one. > > _______________________________________________ > alto mailing list > alto@ietf.org > https://www.ietf.org/mailman/listinfo/alto > >