[alto] Review for draft-ietf-alto-performance-metrics-12

Qin Wu <bill.wu@huawei.com> Tue, 12 January 2021 06:22 UTC

Return-Path: <bill.wu@huawei.com>
X-Original-To: alto@ietfa.amsl.com
Delivered-To: alto@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id 373B03A10F5 for <alto@ietfa.amsl.com>; Mon, 11 Jan 2021 22:22:12 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.899
X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id N8lUA0Awx_yp for <alto@ietfa.amsl.com>; Mon, 11 Jan 2021 22:22:09 -0800 (PST)
Received: from frasgout.his.huawei.com (frasgout.his.huawei.com []) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 31D973A10F3 for <alto@ietf.org>; Mon, 11 Jan 2021 22:22:09 -0800 (PST)
Received: from fraeml702-chm.china.huawei.com (unknown []) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4DFL2Q2sDcz67ZJC for <alto@ietf.org>; Tue, 12 Jan 2021 14:17:02 +0800 (CST)
Received: from fraeml702-chm.china.huawei.com ( by fraeml702-chm.china.huawei.com ( with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2106.2; Tue, 12 Jan 2021 07:22:06 +0100
Received: from DGGEML405-HUB.china.huawei.com ( by fraeml702-chm.china.huawei.com ( with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA_P256) id 15.1.2106.2 via Frontend Transport; Tue, 12 Jan 2021 07:22:06 +0100
Received: from DGGEML531-MBS.china.huawei.com ([]) by dggeml405-hub.china.huawei.com ([]) with mapi id 14.03.0509.000; Tue, 12 Jan 2021 14:22:02 +0800
From: Qin Wu <bill.wu@huawei.com>
To: IETF ALTO <alto@ietf.org>, "Y. Richard Yang" <yry@cs.yale.edu>, Kai Gao <kaigao@scu.edu.cn>
Thread-Topic: [alto] Review for draft-ietf-alto-performance-metrics-12
Thread-Index: AdboqE4cEwTy9aCvR9OcGvHHHvVdSQ==
Date: Tue, 12 Jan 2021 06:22:01 +0000
Message-ID: <B8F9A780D330094D99AF023C5877DABAADCCC5D5@dggeml531-mbs.china.huawei.com>
Accept-Language: zh-CN, en-US
Content-Language: zh-CN
x-originating-ip: []
Content-Type: multipart/alternative; boundary="_000_B8F9A780D330094D99AF023C5877DABAADCCC5D5dggeml531mbschi_"
MIME-Version: 1.0
X-CFilter-Loop: Reflected
Archived-At: <https://mailarchive.ietf.org/arch/msg/alto/C3E4ImwAXrtFU0zaY7eIj6FppiM>
Subject: [alto] Review for draft-ietf-alto-performance-metrics-12
X-BeenThere: alto@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Application-Layer Traffic Optimization \(alto\) WG mailing list" <alto.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/alto>, <mailto:alto-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/alto/>
List-Post: <mailto:alto@ietf.org>
List-Help: <mailto:alto-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/alto>, <mailto:alto-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Jan 2021 06:22:12 -0000

Hi, Jensen:
Speak as individual, My answer to your following question is false as well, even based on RFC7285, defining hopecount as float point value seem also weird.
I think we can rely on implementation or some automation tools for constraints checking, but it is not scope of this document.
For other comments, I think Richard have addressed in v-13. Please double check it. Thanks

[alto] Review for draft-ietf-alto-performance-metrics-12

Jensen Zhang <jingxuan.n.zhang@gmail.com> Tue, 13 October 2020 04:17 UTCShow header<https://mailarchive.ietf.org/arch/msg/alto/qZrkPza-vEUcIqQMR3OJfk8G-uw/>
Dear ALTOers and authors of draft-ietf-alto-performance-metrics-12,

Below is my review for draft-ietf-alto-performance-metrics-12.

Best regards,

General issue:

The document is well written. I only have one question about the design

the base ALTO protocol only uses the cost-mode to infer the value format,
e.g., "numerical" infers the cost value MUST be a floating-point value; but
this document requires different value formats for different cost-metrics,
e.g., "delay-ow" requires the non-negative floating-point value, and
"hopcount" requires the non-negative integer value. But based on
Sec of RFC7285, in the "constraints" field, "ALTO servers SHOULD
use at least IEEE 754 double-precision floating point [IEEE.754.2008] to
store the cost value". I wonder if a test constraint expression like "eq
3.1" for the cost-metric "hopcount" is valid. Should the ALTO server reject
such a request? According to RFC7285, it should be valid. But according to
this document, it is clearly always false.


Nits and writing suggestions:

Section 1., paragraph 5:

>    The purpose of this document is to ensure proper usage of the
>    performance metrics defined in Table 1; it does not claim novelty of
>    the metrics.  The Origin column of Table 1 gives the RFC which
>    defines each metric.

  Origin -> Origin Example (to be consistent with the table)
>    We can rough classify the performance metrics into two categories:
>    those derived from the performance of individual packets (i.e., one-
>    way delay, round-trip delay, delay variation, hop count, and loss
>    rate), and those related with bandwidth (TCP throughput, residue
>    bandwidth and max reservable bandwidth).  These two categories are
>    defined in Section 3 and Section 4 respectively.  Note that all
>    metrics except round trip delay are unidirectional.  Hence, a client
>    will need to query both directions if needed.

Section 2., paragraph 1:

>    When defining the metrics in Table 1, this document considers the
>    guidelines specified in [RFC6390], which requires fine-grained
>    specification of (i) Metric Name, (ii) Metric Description, (iii)
>    Method of Measurement or Calculation, (iv) Units of Measurement, (v)
>    Measurement Points, and (vi) Measurement Timing.  In particular, for
>    each metric, this document defines (i) Metric Name, (ii) Metric
>    Description, and (iv) Units of Measurement.  The Measurement Points
>    are always specified by the specific ALTO services; for example,
>    endpoint cost service is between the two end points.

  end points -> endpoints

Section 2.1., paragraph 11:

>    A particular type of "estimation is direct "import", which indicates
>    that the value of the metric is imported directly from a specific
>    existing protocol or system.  Specifying "import" as source instead

  source -> the source
>    of the more generic "estimation" may allow better tracing of
>    information flow.  For an "import" metric, it is RECOMMENDED that the
>    "parameters" field provides details to the system from which raw data
>    is imported.  In particular, one may notice that the set of end-to-
>    end metrics defined in Table 1 has large overlap with the set defined
>    in [RFC8571], in the setting of IGP traffic engineering performance
>    metrics for each link (i.e., unidirectional link delay, min/max
>    unidirectional link delay, unidirectional delay variation,
>    unidirectional link loss, unidirectional residual bandwidth,
>    unidirectional available bandwidth, unidirectional utilized
>    bandwidth).  Hence, an ALTO server may use "import" to indicate that
>    its end-to-end metrics are computed from link metrics imported from
>    [RFC8571].

Section 2.2., paragraph 2:

>    percentile, with letter p followed by a number p:

  a number p -> a number

Section 2.2., paragraph 16:

>    If a metric has no <stat> (and hence no - as well), the metric MUST

  recommend adding " surrounding -, or using dash character instead;
  if possible, giving the precise BNF grammar will be better, as I
  see some metrics names also include the dash character ("-").
>    be considered as the 50 percentile (median).  Since this scheme is
>    common for all metrics defined in this document, below we only
>    specify the base identifier.

Section 3., paragraph 1:

>    This section introduces ALTO network performance metrics including
>    one way delay, round trip delay, delay variation, hop count, and
>    packet loss rate.  They measure the "quality of experience" of the
>    stream of packets sent from a resource provider to a resource
>    consumer.  The measures of each individual packet (pkt) can include
>    the delay from the time that the packet enters the network to the
>    time that the packet leaves the network (pkt.delay); the number of

  the time that -> the time when
>    network hops that the packet traverses (pkt.hopcount); and whether
>    the packet is dropped before reaching destination (pkt.dropped).  The

  destination -> the destination
>    semantics of the performance metrics defined in this section is that
>    they are statistics (percentiles) computed from these measures; for
>    example, the x-percentile of the one-way delay is the x-percentile of
>    the set of delays {pkt.delay} for the packets in the stream.

Section 3.1.3., paragraph 1:

>    Intended Semantics: To specify spatial and temporal aggregated delay

  spatial -> the spatial
>    of a stream of packets from the specified source and the specified
>    destination.  The spatial aggregation level is specified in the query
>    context (e.g., PID to PID, or endpoint to endpoint).

Section 3.1.4., paragraph 2:

>    "sla": Many networks provide delay in their application-level service
>    level agreements.  It is RECOMMENDED that the "parameters" field of
>    an "sla" one-way delay metric provides a link ("link") to the SLA

  I assume that the second link (the one surrounding with ") means a
  field called "link", and the first link (the one without ") means
  the value of this field is a URI. Please make it clear. Adding an
  example could be better.
>    definition.

Section 5.3., paragraph 2:

>    To address this issue, the only defined "routingcost" metric can be
>    ONLY "estimation".

  "ONLY" is not an RFC 2119 key word, doesn't have to be uppercase.

Section 7., paragraph 3:

>    Since he This document requests the creation of the "ALTO Cost Source
>    Registry" with the following currently defined values:

  This paragraph seems to be incomplete and repeated to the next one.