Re: [alto] Review for draft-ietf-alto-performance-metrics-12

Jensen Zhang <jingxuan.n.zhang@gmail.com> Tue, 12 January 2021 07:36 UTC

Return-Path: <jingxuan.n.zhang@gmail.com>
X-Original-To: alto@ietfa.amsl.com
Delivered-To: alto@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 0AD213A11A5 for <alto@ietfa.amsl.com>; Mon, 11 Jan 2021 23:36:38 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.097
X-Spam-Level:
X-Spam-Status: No, score=-2.097 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PTsG7uuhwFlX for <alto@ietfa.amsl.com>; Mon, 11 Jan 2021 23:36:35 -0800 (PST)
Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E19853A11A8 for <alto@ietf.org>; Mon, 11 Jan 2021 23:36:34 -0800 (PST)
Received: by mail-wm1-x336.google.com with SMTP id 190so986263wmz.0 for <alto@ietf.org>; Mon, 11 Jan 2021 23:36:34 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=b0QABqJHpOGBxdXRGzn4phA3ZGrk6f3MjVYdi9qGdI8=; b=fgYabC6PxgZmUxB0T5rG+beBhVt+FOHowTznqJKjjp/+lRXRQzvctUpcU+vXfCQ7Ej xefUuRUq1RkdxtH1DgPp6arOosGs0O3bp2OkFlRIq9za7g/NI7p/rUMpeDFS38U21uGL 2SBrOtYrECgMGI8fdMe60CJg2wtg6r7Cb0+J83Hk4SSdTitq8CDS3cj9ImCwhc0jH7nm KHMdYd4ldHGf5uVof2CgcGAHP+VQ6LLQko8WGuhE9cdSJwAU24FdEVLXuhs3ixVThDXe vwtdTiFDXQGsdrUk/xyhztgpYUnQJIDkYLo0eHot6nBy5uAjGdVzh0aDJQ7cMkdkDGIF uHeQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=b0QABqJHpOGBxdXRGzn4phA3ZGrk6f3MjVYdi9qGdI8=; b=Ztj5OxX02W4FAmjqFdkaXJBa1vsWMY1AWmBQI/RFMdd2OYdfX4J+bwmrfjaxnsQAoz UzFmJs30Pwe9LxLGinAuLCNiAKA7ld0pSLVbRUZQKkbrVzUZci8jXuSEuJioiC+Mcqiz 9hrGs4KN3YAw8VSaFECNEIx6cOAU+VB/WrgAA/HKahrbNMluERmBUEZUSJCmZVE2eUcm e9pIsjV2xrPQ5sLQjUiJfRRDvZJ+21y4REf4ne0zSzQdUZkK3Xw5lizTSHV0jKYQctNp F6v9eg+Bkw6Lq7Astxj/Sr5wn5Xkbx71ScXQmgpzavpurBfNp9UPqg3yZcjr9sBmao5w WCVQ==
X-Gm-Message-State: AOAM530LoyO3jjMNpVybm5hU5E5VUFUjiSf3/U275nlBmPHvwRJUcdwk X6XgdI/NHkokAKmu7MqxEqk7C/+Whph91SeD05g=
X-Google-Smtp-Source: ABdhPJz/DDN40lazpdKRJ6RMj2k3xHxmnwfWLCuFQgQ5ezS+VTZo7d9+Hv+rbJcnWPuFh/o9AQ2mHIz9CkzYfVqgK8s=
X-Received: by 2002:a1c:730f:: with SMTP id d15mr2156078wmb.135.1610436993148; Mon, 11 Jan 2021 23:36:33 -0800 (PST)
MIME-Version: 1.0
References: <B8F9A780D330094D99AF023C5877DABAADCCC5D5@dggeml531-mbs.china.huawei.com>
In-Reply-To: <B8F9A780D330094D99AF023C5877DABAADCCC5D5@dggeml531-mbs.china.huawei.com>
From: Jensen Zhang <jingxuan.n.zhang@gmail.com>
Date: Tue, 12 Jan 2021 15:36:21 +0800
Message-ID: <CAAbpuyph9D4p0AWGx9opHMahCTxy_d3O3f0=j7qjzomZtADkmw@mail.gmail.com>
To: Qin Wu <bill.wu@huawei.com>
Cc: IETF ALTO <alto@ietf.org>, "Y. Richard Yang" <yry@cs.yale.edu>, Kai Gao <kaigao@scu.edu.cn>
Content-Type: multipart/alternative; boundary="000000000000e8ead705b8af14e7"
Archived-At: <https://mailarchive.ietf.org/arch/msg/alto/BcB2I3CKjuTX-RUpJZ0-owLLeKg>
Subject: Re: [alto] Review for draft-ietf-alto-performance-metrics-12
X-BeenThere: alto@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Application-Layer Traffic Optimization \(alto\) WG mailing list" <alto.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/alto>, <mailto:alto-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/alto/>
List-Post: <mailto:alto@ietf.org>
List-Help: <mailto:alto-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/alto>, <mailto:alto-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 12 Jan 2021 07:36:38 -0000

Hi Qin,

I agree with you that the constraints checking should rely on the
implementation. I'm OK that it is not in the scope of this document.

For other comments, I have checked v-13. I think most of them have been
addressed, except for the following one.

Section 2.2., paragraph 16:



>    If a metric has no <stat> (and hence no - as well), the metric MUST



  recommend adding " surrounding -, or using dash character instead;

  if possible, giving the precise BNF grammar will be better, as I

  see some metrics names also include the dash character ("-").

>    be considered as the 50 percentile (median).  Since this scheme is

>    common for all metrics defined in this document, below we only

>    specify the base identifier.

Although I can understand this sentence, I still think it should be better
clarified.
I would suggest giving the BNF grammar at the beginning of this section,
e.g.,

   ... Hence, each performance metric's identifier
   should indicate the statistic (i.e., an aggregation operation), to
   become


      <metric-identifier> ::= <metric-base-identifier> [ '-' <stat> ]


   where <stat> MUST be one of the following:


And changing the last paragraph of the section to:

   If '-' <stat> is not present in <metric-identifier>, the metric MUST

   be considered as the 50 percentile (median).


Thanks,
Jensen


On Tue, Jan 12, 2021 at 2:22 PM Qin Wu <bill.wu@huawei.com> wrote:

> Hi, Jensen:
>
> Speak as individual, My answer to your following question is false as
> well, even based on RFC7285, defining hopecount as float point value seem
> also weird.
>
> I think we can rely on implementation or some automation tools for
> constraints checking, but it is not scope of this document.
>
> For other comments, I think Richard have addressed in v-13. Please double
> check it. Thanks
>
>
>
> -Qin
>
> [alto] Review for draft-ietf-alto-performance-metrics-12
>
> Jensen Zhang <jingxuan.n.zhang@gmail.com> Tue, 13 October 2020 04:17 UTCShow
> header
> <https://mailarchive.ietf.org/arch/msg/alto/qZrkPza-vEUcIqQMR3OJfk8G-uw/>
>
> Dear ALTOers and authors of draft-ietf-alto-performance-metrics-12,
>
>
>
> Below is my review for draft-ietf-alto-performance-metrics-12.
>
>
>
> Best regards,
>
> Jensen
>
>
>
> ==============================================
>
> General issue:
>
>
>
> The document is well written. I only have one question about the design
>
> part:
>
>
>
> the base ALTO protocol only uses the cost-mode to infer the value format,
>
> e.g., "numerical" infers the cost value MUST be a floating-point value; but
>
> this document requires different value formats for different cost-metrics,
>
> e.g., "delay-ow" requires the non-negative floating-point value, and
>
> "hopcount" requires the non-negative integer value. But based on
>
> Sec 11.3.2.3 of RFC7285, in the "constraints" field, "ALTO servers SHOULD
>
> use at least IEEE 754 double-precision floating point [IEEE.754.2008] to
>
> store the cost value". I wonder if a test constraint expression like "eq
>
> 3.1" for the cost-metric "hopcount" is valid. Should the ALTO server reject
>
> such a request? According to RFC7285, it should be valid. But according to
>
> this document, it is clearly always false.
>
>
>
> ==============================================
>
>
>
> Nits and writing suggestions:
>
>
>
> Section 1., paragraph 5:
>
>
>
> >    The purpose of this document is to ensure proper usage of the
>
> >    performance metrics defined in Table 1; it does not claim novelty of
>
> >    the metrics.  The Origin column of Table 1 gives the RFC which
>
> >    defines each metric.
>
>
>
>   Origin -> Origin Example (to be consistent with the table)
>
> >    We can rough classify the performance metrics into two categories:
>
> >    those derived from the performance of individual packets (i.e., one-
>
> >    way delay, round-trip delay, delay variation, hop count, and loss
>
> >    rate), and those related with bandwidth (TCP throughput, residue
>
> >    bandwidth and max reservable bandwidth).  These two categories are
>
> >    defined in Section 3 and Section 4 respectively.  Note that all
>
> >    metrics except round trip delay are unidirectional.  Hence, a client
>
> >    will need to query both directions if needed.
>
>
>
>
>
> Section 2., paragraph 1:
>
>
>
> >    When defining the metrics in Table 1, this document considers the
>
> >    guidelines specified in [RFC6390], which requires fine-grained
>
> >    specification of (i) Metric Name, (ii) Metric Description, (iii)
>
> >    Method of Measurement or Calculation, (iv) Units of Measurement, (v)
>
> >    Measurement Points, and (vi) Measurement Timing.  In particular, for
>
> >    each metric, this document defines (i) Metric Name, (ii) Metric
>
> >    Description, and (iv) Units of Measurement.  The Measurement Points
>
> >    are always specified by the specific ALTO services; for example,
>
> >    endpoint cost service is between the two end points.
>
>
>
>   end points -> endpoints
>
>
>
>
>
> Section 2.1., paragraph 11:
>
>
>
> >    A particular type of "estimation is direct "import", which indicates
>
> >    that the value of the metric is imported directly from a specific
>
> >    existing protocol or system.  Specifying "import" as source instead
>
>
>
>   source -> the source
>
> >    of the more generic "estimation" may allow better tracing of
>
> >    information flow.  For an "import" metric, it is RECOMMENDED that the
>
> >    "parameters" field provides details to the system from which raw data
>
> >    is imported.  In particular, one may notice that the set of end-to-
>
> >    end metrics defined in Table 1 has large overlap with the set defined
>
> >    in [RFC8571], in the setting of IGP traffic engineering performance
>
> >    metrics for each link (i.e., unidirectional link delay, min/max
>
> >    unidirectional link delay, unidirectional delay variation,
>
> >    unidirectional link loss, unidirectional residual bandwidth,
>
> >    unidirectional available bandwidth, unidirectional utilized
>
> >    bandwidth).  Hence, an ALTO server may use "import" to indicate that
>
> >    its end-to-end metrics are computed from link metrics imported from
>
> >    [RFC8571].
>
>
>
>
>
> Section 2.2., paragraph 2:
>
>
>
> >    percentile, with letter p followed by a number p:
>
>
>
>   a number p -> a number
>
>
>
>
>
> Section 2.2., paragraph 16:
>
>
>
> >    If a metric has no <stat> (and hence no - as well), the metric MUST
>
>
>
>   recommend adding " surrounding -, or using dash character instead;
>
>   if possible, giving the precise BNF grammar will be better, as I
>
>   see some metrics names also include the dash character ("-").
>
> >    be considered as the 50 percentile (median).  Since this scheme is
>
> >    common for all metrics defined in this document, below we only
>
> >    specify the base identifier.
>
>
>
>
>
> Section 3., paragraph 1:
>
>
>
> >    This section introduces ALTO network performance metrics including
>
> >    one way delay, round trip delay, delay variation, hop count, and
>
> >    packet loss rate.  They measure the "quality of experience" of the
>
> >    stream of packets sent from a resource provider to a resource
>
> >    consumer.  The measures of each individual packet (pkt) can include
>
> >    the delay from the time that the packet enters the network to the
>
> >    time that the packet leaves the network (pkt.delay); the number of
>
>
>
>   the time that -> the time when
>
> >    network hops that the packet traverses (pkt.hopcount); and whether
>
> >    the packet is dropped before reaching destination (pkt.dropped).  The
>
>
>
>   destination -> the destination
>
> >    semantics of the performance metrics defined in this section is that
>
> >    they are statistics (percentiles) computed from these measures; for
>
> >    example, the x-percentile of the one-way delay is the x-percentile of
>
> >    the set of delays {pkt.delay} for the packets in the stream.
>
>
>
>
>
> Section 3.1.3., paragraph 1:
>
>
>
> >    Intended Semantics: To specify spatial and temporal aggregated delay
>
>
>
>   spatial -> the spatial
>
> >    of a stream of packets from the specified source and the specified
>
> >    destination.  The spatial aggregation level is specified in the query
>
> >    context (e.g., PID to PID, or endpoint to endpoint).
>
>
>
>
>
> Section 3.1.4., paragraph 2:
>
>
>
> >    "sla": Many networks provide delay in their application-level service
>
> >    level agreements.  It is RECOMMENDED that the "parameters" field of
>
> >    an "sla" one-way delay metric provides a link ("link") to the SLA
>
>
>
>   I assume that the second link (the one surrounding with ") means a
>
>   field called "link", and the first link (the one without ") means
>
>   the value of this field is a URI. Please make it clear. Adding an
>
>   example could be better.
>
> >    definition.
>
>
>
>
>
> Section 5.3., paragraph 2:
>
>
>
> >    To address this issue, the only defined "routingcost" metric can be
>
> >    ONLY "estimation".
>
>
>
>   "ONLY" is not an RFC 2119 key word, doesn't have to be uppercase.
>
>
>
>
>
> Section 7., paragraph 3:
>
>
>
> >    Since he This document requests the creation of the "ALTO Cost Source
>
> >    Registry" with the following currently defined values:
>
>
>
>   This paragraph seems to be incomplete and repeated to the next one.
> _______________________________________________
> alto mailing list
> alto@ietf.org
> https://www.ietf.org/mailman/listinfo/alto
>