From nobody Fri Apr 2 18:20:39 2021 Return-Path: X-Original-To: alto@ietfa.amsl.com Delivered-To: alto@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DA2673A1336 for ; Fri, 2 Apr 2021 18:20:37 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.398 X-Spam-Level: X-Spam-Status: No, score=-1.398 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, FREEMAIL_FORGED_FROMDOMAIN=0.25, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.249, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_BLOCKED=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id fvlSZUVvTmdL for ; Fri, 2 Apr 2021 18:20:33 -0700 (PDT) Received: from mail-lf1-f46.google.com (mail-lf1-f46.google.com [209.85.167.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 022663A1335 for ; Fri, 2 Apr 2021 18:20:32 -0700 (PDT) Received: by mail-lf1-f46.google.com with SMTP id m12so9637736lfq.10 for ; Fri, 02 Apr 2021 18:20:32 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=v7JdyCj1XCGC7NEUTVbYm/hhnIe3jEyoZzFf0TM2x1c=; b=djT6HcDGJD5pX8yOc+FXLJ0fjdRe0GF0JFJrbot4BBLCmhkc/anbGe0Z5uxpk+Y0jb GahCABikfYYeAAQGvPQNe4+MbLAagz6fWyld6blomGWixi93vpeTo1f+5rxaQkgqMqdi 5O2GYSz21fq2x/MutCyjYBfkOEQ5gXwWAShnNIxlyZoDyaluxmo6kW0nV7LDNQOAmJpu o15KTawytPTr9bAZKx6NuyjrpfdgqD2G5TN+ecA7BwoU0hg06syfCKFMjGy9MAHCW+Xu U7x8hgV0XM9A2a8znurUebCEgOp3cOcbXQNPZAbV8c1Q0sx8ayqUOlfrCDYWtNythuTi szdA== X-Gm-Message-State: AOAM531T2P2ZHL9/wkr0zXC4TUvWvfAXBbR3weQ0Dlyqw6eQ5MFajEM+ XJyzS36+MYvdDwKEORuU12QZytAi5pY1VkmGd+w= X-Google-Smtp-Source: ABdhPJyxflG5+sGUyvl26sZQeVpfu5S42nCBdzNGzCemG9v+uUL06pWcIEaTnpL+pf7supK0MjuaDJogmjn9BcP41c4= X-Received: by 2002:a05:6512:70:: with SMTP id i16mr10469130lfo.508.1617412831093; Fri, 02 Apr 2021 18:20:31 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: "Y. Richard Yang" Date: Fri, 2 Apr 2021 21:20:20 -0400 Message-ID: To: Martin Duke Cc: IETF ALTO , Brian Trammell Content-Type: multipart/alternative; boundary="00000000000040a90605bf0745b9" Archived-At: Subject: Re: [alto] AD review of draft-ietf-alto-performance-metrics-15 X-BeenThere: alto@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Application-Layer Traffic Optimization \(alto\) WG mailing list" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Apr 2021 01:20:38 -0000 --00000000000040a90605bf0745b9 Content-Type: text/plain; charset="UTF-8" Hi Martin, On Tue, Mar 30, 2021 at 2:24 PM Martin Duke wrote: > Sorry for the fragmented review, but there are a couple of more issues: > > - The authors should do a review of all lower-case occurrences of must, > should, may, and recommended. At least a few of them should be capitalized > to indicate normative requirements. > Yes. We are taking a pass and are almost done. > > - IMO, from a quick review, I-D.ietf-ippm-initial-registry as written is > normative and should be listed as such. However, I think it would be better > to simply refer to the actual registry ( > https://www.iana.org/assignments/performance-metrics/performance-metrics.xhtml) > rather than tie it to the initial entries. > > Agree that referring to the registry is a better design and we will do the normative reference in the update. Thanks! Richard > On Mon, Mar 29, 2021 at 5:30 PM Martin Duke > wrote: > >> One small correction: I'm jumping the gun on the author policy; 6 is >> probably OK for now. >> >> On Mon, Mar 29, 2021 at 11:33 AM Martin Duke >> wrote: >> >>> Hello authors, >>> >>> Thank you very much for writing this draft. It is clearly a useful >>> extension to ALTO and is quite clearly written, even to someone who is not >>> a practitioner. I have numerous comments/questions and a few nits. >>> >>> These points are all invitations to discussion, rather than commands >>> about what to change, as I've missed much of the WG deliberations that led >>> to this text. >>> >>> COMMENTS: >>> - There are six authors. Having more than 5 editors/authors listed on >>> the front page requires strong justification and chair/AD approval. See the >>> RFC Editor statement [1]. You are encouraged to designate a few editors for >>> the front page and list as many authors as desired at the end. >>> >>> - Sec 2.1. The cost-source model is conceptually sound, but the >>> justification for it seems underexplained. What exactly is a client going >>> to do with this information? What different behaviors would a client >>> execute if the context was e.g. "sla" instead of "nominal?" To the extent >>> the parameters are not machine readable, like links to webpages, are we >>> really expecting this information to be presented to the humans behind ALTO >>> clients? >>> >>> - Sec 2.1 I am confused about the meaning of the "sla" cost-source. Does >>> this refer to an SLA the ALTO client has with the network? Between the >>> target IP and the network? Or something else? If the first, does this link >>> to client authentication in some way? If the second, what are the privacy >>> implications of exposing these SLAs? >>> >>> - Sec 2.1. Related to the above, the text suggests that any cost-source >>> expressed as "import" could also be expressed as "estimation". Why would >>> the server do this? The text should say, or perhaps it would be >>> conceptually cleaner if "estimation" and "import" were mutually exclusive >>> sources by definition. >>> >>> - Sec 3. I would prefer it if the parameters field in each of these >>> definitions was a bit more strict. This relates to my confusion about >>> machine-readable vs. human readable data; if this is meant to be >>> machine-readable, then e.g. Sec 3.4.4 should be more specific in spelling >>> out that the IGP protocols should be in a format with the RFC number, for >>> instance. If it's to be human readable for a purpose I don't understand, >>> then these looser definitions are probably OK. >>> >>> - Sec 3.4 Unlike the other metrics, I have no idea what a client is to >>> do with the hop count metric, since clients don't care about hop count. >>> Hops only affect users through delay and loss rate, which is present in >>> other metrics. Is the intent here to provide a proxy for delay when direct >>> delay information is not available? If so, we should say so. >>> >>> - Sec 5.3. I suggest a reword. >>> >>> OLD: >>> To address this issue, the only defined "routingcost" metric can be >>> only "estimation". >>> >>> NEW: >>> To address this issue, if the "routingcost" metric contains a >>> cost-context field, it MUST be "estimation." >>> >>> What should clients do if it's not "estimation?" Can they use it or >>> reject the metric >>> as malformed? >>> >>> - Sec 5.4.1: "...the ALTO server may provide the client with the >>> validity period of the exposed metric values." >>> >>> Shouldn't there be a standard format for this? Or are you implying the >>> use of cost-calendar? >>> >>> - Sec 5.4.2: I don't understand what this section is saying. Can the >>> server provide new metrics not in the spec? Or is it saying that the server >>> can take singletons about link one-way delays and compose path one-way and >>> two-way delays, for example? >>> >>> NITS: >>> - Sec 1. An initial sentence introducing ALTO at the beginning would be >>> helpful, e.g. >>> >>> "ALTO [RFC 7285] provides a means for client to identify the most >>> efficient information source when multiple copies of such information >>> exist, by quering path information from an HTTP server." >>> >>> - Sec 2. The second paragraph is a little hard to read. Suggestion: >>> >>> OLD: >>> >>> On the other hand, to be able to use coarse-grained information such >>> as routing system information (e.g., [RFC8571 ]), which may not >>> provide fine-grained information such as (iii) Method of Measurement >>> or Calculation and (vi) Measurement Timing, this document provides >>> context information to indicate the source of information and hence >>> available metric details. >>> >>> NEW: >>> >>> This document specifies context information to indicate the metric >>> source, which can allow clients to obtain fine-grained information like >>> (ii) Method of Measurement or Calculation and (vi) Measurement Timing." >>> >>> - Sec 2.1 In Fig. 1, please expand "NBI" on first use. >>> >>> - Sec 3.1.3 Expand "PID" on first use. >>> >>> - Sec 3.1.4 s/recommended/RECOMMENDED >>> >>> - Sec 3.4 s/metric hopcount/hopcount metric >>> >>> - Sec 4.1.3 would this be correct: s/give the throughput/give the >>> maximum throughput >>> >>> - Sec 6. s/is a highly sensitive/is highly sensitive >>> >>> Thanks >>> Martin >>> >>> [1] >>> https://www.rfc-editor.org/pipermail/rfc-interest/2015-May/008869.html >>> >> _______________________________________________ > alto mailing list > alto@ietf.org > https://www.ietf.org/mailman/listinfo/alto --00000000000040a90605bf0745b9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Martin,

On Tue, Mar 30, 2021 at 2:24 PM= Martin Duke <martin.h.duke@g= mail.com> wrote:
Sorry for the fragmented review, but there ar= e a couple of more issues:

- The authors should do= a review of all lower-case occurrences of must, should, may, and recommend= ed. At least a few of them should be capitalized to indicate normative requ= irements.

Yes. We are taking a = pass and are almost=C2=A0done.

- IMO, from a quick review,= =C2=A0 I-D.ietf-ippm-initial-registry as written is normative and should be listed= as such. However, I think it would be better to simply refer to the actual= registry (https://www.iana.org/assignmen= ts/performance-metrics/performance-metrics.xhtml) rather than tie it to= the initial entries.

Agree= that referring to the registry is a better design and we will do the norma= tive reference in the update.

Thanks!
Ri= chard
=C2=A0
=C2=A0
On Mon, Mar 29, 2021 at 5:30 PM Martin Duke <martin.h.duke@gmail.com>= wrote:
One small correction: I'm jumping the gun on the author policy= ; 6 is probably OK for now.

On Mon, Mar 29, 2021 at 11:33 AM Martin Duke= <martin.h.= duke@gmail.com> wrote:
Hello authors,

Thank you very much for writing this draft. It is clearly a useful extensi= on to ALTO and is quite clearly written, even to someone who is not a pract= itioner. I have numerous comments/questions and a few nits.

<= /div>
These points are all invitations to discussion, rather than comma= nds about what to change, as I've missed much of the WG deliberations t= hat led to this text.

COMMENTS:
- Th= ere are six authors. Having more than 5 editors/authors listed on the front= page requires strong justification and chair/AD approval. See the RFC Edit= or statement [1]. You are encouraged to designate a few editors for the fro= nt page and list as many authors as desired at the end.

- Sec 2.1. The cost-source model is conceptually sound, but the justi= fication for it seems underexplained. What exactly is a client going to do = with this information? What different behaviors would a client execute if t= he context was e.g. "sla" instead of "nominal?" To the = extent the parameters are not machine readable, like links to webpages, are= we really expecting this information to be presented to the humans behind = ALTO clients?

- Sec 2.1 I am confused about the me= aning of the "sla" cost-source. Does this refer to an SLA the ALT= O client has with the network? Between the target IP and the network? Or so= mething else? If the first, does this link to client authentication in some= way? If the second, what are the privacy implications of exposing these SL= As?

- Sec 2.1. Related to the above, the text sugg= ests that any cost-source expressed as "import" could also be exp= ressed as "estimation". Why would the server do this? The text sh= ould say, or perhaps it would be conceptually cleaner if "estimation&q= uot; and "import" were mutually exclusive sources by definition.<= /div>

- Sec 3. I would prefer it if the parameters field= in each of these definitions was a bit more strict. This relates to my con= fusion about machine-readable vs. human readable data; if this is meant to = be machine-readable, then e.g. Sec 3.4.4 should be more specific in spellin= g out that the IGP protocols should be in a format with the RFC number, for= instance. If it's to be human readable for a purpose I don't under= stand, then these looser definitions are probably OK.

- Sec 3.4 Unlike the other metrics, I have no idea what a client is= to do with the hop count metric, since clients don't care about hop co= unt. Hops only affect users through delay and loss rate, which is present i= n other metrics. Is the intent here to provide a proxy for delay when direc= t delay information is not available? If so, we should say so.
- Sec 5.3. I suggest a reword.

OL= D:
To address this issue, the only defined "routingcost"= ; metric can be
=C2=A0 =C2=A0only "estimation".

=
NEW:
To address this issue, if the "routingcost&q= uot; metric contains a cost-context field, it MUST be "estimation.&quo= t;

What should clients do if it's not "es= timation?" Can they use it or reject the metric
as malformed= ?

- Sec 5.4.1: "...the ALTO server may provid= e the client with the validity period of the exposed metric values."

Shouldn't there be a standard format for this? = Or are you implying the use of cost-calendar?

- Se= c 5.4.2: I don't understand what this section is saying. Can the server= provide new metrics not in the spec? Or is it saying that the server can t= ake singletons about link one-way delays and compose path one-way and two-w= ay delays, for example?

NITS:
- Sec = 1. An initial sentence introducing ALTO at the beginning would be helpful, = e.g.

"ALTO [RFC 7285] provides a means for cl= ient to identify the most efficient information source when multiple copies= of such information exist, by quering path information from an HTTP server= ."

- Sec 2. The second paragraph is a little = hard to read. Suggestion:

OLD:
On the other hand, to be =
able to use coarse-grained information such
   as routing system information (e.g., [RFC8571]), which may not
   provide fine-grained information such as (iii) Method of Measurement
   or Calculation and (vi) Measurement Timing, this document provides
   context information to indicate the source of information and hence
   available metric details.
NEW:
  This document specifies context information to indicate t=
he metric
source, which can allow clients to obtain fine-grained informa=
tion like
(ii) Method of Measurement or Calculation and (vi) Measurement=
 Timing."
- Sec 2.1 In Fig. 1, please expand "NBI" on first use.=

- Sec 3.1.3 Expand "PID" on first use.<= /div>

- Sec 3.1.4 s/recommended/RECOMMENDED
- Sec 3.4 s/metric hopcount/hopcount metric

- Sec 4.1.3 would this be correct: s/give the throughput/give the ma= ximum throughput

- Sec 6. s/is a highly sensitive/= is highly sensitive

Thanks
Martin

_______________________________________________
alto mailing list
alto@ietf.org
https://www.ietf.org/mailman/listinfo/alto
--00000000000040a90605bf0745b9--