Re: [auth48] AUTH48: RFC-to-be 9439 <draft-ietf-alto-performance-metrics-28> for your review

[Dhruv Dhody <[e-mail]>]

Hi Lynne,

Thanks for editing. The diff looks good to me. Two changes

(1) Can you update my address as

OLD

   Dhruv Dhody
   Huawei
   Leela Palace
   Bangalore 560008
   Karnataka
   India
   Email: [e-mail]

NEW

   Dhruv Dhody
   Huawei
   India
   Email: [e-mail]

END

(2) I find the number of figures to be a bit weird. Figure
2,3,4,6,7,8,9,10 are shown in the middle, whereas Figure 5 at the
bottom. IMO the bottom is the right thing to do. BTW in the author's
version we did not have figure numbering.


With that, please note my approval.


Thanks!

Dhruv



On Fri, Jul 7, 2023 at 2:17 AM Lynne Bartholomew <[e-mail]>
wrote:

> Hi, Qin and Richard.
>
> Qin, thank you for your replies.  We have some follow-up items for you:
>
> Regarding this question and your reply:
>
> >> 3) <!-- [rfced] Sections 1 and 3.1:  Are the quotes around the words
> "where" and "how" necessary?
> >>
> >> Original:
> >> This document registers a set of new cost metrics (Table 1) to allow
> applications to determine "where" to connect based on network  performance
> criteria including delay and bandwidth related metrics.
> >> ...
> >> The core additional details of a performance metric specify "how" the
> metric is obtained. -->
> >
> >
> > [Qin Wu] I think the quotes can be deleted. What it does is just to put
> emphasize on the key words.
>
> We have left the quotes in place for now.  We'd like to point out that one
> available option in xml2rfc v3 is to replace the quotes with "<em>"; per
> Section 2.2 of RFC 7991, <em> indicates "text that is semantically
> emphasized.  Text enclosed within this element will be displayed as italic
> after processing."
>
> Another option is to use <strong>; per Section 2.50 of RFC 7991, <strong>
> indicates "text that is semantically strong.  Text enclosed within this
> element will be displayed as bold after processing."
>
> Would you like to implement either of these options?  Please note that
> <em> would italicize the text in the HTML and PDF output and yield
> underscores in the TXT output, and <strong> would boldface the text in the
> HTML and PDF output and yield asterisks in the TXT output.  Please advise;
> if you'd prefer not to use <em> or <strong>, we will then remove the quotes.
>
> = = = = =
>
> Regarding this question and your reply:
>
> >> 5) <!-- [rfced] Section 1:  This sentence is difficult to interpret.
> >> Please clarify "... metrics, to expose to application-layer traffic
> optimization, can be a typical mechanism by network operators".
> >>
> >> Original:
> >> Deriving ALTO cost
> >> performance metrics from existing network-layer traffic engineering
> performance metrics, to expose to application-layer traffic  optimization,
> can be a typical mechanism by network operators to  deploy ALTO [RFC7971],
> [FlowDirector]. -->
> > [Qin Wu] Here is the proposed change:
> > OLD TEXT:
> > "
> > Deriving ALTO cost performance metrics from existing network-layer
> traffic engineering performance metrics, and making it exposed to
> application-layer traffic optimization, can be a typical mechanism by
> network operators to deploy ALTO [RFC7971], [FlowDirector].
> > "
>
>
> Does "making it" mean "making the metrics", in which case "it" should be
> either "them" or "the metrics"?
>
> = = = = =
>
> Regarding this item:  Please review our updates in the XML file, and let
> us know if (1) any of the remaining <artwork> items should be <sourcecode>
> with type="json" or (2) we mislabeled any artwork as <sourcecode> and
> should change it back to <artwork>:
>
> >> 8) <!-- [rfced] Please review each artwork element in this document.
> >> Should any of the artwork elements be tagged as sourcecode (e.g.,
> perhaps "http-message" or "json") or another type of element?
> >> Please see
> >> <https://www.rfc-editor.org/materials/sourcecode-types.txt; if the
> list on that page does not contain an applicable type, please let us know.
> -->
> > [Qin Wu] Yes, most of artwork elements are json examples, but table
> 1,2,3 are not.
>
>
> = = = = =
>
> Regarding our question 14)c):  We could not find a reply to this question;
> apologies if we missed it:
>
> >> We found "value on" a bit confusing.  Does it mean "value for", "value
> of", or something else?
>
> = = = = =
>
> Regarding our question 27)c) and your reply:  The first reply ("Security
> consideration doesn't need to be reflected in the IANA registry") conflicts
> with the subsequent July 4 email from you in the thread below ("One more
> suggested change to section 8.2 ... Please help sync up with IANA to update
> their page accordingly, thanks!").  The updated table includes the Security
> Considerations column:
>
> Please let us know how Table 3 should look, and we will ensure that this
> document and the IANA page in question are in sync. before this document is
> published.
>
> >> c) Section 8.2. The last column in Table 3 does not match the last
> column in the "ALTO Cost Source" registry at <
> https://www.iana.org/assignments/alto-protocol/>. In Table 3, the column
> is named "Security Considerations" and points to Section 7 of this document
> whereas the column in the IANA registry is named "References" and points to
> this document. Should the IANA registry be updated to match Table 3? -->
> >
> > [Qin Wu] I can see inconsistency issue but to align with other
> registries at <https://www.iana.org/assignments/alto-protocol/> , I
> prefer to keep as it does. Security consideration doesn't need to be
> reflected in the IANA registry in my understanding.
>
>
> = = = = =
>
> Please advise regarding the following items from our question 30):
>
> >> an "sla" value (Sections 5.2.4 and 5.3.4) /
> >>   an SLA value (Sections 4.5.4 and 5.1.4)
>
>
> > Hop Count / hop count (in running text)*
>
> [rfced]  Should "Hop Count is specified to be an integer" be "hop count is
> specified to be an integer"?
>
>
> >> method of measurement or calculation /
> >>   Method of Measurement or Calculation (e.g., "about the
> >>    method of measurement or calculation",
> >>    "such as Method of Measurement or Calculation")
>
>
> = = = = =
>
> The latest files are posted here (please refresh your browser):
>
>    https://www.rfc-editor.org/authors/rfc9439.txt
>    https://www.rfc-editor.org/authors/rfc9439.pdf
>    https://www.rfc-editor.org/authors/rfc9439.html
>    https://www.rfc-editor.org/authors/rfc9439.xml
>    https://www.rfc-editor.org/authors/rfc9439-diff.html
>    https://www.rfc-editor.org/authors/rfc9439-rfcdiff.html
>    https://www.rfc-editor.org/authors/rfc9439-auth48diff.html
>    https://www.rfc-editor.org/authors/rfc9439-lastdiff.html
>    https://www.rfc-editor.org/authors/rfc9439-lastrfcdiff.html
>
>    https://www.rfc-editor.org/authors/rfc9439-xmldiff1.html
>    https://www.rfc-editor.org/authors/rfc9439-xmldiff2.html
>    https://www.rfc-editor.org/authors/rfc9439-alt-diff.html
>
> Richard, we have noted your approval on the AUTH48 status page.  Because
> updates to this document are ongoing, we assume that if you object to any
> subsequent changes you will let us know:
>
>    https://www.rfc-editor.org/auth48/rfc9439
>
> Thanks again!
>
> RFC Editor/lb
>
>
> > On Jul 5, 2023, at 12:47 PM, Y. Richard Yang <[e-mail]> wrote:
> >
> > Hi all,
> >
> > Sorry for the late reply. I looked at all of the changes and do not have
> anything to add and sign it off.
> >
> > Thank you so much!
> > Richard
>
>
> > On Jul 4, 2023, at 7:28 AM, Qin Wu <bill.wu=[e-mail]>
> wrote:
> >
> > One more suggested change to section 8.2
> > OLD TEXT:
> > "
> >      +============+===============================+================+
> >      | Identifier | Intended Semantics            | Security       |
> >      |            |                               | Considerations |
> >      +============+===============================+================+
> >      | nominal    | Values in nominal cases       | Section 7      |
> >      |            | (Section 3.1)                 |                |
> >      +------------+-------------------------------+----------------+
> >      | sla        | Values reflecting service     | Section 7      |
> >      |            | level agreement (Section 3.1) |                |
> >      +------------+-------------------------------+----------------+
> >      | estimation | Values by estimation          | Section 7      |
> >      |            | (Section 3.1)                 |                |
> >      +------------+-------------------------------+----------------+
> >
> > "
> > NEW TEXT:
> > "
> >
> +============+===============================+================+===========+
> > | Identifier | Intended Semantics            | Security       |
> Reference |
> > |            |                               | Considerations |
>  |
> >
> +============+===============================+================+===========+
> > | nominal    | Values in nominal cases       | Section 7      | RFC
> 9439  |
> > |            | (Section 3.1)                 |                |
>  |
> >
> +------------+-------------------------------+----------------+-----------+
> > | sla        | Values reflecting service     | Section 7      | RFC
> 9439  |
> > |            | level agreement (Section 3.1) |                |
>  |
> >
> +------------+-------------------------------+----------------+-----------+
> > | estimation | Values by estimation          | Section 7      | RFC
> 9439  |
> > |            | (Section 3.1)                 |                |
>  |
> >
> +------------+-------------------------------+----------------+-----------+
> > "
> > Please help sync up with IANA to update their page accordingly, thanks!
> >
> > -Qin
>
> > On Jul 4, 2023, at 6:42 AM, Qin Wu <bill.wu=[e-mail]>
> wrote:
> >
> > Hi, RFC Editors:
> > Sorry for late followup, please see my reply inline below on behalf of
> all other authors.
> >
> > -----邮件原件-----
> > 发件人: [e-mail] [mailto:[e-mail]]
> > 发送时间: 2023年6月24日 7:53
> > 收件人: Qin Wu <[e-mail]>; [e-mail]; [e-mail];
> [e-mail]; [e-mail];
> [e-mail]
> > 抄送: [e-mail]; [e-mail]; [e-mail];
> [e-mail]; [e-mail]; [e-mail]
> > 主题: Re: AUTH48: RFC-to-be 9439 <draft-ietf-alto-performance-metrics-28>
> for your review
> >
> > Authors,
> >
> > While reviewing this document during AUTH48, please resolve (as
> necessary) the following questions, which are also in the XML file.
> >
> > 1) <!-- [rfced] Please note that we expanded "ALTO" in the document
> title, per Section 3.6 of RFC 7322 ("RFC Style Guide") (
> https://www.rfc-editor.org/info/rfc7322).  Please review.
> >
> > Original:
> > ALTO Performance Cost Metrics
> >
> > Currently:
> > Application-Layer Traffic Optimization (ALTO) Performance Cost Metrics
> > -->
> > [Qin Wu] Sounds good to me.
> >
> > 2) <!-- [rfced] Authors' Addresses:  Please advise regarding the listed
> zip code for Y. Richard Yang.  We see "06520" here, but please note that
> RFCs 7285, 9240, 9241, and 9275 use 06511 but RFC 8896 uses 06520.  Which
> is correct?
> >
> > Original:
> > Y. Richard Yang
> > Yale University
> > 51 Prospect St
> > New Haven, CT 06520
> > United States of America
> > Email: [e-mail] -->
> >
> > [Qin Wu] I google address of Yale University, I believe '06520' is
> correct one.
> >
> > 3) <!-- [rfced] Sections 1 and 3.1:  Are the quotes around the words
> "where" and "how" necessary?
> >
> > Original:
> > This document registers a set of new cost metrics (Table 1) to allow
> applications to determine "where" to connect based on network  performance
> criteria including delay and bandwidth related metrics.
> > ...
> > The core additional details of a performance metric specify "how" the
> metric is obtained. -->
> >
> >
> > [Qin Wu] I think the quotes can be deleted. What it does is just to put
> emphasize on the key words.
> >
> > 4) <!-- [rfced] Table 1:
> >
> > a) Should "sum Unidirectional Delay" be "sum of Unidirectional Delays"?
> It appears that one or more words are missing.
> > [Qin Wu] Yes, For space limit in the table, the "of" is omitted but I
> believe we should make it complete, I agree to make the change, for the
> first word, we should use Capital letter, i.e.,
> > s/sum Unidirectional Delay/Sum of Unidirectional Delay of links along
> the path
> > b) To what does "from above" refer?
> > [Qin Wu] "from above" is referred to one way delay in direction or
> Unidirectional Delay
> > c) Should "sum of Unidirectional Delay Variation" be "sum of
> Unidirectional Delay Variations"?
> > [Qin Wu] No, sum of Unidirectional Delay Variation is referred to Sum of
> Unidirectional Delay Variation of links along the path
> > Original:
> > sum Unidirectional Delay
> > ...
> > Base: Sum of two directions
> >  from above
> > ...
> > sum of Unidirectional Delay Variation -->
> > [Qin Wu] Here is the proposed changes:
> > OLD TEXT:
> > "
> > sum Unidirectional Delay
> > ...
> > Base: Sum of two directions
> >  from above
> > ...
> > sum of Unidirectional Delay Variation -->
> > "
> > NEW TEXT：
> > "
> > Sum of Unidirectional Delay of links along the path
> > Base: Sum of two directions of unidirectional delay
> >
> > Sum of Unidirectional Delay Variation of links along the path
> > "
> > 5) <!-- [rfced] Section 1:  This sentence is difficult to interpret.
> > Please clarify "... metrics, to expose to application-layer traffic
> optimization, can be a typical mechanism by network operators".
> >
> > Original:
> > Deriving ALTO cost
> > performance metrics from existing network-layer traffic engineering
> performance metrics, to expose to application-layer traffic  optimization,
> can be a typical mechanism by network operators to  deploy ALTO [RFC7971],
> [FlowDirector]. -->
> > [Qin Wu] Here is the proposed change:
> > OLD TEXT:
> > "
> > Deriving ALTO cost performance metrics from existing network-layer
> traffic engineering performance metrics, and making it exposed to
> application-layer traffic optimization, can be a typical mechanism by
> network operators to deploy ALTO [RFC7971], [FlowDirector].
> > "
> >
> > 6) <!-- [rfced] Section 1:  Because the capitalization of the terms in
> this paragraph indicates that these are proper terms but we could not find
> these specific terms in the RFCs cited here, may we update as suggested?
> If not, please provide clarifying text.
> >
> > Original:
> > The common metrics Min/Max Unidirectional Delay defined in
> [RFC8570][RFC8571] and Max Link Bandwidth defined in  [RFC3630,RFC5305] are
> not listed in Table 1 because they can be  handled by applying the
> statistical operators defined in this  document.  The metrics related with
> utilized bandwidth and reservable  bandwidth (i.e., Max Reservable BW and
> Unreserved BW defined in
> > [RFC3630,RFC5305]) are outside the scope of this document.
> >
> > Suggested:
> > The Min/Max Unidirectional Link Delay metric as defined in  [RFC8570]
> and [RFC8571], and Maximum (Link) Bandwidth as defined in  [RFC3630] and
> [RFC5305], are not listed in Table 1 because they can  be handled by
> applying the statistical operators defined in this  document.  The metrics
> related to utilized bandwidth and reservable  bandwidth (i.e., Maximum
> Reservable (Link) Bandwidth and Unreserved  Bandwidth as defined in
> [RFC3630] and [RFC5305]) are outside the  scope of this document. -->
> > [Qin Wu] The proposed changes sound good to me.
> >
> > 7) <!-- [rfced] Section 3:  As this list indicated five items but only
> contained four (i, ii, iv, and v), we renumbered it.  If a fifth item is
> missing here, please let us know what it is.
> >
> > Original:
> > To address the issue and realize ALTO use cases, for metrics in  Table
> 1, this document defines performance metric identifiers which  can be used
> in the ALTO protocol with well-defined (i) Metric Name,
> > (ii) Metric Description, (iv) Units of Measurement, and (v)  Measurement
> Points, which are always specified by the specific ALTO  services; for
> example, endpoint cost service is between the two  endpoints.
> >
> > Currently:
> > To address the issue and realize ALTO use cases for the metrics  listed
> in Table 1, this document defines performance metric  identifiers that can
> be used in the ALTO protocol with the following  well-defined items: (1)
> Metric Name, (2) Metric Description, (3)  Units of Measurement, and (4)
> Measurement Points, which are always  specified by the specific ALTO
> services; for example, the endpoint  cost service is between the two
> endpoints.  Hence, the ALTO  performance metric identifiers provide basic
> metric attributes. -->
> >
> > [Qin Wu] The proposed change looks good, thanks.
> > 8) <!-- [rfced] Please review each artwork element in this document.
> > Should any of the artwork elements be tagged as sourcecode (e.g.,
> perhaps "http-message" or "json") or another type of element?
> > Please see
> > <https://www.rfc-editor.org/materials/sourcecode-types.txt; if the list
> on that page does not contain an applicable type, please let us know. -->
> > [Qin Wu] Yes, most of artwork elements are json examples, but table
> 1,2,3 are not.
> >
> > 9) <!-- [rfced] Figure 1:  Please clarify the meaning of "estimation to".
> > Does it mean "estimation of" or something else?
> >
> > Original:
> > Figure 1. A framework to compute estimation to performance metrics -->
> > [Qin Wu] Yes, s/estimation to/estimation of
> >
> > 10) <!-- [rfced] Section 3.2:  "99% percentile" and "99.9% percentile"
> > read oddly, because they read as "99 percent percentile" and "99.9
> percent percentile".  Should these be "99th percentile" and "99.9th
> percentile", per "95th percentile" as used in Section 4?
> > [Qin Wu] Your understanding is correct.
> > Original:
> > For example, delay-
> > ow:p99 gives the 99% percentile of observed one-way delay; delay-
> > ow:p99.9 gives the 99.9% percentile. -->
> > [Qin Wu] s/99% percentile/99th percentile,  s/99.9% percentile/99.9th
> percentile
> >
> > 11) <!-- [rfced] Section 3.2:  Please note that because we did not see
> any mention of "32", "character", or "metric" in RFC 7258 ("Pervasive
> Monitoring Is an Attack"), we did a search and found the relevant text in
> Section 10.6 of RFC 7285.  We updated the number accordingly.
> > Please let us know any concerns.
> >
> > Original:
> > Note that RFC 7258 limits the overall cost metric identifier to 32
> characters.
> >
> > Currently:
> > Note that [RFC7285] limits the overall cost metric identifier to 32
> characters. -->
> >
> > [Qin Wu] Sounds good, thanks.
> >
> > 12) <!-- [rfced] Section 4:  Should "set of delays {pkt.delay}" be "set
> of (pkt.delay) entries" per "(pkt.delay)", "(pkt.dropped)", and
> "(pkt.hopcount)" earlier in this paragraph?  We ask because we do not see
> curly brackets used around "pkt." items anywhere else in this document.
> > [Qin Wu] I want other authors to confirm this. My understanding is YES.
> > Original (the previous sentence is included for context):
> > The measures of each individual packet (pkt) can include the delay from
> the time when the packet enters the network to the time when the  packet
> leaves the network (pkt.delay); whether the packet is dropped  before
> reaching the destination (pkt.dropped); the number of network  hops that
> the packet traverses (pkt.hopcount).  The semantics of the  performance
> metrics defined in this section are that they are  statistics computed from
> these measures; for example, the  x-percentile of the one-way delay is the
> x-percentile of the set of  delays {pkt.delay} for the packets in the
> stream. -->
> > [Qin Wu] I prefer to keep as it does.
> >
> > 13) <!-- [rfced] Sections 4.1.3, 4.2.3, and 4.3.3:  We had trouble
> following the meaning of these sentences.  If the suggested text is not
> correct, please provide clarifying text.
> >
> > Original:
> > A non-normative reference definition of end-to-end one-way delay is
> [RFC7679].
> > ...
> > A non-normative reference definition of end-to-end  round-trip delay is
> [RFC2681].
> > ...
> > A non-normative reference definition of end-to-end  one-way delay
> variation is [RFC3393].
> >
> > Suggested:
> > A non-normative definition of the end-to-end one-way delay metric is
> provided in [RFC7679].
> > ...
> > A non-normative definition of the end-to-end round-trip delay metric  is
> provided in [RFC2681].
> > ...
> > A non-normative definition of the one-way delay variation metric is
> provided in [RFC3393]. -->
> >
> > [Qin Wu] I believe it is correct, thanks.
> >
> > 14) <!-- [rfced] Examples 1 through 8 (Sections 4.1.3 and subsequent):
> >
> > a) We moved the "Example" information (e.g., "Example 1") from the body
> of the <artwork> elements to figure titles.  Please let us know any
> concerns.
> >
> > b) As the numbering scheme for the examples was misordered (... 3, 5, 4,
> 5, 7, 8), we also ordered the numbering.
> >
> > c) We found "value on" a bit confusing.  Does it mean "value for",
> "value of", or something else?
> >
> > Original numbering scheme:
> > Example 1: Delay value on source-destination endpoint pairs ...
> > Example 1a: Delay value on source-destination endpoint pairs for IPv6 ...
> > Example 2: Round-trip Delay of source-destination endpoint pairs ...
> > Example 3: Delay variation value on source-destination endpoint pairs ...
> > Example 5: Loss rate value on source-destination endpoint pairs ...
> > Example 4: hopcount value on source-destination endpoint pairs ...
> > Example 5: TCP throughput value on source-destination endpoint pairs ...
> > Example 7: bw-residual value on source-destination endpoint pairs ...
> > Example 8: bw-available value on source-destination endpoint pairs
> >
> > Currently:
> >         Figure 2: Delay Value on Source-Destination Endpoint Pairs
> >                                (Example 1) ...
> >       Figure 3: Delay Value on Source-Destination Endpoint Pairs for
> >                             IPv6 (Example 1a) ...
> >      Figure 4: Round-Trip Delay of Source-Destination Endpoint Pairs
> >                               (Example 2) ...
> >       Figure 5: Delay Variation Value on Source-Destination Endpoint
> >                            Pairs (Example 3) ...
> >       Figure 6: Loss Rate Value on Source-Destination Endpoint Pairs
> >                                (Example 4) ...
> >       Figure 7: Hop Count Value on Source-Destination Endpoint Pairs
> >                                (Example 5) ...
> >       Figure 8: TCP Throughput Value on Source-Destination Endpoint
> >                             Pairs (Example 6) ...
> >     Figure 9: Residual Bandwidth Value on Source-Destination Endpoint
> >                             Pairs (Example 7) ...
> >         Figure 10: Available Bandwidth Value on Source-Destination
> >                         Endpoint Pairs (Example 8) -->
> >
> > [Qin Wu] Sounds good to me.
> > 15) <!-- [rfced] Section 4.1.4:  Please clarify the meaning of "the URI
> to the URI field" in this sentence.
> >
> > Original ("RECOMMEDED" has been fixed):
> > If the estimation is from the IPPM measurement framework, it is
> RECOMMEDED that the "parameters" field of an "estimation" one-way  delay
> metric includes the following information: the URI to the URI  field of the
> IPPM metric defined in the IPPM performance metric  [IANA-IPPM] registry
> (e.g., https://www.iana.org/assignments/
> > performance-metrics/OWDelay_Active_IP-UDP-Poisson-
> > Payload250B_RFC8912sec7_Seconds_95Percentile).
> >
> > Possibly (as we see that all other field names are quoted):
> > If the estimation is from the IPPM measurement framework, it is
> RECOMMENDED that the "parameters" field of an "estimation" one-way  delay
> metric include the URI in the "URI" field of the IPPM metric  defined in
> the IPPM "Performance Metrics" registry [IANA-IPPM]  (e.g., <
> https://www.iana.org/assignments/performance-metrics/
> > OWDelay_Active_IP-UDP-Poisson-Payload250B_RFC8912sec7_Seconds_
> > 95Percentile>). -->
> >
> > [Qin Wu] your proposed change seems more clear than the original one,
> thanks!
> >
> > 16) <!-- [rfced] Section 4.3.3:  To what does "the one" refer in this
> sentence?  If the suggested text is not correct, please provide clarifying
> text.
> >
> > Original:
> > This document defines the specific case that F selects as the "first"
> > packet the one with the smallest one-way delay.
> >
> > Suggested:
> > This document defines the specific case where F selects the packet  with
> the smallest one-way delay as the "first" packet. -->
> >
> > [Qin Wu] I believe the original and suggested text is equivalent, your
> suggested change works for me.
> >
> > 17) <!-- [rfced] Section 4.3.3:  To what does "it" refer in this
> sentence - the mean, or something else?
> >
> > Original (the previous sentence is included for context):
> > Note that in statistics, variations are typically evaluated by the
> distance from samples relative to the mean.  In networking context,  it is
> more commonly defined from samples relative to the min. -->
> >
> > [Qin Wu] It is referred to variation. To avoid confusion, suggested to
> make change to the previous sentence
> > OLD TEXT:
> > " variations are typically evaluated by the distance from samples
> relative to the mean. "
> > NEW TEXT:
> > " variation is typically evaluated by the distance from samples relative
> to the mean. "
> > 18) <!-- [rfced] Section 4.4.4:  Does "of the average of loss rate" mean
> "of the average loss rate" or something else?
> >
> > Original ("link link" has been fixed):
> > For estimation by aggregation of routing protocol link metrics, the
> default aggregation of the average of loss rate is the sum of the  link
> link loss rates. -->
> >
> > [Qin Wu] Here is the proposed change:
> > NEW TEXT:
> > "
> > For estimation by aggregation of routing protocol link metrics, the
> default aggregation of the average loss rate is the sum of the link loss
> rates.
> > "
> >
> > 19) <!-- [rfced] Section 4.5.4:  Does "hop count" in these sentences
> mean "the hop count", "the hop count metric", or something else?
> >
> > Also, we see "estimating hopcounts" a couple lines later; does that mean
> "hopcount values", "hop counts", or something else?
> >
> > Original:
> > "nominal": Typically hop count does not have a nominal value.
> > ...
> > [Qin Wu] I believe "the" is needed before "hop count does not".
> > "sla": Typically hop count does not have an SLA value.
> > [Qin Wu] same as above, s/ hop count does not have/ the hop count does
> not have
> > "estimation": The exact estimation method is out of the scope of this
> document.  An example of estimating hopcounts is by importing from IGP
> routing protocols. -->
> >
> > [Qin Wu] s/ An example of estimating hopcounts/ An example of estimating
> hop count values
> >
> > 20) <!-- [rfced] Section 5:  We could only find three metrics related to
> throughput and bandwidth:  "tput", "bw-residual", and "bw-available".
> > What is the fourth metric?
> >
> > Original:
> > This section introduces four throughput/bandwidth related metrics. -->
> >
> > [Qin Wu] s/ This section introduces four throughput/bandwidth related
> metrics./ This section introduces three throughput/bandwidth related
> metrics.
> >
> > 21) <!-- [rfced] Section 5.1.3:  "a TCP congestion-control conforming
> flow" reads a bit oddly.  Does it mean "a conformant TCP congestion control
> flow" or something else?
> >
> > [Qin Wu] Maybe we should change "a TCP congestion-control conforming
> flow" into "a congestion-control conforming TCP flow" or ""a TCP flow"".
> > Original:
> > Intended Semantics: To give the throughput of a TCP congestion-  control
> conforming flow from the specified source to the specified  destination. -->
> >
> >
> > 22) <!-- [rfced] Section 5.1.4: This sentence does not parse.  Please
> clarify "provide estimation to", "set to [I-D.ietf-tcpm-rfc8312bis]", and
> "for an ongoing congestion control algorithm such as BBR, a a link to its
> specification".
> > Also, should "Cubic" be "CUBIC" per [I-D.ietf-tcpm-rfc8312bis]?
> >
> > Original ("a a" has been fixed):
> > To specify (1), it is RECOMMENDED that the "parameters"
> > field (object) include a field named "congestion-control-algorithm",
> which provides a URI for the specification of the algorithm; for example,
> for an ALTO server to provide estimation to the throughput  of a Cubic
> Congestion control flow,
> > [Qin Wu] s/ provide estimation to the throughput of a Cubic Congestion
> control flow,/ provide estimation of the throughput for a CUBIC Congestion
> control flow,
> > its "parameters" includes a field "congestion-control-algorithm", with
> value being set to [I-D.ietf-tcpm-rfc8312bis]; for an ongoing congestion
> control algorithm such as BBR, a a link to its specification. -->
> > [Qin Wu] s/ with value being set to [I-D.ietf-tcpm-rfc8312bis]/ with
> value being set to the URI for [I-D.ietf-tcpm-rfc8312bis]
> > [Qin Wu] s/ for an ongoing congestion control algorithm such as BBR, a a
> link to its specification. / for an ongoing congestion control algorithm
> such as BBR, a link to its specification can be added.
> >
> > 23) <!-- [rfced] Section 5.2.3:  Should "residual bandwidth from the
> specified source and the specified destination" be "residual bandwidth from
> the specified source to the specified destination"?
> > We ask because we see "available bandwidth from the specified source to
> the specified destination" in Section 5.3.3.
> >
> > [Qin Wu] Correct.
> > Also, we could not find the word "capacity" or the string "max"
> > used in the context of maximum bandwidth in RFC 8571.  Will this
> particular citation be clear to readers?  (We see "Maximum Bandwidth (i.e.,
> the link capacity)" and "maximum bandwidth (i.e., the link capacity)" in
> RFCs 7471 and 8570, respectively, but nothing similar in RFC 8571.)
> > [Qin Wu] I think it has explained the relation between " residual
> bandwidth " and " link capacity ". I think it is clear.
> > Original:
> > Intended Semantics: To specify temporal and spatial residual bandwidth
> from the specified source and the specified destination.
> > ...[Qin Wu] s/ from the specified source and the specified destination./
> from the specified source to the specified destination.
> > When the max statistical operator is defined for the metric, it
> typically provides the minimum of the link capacities along the path, as
> the default value of the residual bandwidth of a link is its link capacity
> [RFC8571,8570,7471]. -->
> >
> >
> > 24) <!-- [rfced] Sections 5.2.4 and 5.3.4:  "entry in Section 4.1.4 on
> aggregation of routing protocol link metrics" in these two sentences reads
> oddly and doesn't seem necessary.  If the suggested text is not correct,
> please clarify.
> >
> > Original:
> > "estimation": See the "estimation" entry in Section 4.1.4 on
> aggregation of routing protocol link metrics.
> > ...
> > "estimation": See the "estimation" entry in Section 4.1.4 on
> aggregation of routing protocol link metrics.
> >
> > Suggested:
> > "estimation":  See the "estimation" entry in Section 4.1.4.
> > ...
> > "estimation":  See the "estimation" entry in Section 4.1.4. -->
> > [Qin Wu] Sounds good to me.
> >
> > 25) <!-- [rfced] Section 6:  This sentence is not clear as written.
> > If the suggested text is not correct, please clarify the meaning of
> "this document specifies common issues unless one metric has its unique
> challenges".
> >
> > Original (the previous sentence is included for context):
> > Also, the performance metrics specified in this document are similar,
> in that they may use similar data sources and have similar issues in  their
> calculation.  Hence, this document specifies common issues  unless one
> metric has its unique challenges.
> >
> > Suggested:
> > Hence, this document specifies issues that the  performance metrics
> might have in common and also discusses  challenges regarding the
> computation of ALTO performance metrics  (Section 6.4). -->
> > [Qin Wu] Sounds good to me.
> >
> > 26) <!-- [rfced] Section 7:  Please review the following.  In the case
> of item c), please let us know how this document should be updated.
> >
> > a) We changed "ALTO Server" and "ALTO Client" to "ALTO server" and "ALTO
> client" per usage in RFC 7285.
> >
> > b) We removed the quotes in the second sentence, as the comparable
> sentence in Section 8.3.5 of RFC 7285 cites RFCs 2818 and 5246, both of
> which are obsolete.  From Section 8.3.5 of RFC 7285:
> >
> > ALTO server implementations as well as ALTO client implementations  MUST
> support the "https" URI scheme [RFC2818] and Transport Layer  Security
> (TLS) [RFC5246].
> >
> > c) RFC 7230 has been obsoleted by RFCs 9110 and 9112.  In the case of
> this document, it is not clear whether we should cite RFC 9110 and/or RFC
> 9112.  Please let us know which RFC should be cited.
> >
> > Original (this document):
> > As specified in "Protection Strategies" (Section 15.3.2 of [RFC7285]),
> the ALTO Server should authenticate ALTO Clients when transmitting an  ALTO
> information resource containing sensitive TE performance  metrics.
> "Authentication and Encryption" (Section 8.3.5 of
> > [RFC7285]) specifies that "ALTO Server implementations as well as  ALTO
> Client implementations MUST support the "https" URI scheme of  [RFC7230]
> and Transport Layer Security (TLS) of [RFC8446]".
> >
> > Currently (still cites obsoleted RFC 7230):
> > As specified in Section 15.3.2 ("Protection Strategies") of [RFC7285],
> the ALTO server should authenticate ALTO clients when transmitting an  ALTO
> information resource containing sensitive TE performance  metrics.  Section
> 8.3.5 ("Authentication and Encryption") of  [RFC7285] specifies that ALTO
> server implementations as well as ALTO  client implementations MUST support
> the "https" URI scheme [RFC7230]  and Transport Layer Security (TLS)
> [RFC8446]. -->
> >
> > [Qin Wu] The proposed change looks good, I believe RFC7230 should be
> replaced by RFC9110 since https URI scheme is defined in RFC9110.
> >
> > 27) <!-- [rfced] We have included some specific questions about the IANA
> text below. In addition to responding to those questions, please review all
> of the IANA-related updates carefully and let us know if any further
> updates are needed.
> >
> > a) FYI - For clarity, we created subsections within Section 8.
> >
> > Currently:
> >   8.1 ALTO Cost Metric Registry
> >   8.2 ALTO Cost Source Registry
> >
> > b) Section 8.2. May we change '"ALTO Cost Source" registry'
> > here and in Section 3.1 to '"ALTO Cost Source Types" registry', to match
> the plural style used for all other registries listed on <
> https://www.iana.org/assignments/alto-protocol/>?  If you agree to this
> change, we will ask IANA to update their page accordingly.
> >
> > Original (Section 3.1 and this section):
> > The "cost-source" field
> > of the "cost-context" field MUST be one registered in "ALTO Cost
> Source" registry (Section 8).
> > ...
> > This document requests the creation of the "ALTO Cost Source"
> > registry.
> >
> > Suggested:
> > The "cost-source" field
> > of the "cost-context" field MUST be one that is registered in the  "ALTO
> Cost Source Types" registry (Section 8).
> > ...
> > This document has created the "ALTO Cost Source Types" registry.
> >
> > [Qin Wu] Sounds good to me.
> > c) Section 8.2. The last column in Table 3 does not match the last
> column in the "ALTO Cost Source" registry at <
> https://www.iana.org/assignments/alto-protocol/>. In Table 3, the column
> is named "Security Considerations" and points to Section 7 of this document
> whereas the column in the IANA registry is named "References" and points to
> this document. Should the IANA registry be updated to match Table 3? -->
> >
> > [Qin Wu] I can see inconsistency issue but to align with other
> registries at <https://www.iana.org/assignments/alto-protocol/> , I
> prefer to keep as it does. Security consideration doesn't need to be
> reflected in the IANA registry in my understanding.
> >
> > 28) <!-- [rfced] References:  We could not find a URL or DOI for
> [Prophet].  Also, we couldn't locate it via ACM/IEEE Transactions on
> Networking 2020.  Is this article available online somewhere and not behind
> a paywall?
> >
> > Original:
> > [Prophet]  Gao, K., Zhang, J., and YR. Yang, "Prophet: Fast, Accurate
> >            Throughput Prediction with Reactive Flows", ACM/IEEE
> >            Transactions on Networking July, 2020. -->
> >
> > [Qin Wu] The Prophet paper can be found at the following URL:
> > https://dl.acm.org/doi/10.1109/TNET.2020.3016838
> >
> > 29) <!-- [rfced] Please review the "Inclusive Language" portion of the
> online Style Guide at <
> https://www.rfc-editor.org/styleguide/part2/#inclusive_language>,
> > and let us know if any changes are needed.
> >
> > Note that our script did not flag any words in particular, but this
> should still be reviewed as a best practice. -->
> >
> > [Qin Wu] No, thanks.
> >
> > 30) <!-- [rfced] The following terms appear to be used inconsistently in
> this document.  Please let us know which form is preferred.
> >
> > an "sla" value (Sections 5.2.4 and 5.3.4) /
> >   an SLA value (Sections 4.5.4 and 5.1.4)
> >
> > Available Bandwidth / available bandwidth (used generally in
> >   running text)
> >
> > Delay Variation / delay variation (in running text)*
> >
> > Hop Count / hop count (in running text)*
> >
> > Loss Rate / loss rate (in running text)*
> >
> > method of measurement or calculation /
> >   Method of Measurement or Calculation (e.g., "about the
> >    method of measurement or calculation",
> >    "such as Method of Measurement or Calculation")
> >
> > One-way Delay / one-way delay (in running text)*
> >
> > Residual Bandwidth / Residual bandwidth (metric)
> >   (in running text) (We also see "residual bandwidth".)
> >
> > Round-trip Delay / round-trip delay (in running text)*
> >
> > * For example, compare the sentence just after Table 1, the
> >   "These eight performance metrics can be classified" sentence, and
> >   the first sentence of Section 4. -->
> > [Qin Wu] Here is the proposed change to section 1:
> > OLD TEXT:
> > "
> > The first six metrics listed in Table 1 (i.e., One-way Delay, Round-
> >   trip Delay, Delay Variation, Loss Rate, Residual Bandwidth, and
> >   Available Bandwidth)
> > "
> > NEW TEXT:
> > "
> > The first six metrics listed in Table 1 (i.e., one-way delay, round-
> >   trip delay, delay variation, loss rate, residual bandwidth, and
> >   available bandwidth)
> > "
> > OLD TEXT:
> > "
> > These eight performance metrics can be classified into two
> >   categories: those derived from the performance of individual packets
> >   (i.e., One-way Delay, Round-trip Delay, Delay Variation, Loss Rate,
> >   and Hop Count) and those related to bandwidth/throughput (Residual
> >   bandwidth, Available Bandwidth, and TCP throughput).
> > "
> > NEW TEXT:
> > "
> > These eight performance metrics can be classified into two
> >   categories: those derived from the performance of individual packets
> >   (i.e., one-way delay, round-trip delay, delay variation, loss rate,
> >   and hop count) and those related to bandwidth/throughput (residual
> >   bandwidth, available bandwidth, and TCP throughput).
> > "
> > Thank you.
> >
> > RFC Editor/lb/kc
> >
> >
> > On Jun 23, 2023, at 4:50 PM, [e-mail] wrote:
> >
> > *****IMPORTANT*****
> >
> > Updated 2023/06/23
> >
> > RFC Author(s):
> > --------------
> >
> > Instructions for Completing AUTH48
> >
> > Your document has now entered AUTH48.  Once it has been reviewed and
> approved by you and all coauthors, it will be published as an RFC.
> > If an author is no longer available, there are several remedies
> available as listed in the FAQ (https://www.rfc-editor.org/faq/).
> >
> > You and you coauthors are responsible for engaging other parties (e.g.,
> Contributors or Working Group) as necessary before providing your approval.
> >
> > Planning your review
> > ---------------------
> >
> > Please review the following aspects of your document:
> >
> > *  RFC Editor questions
> >
> >  Please review and resolve any questions raised by the RFC Editor
> >  that have been included in the XML file as comments marked as
> >  follows:
> >
> >  <!-- [rfced] ... -->
> >
> >  These questions will also be sent in a subsequent email.
> >
> > *  Changes submitted by coauthors
> >
> >  Please ensure that you review any changes submitted by your
> >  coauthors.  We assume that if you do not speak up that you
> >  agree to changes submitted by your coauthors.
> >
> > *  Content
> >
> >  Please review the full content of the document, as this cannot
> >  change once the RFC is published.  Please pay particular attention to:
> >  - IANA considerations updates (if applicable)
> >  - contact information
> >  - references
> >
> > *  Copyright notices and legends
> >
> >  Please review the copyright notice and legends as defined in
> >  RFC 5378 and the Trust Legal Provisions
> >  (TLP – https://trustee.ietf.org/license-info/).
> >
> > *  Semantic markup
> >
> >  Please review the markup in the XML file to ensure that elements of
> >  content are correctly tagged.  For example, ensure that <sourcecode>
> >  and <artwork> are set correctly.  See details at
> >  <https://authors.ietf.org/rfcxml-vocabulary>.
> >
> > *  Formatted output
> >
> >  Please review the PDF, HTML, and TXT files to ensure that the
> >  formatted output, as generated from the markup in the XML file, is
> >  reasonable.  Please note that the TXT will have formatting
> >  limitations compared to the PDF and HTML.
> >
> >
> > Submitting changes
> > ------------------
> >
> > To submit changes, please reply to this email using ‘REPLY ALL’ as all
> the parties CCed on this message need to see your changes. The parties
> > include:
> >
> >  *  your coauthors
> >
> >  *  [e-mail] (the RPC team)
> >
> >  *  other document participants, depending on the stream (e.g.,
> >     IETF Stream participants are your working group chairs, the
> >     responsible ADs, and the document shepherd).
> >
> >  *  [e-mail], which is a new archival mailing list
> >     to preserve AUTH48 conversations; it is not an active discussion
> >     list:
> >
> >    *  More info:
> >
> https://mailarchive.ietf.org/arch/msg/ietf-announce/yb6lpIGh-4Q9l2USxIAe6P8O4Zc
> >
> >    *  The archive itself:
> >       https://mailarchive.ietf.org/arch/browse/auth48archive/
> >
> >    *  Note: If only absolutely necessary, you may temporarily opt out
> >       of the archiving of messages (e.g., to discuss a sensitive matter).
> >       If needed, please add a note at the top of the message that you
> >       have dropped the address. When the discussion is concluded,
> >       [e-mail] will be re-added to the CC list and
> >       its addition will be noted at the top of the message.
> >
> > You may submit your changes in one of two ways:
> >
> > An update to the provided XML file
> > — OR —
> > An explicit list of changes in this format
> >
> > Section # (or indicate Global)
> >
> > OLD:
> > old text
> >
> > NEW:
> > new text
> >
> > You do not need to reply with both an updated XML file and an explicit
> list of changes, as either form is sufficient.
> >
> > We will ask a stream manager to review and approve any changes that seem
> beyond editorial in nature, e.g., addition of new text, deletion of text,
> and technical changes.  Information about stream managers can be found in
> the FAQ.  Editorial changes do not require approval from a stream manager.
> >
> >
> > Approving for publication
> > --------------------------
> >
> > To approve your RFC for publication, please reply to this email stating
> that you approve this RFC for publication.  Please use ‘REPLY ALL’, as all
> the parties CCed on this message need to see your approval.
> >
> >
> > Files
> > -----
> >
> > The files are available here:
> >  https://www.rfc-editor.org/authors/rfc9439.xml
> >  https://www.rfc-editor.org/authors/rfc9439.html
> >  https://www.rfc-editor.org/authors/rfc9439.pdf
> >  https://www.rfc-editor.org/authors/rfc9439.txt
> >
> > Diff file of the text:
> >  https://www.rfc-editor.org/authors/rfc9439-diff.html
> >  https://www.rfc-editor.org/authors/rfc9439-rfcdiff.html (side by side)
> >
> > Diff of the XML:
> >  https://www.rfc-editor.org/authors/rfc9439-xmldiff1.html
> >
> > The following files are provided to facilitate creation of your own diff
> files of the XML.
> >
> > Initial XMLv3 created using XMLv2 as input:
> >  https://www.rfc-editor.org/authors/rfc9439.original.v2v3.xml
> >
> > XMLv3 file that is a best effort to capture v3-related format updates
> > only:
> >  https://www.rfc-editor.org/authors/rfc9439.form.xml
> >
> >
> > Tracking progress
> > -----------------
> >
> > The details of the AUTH48 status of your document are here:
> >  https://www.rfc-editor.org/auth48/rfc9439
> >
> > Please let us know if you have any questions.
> >
> > Thank you for your cooperation,
> >
> > RFC Editor
> >
> > --------------------------------------
> > RFC9439 (draft-ietf-alto-performance-metrics-28)
> >
> > Title            : ALTO Performance Cost Metrics
> > Author(s)        : Q. Wu, Y. Yang, Y. Lee, D. Dhody, S. Randriamasy, L.
> Contreras
> > WG Chair(s)      : Jan Seedorf, Mohamed Boucadair, Qin Wu
> > Area Director(s) : Martin Duke, Zaheduzzaman Sarker
> >
> >
> >
> >
>
>
>