Re: [auth48] AUTH48: RFC-to-be 9439 <draft-ietf-alto-performance-metrics-28> for your review

rfc-editor@rfc-editor.org Fri, 23 June 2023 23:52 UTC

Return-Path: <wwwrun@rfcpa.amsl.com>
X-Original-To: auth48archive@ietfa.amsl.com
Delivered-To: auth48archive@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 7430DC151099; Fri, 23 Jun 2023 16:52:47 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -0.808
X-Spam-Level:
X-Spam-Status: No, score=-0.808 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, CTE_8BIT_MISMATCH=0.84, GB_SUMOF=5, HEADER_FROM_DIFFERENT_DOMAINS=0.25, RCVD_IN_DNSWL_HI=-5, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_PASS=-0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OLJWBnABCpp1; Fri, 23 Jun 2023 16:52:43 -0700 (PDT)
Received: from rfcpa.amsl.com (rfc-editor.org [50.223.129.200]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7F071C15152D; Fri, 23 Jun 2023 16:52:43 -0700 (PDT)
Received: by rfcpa.amsl.com (Postfix, from userid 499) id 5668110D6B; Fri, 23 Jun 2023 16:52:43 -0700 (PDT)
To: bill.wu@huawei.com, yry@cs.yale.edu, young.lee@gmail.com, dhruv.ietf@gmail.com, sabine.randriamasy@nokia-bell-labs.com, luismiguel.contrerasmurillo@telefonica.com
From: rfc-editor@rfc-editor.org
Cc: rfc-editor@rfc-editor.org, alto-ads@ietf.org, alto-chairs@ietf.org, ietf@j-f-s.de, martin.h.duke@gmail.com, auth48archive@rfc-editor.org
Content-type: text/plain; charset="UTF-8"
Message-Id: <20230623235243.5668110D6B@rfcpa.amsl.com>
Date: Fri, 23 Jun 2023 16:52:43 -0700
Archived-At: <https://mailarchive.ietf.org/arch/msg/auth48archive/dgxip6eCjwkp6p4q-6FZ9TrQ234>
Subject: Re: [auth48] AUTH48: RFC-to-be 9439 <draft-ietf-alto-performance-metrics-28> for your review
X-BeenThere: auth48archive@rfc-editor.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: "Archiving AUTH48 exchanges between the RFC Production Center, the authors, and other related parties" <auth48archive.rfc-editor.org>
List-Unsubscribe: <https://mailman.rfc-editor.org/mailman/options/auth48archive>, <mailto:auth48archive-request@rfc-editor.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/auth48archive/>
List-Post: <mailto:auth48archive@rfc-editor.org>
List-Help: <mailto:auth48archive-request@rfc-editor.org?subject=help>
List-Subscribe: <https://mailman.rfc-editor.org/mailman/listinfo/auth48archive>, <mailto:auth48archive-request@rfc-editor.org?subject=subscribe>
X-List-Received-Date: Fri, 23 Jun 2023 23:52:47 -0000

Authors,

While reviewing this document during AUTH48, please resolve (as necessary) the following questions, which are also in the XML file.

1) <!-- [rfced] Please note that we expanded "ALTO" in the document
title, per Section 3.6 of RFC 7322 ("RFC Style Guide")
(https://www.rfc-editor.org/info/rfc7322).  Please review.

Original:
 ALTO Performance Cost Metrics

Currently:
 Application-Layer Traffic Optimization (ALTO) Performance Cost Metrics
-->


2) <!-- [rfced] Authors' Addresses:  Please advise regarding the listed
zip code for Y. Richard Yang.  We see "06520" here, but please note
that RFCs 7285, 9240, 9241, and 9275 use 06511 but RFC 8896 uses
06520.  Which is correct?

Original:
 Y. Richard Yang
 Yale University
 51 Prospect St
 New Haven, CT 06520
 United States of America
 Email: yry@cs.yale.edu -->


3) <!-- [rfced] Sections 1 and 3.1:  Are the quotes around the words
"where" and "how" necessary?

Original:
 This document registers a set of new cost metrics (Table 1) to allow
 applications to determine "where" to connect based on network
 performance criteria including delay and bandwidth related metrics.
...
 The core additional details of a performance metric specify "how" the
 metric is obtained. -->


4) <!-- [rfced] Table 1:  

a) Should "sum Unidirectional Delay" be "sum of Unidirectional
Delays"?  It appears that one or more words are missing.

b) To what does "from above" refer?

c) Should "sum of Unidirectional Delay Variation" be "sum of
Unidirectional Delay Variations"?

Original:
 sum Unidirectional Delay
...
 Base: Sum of two directions
  from above
...
 sum of Unidirectional Delay Variation -->


5) <!-- [rfced] Section 1:  This sentence is difficult to interpret.
Please clarify "... metrics, to expose to application-layer traffic
 optimization, can be a typical mechanism by network operators".

Original:
 Deriving ALTO cost
 performance metrics from existing network-layer traffic engineering
 performance metrics, to expose to application-layer traffic
 optimization, can be a typical mechanism by network operators to
 deploy ALTO [RFC7971], [FlowDirector]. -->


6) <!-- [rfced] Section 1:  Because the capitalization of the terms in
this paragraph indicates that these are proper terms but we could not
find these specific terms in the RFCs cited here, may we update as
suggested?  If not, please provide clarifying text.

Original:
 The common metrics Min/Max Unidirectional Delay defined in
 [RFC8570][RFC8571] and Max Link Bandwidth defined in
 [RFC3630,RFC5305] are not listed in Table 1 because they can be
 handled by applying the statistical operators defined in this
 document.  The metrics related with utilized bandwidth and reservable
 bandwidth (i.e., Max Reservable BW and Unreserved BW defined in
 [RFC3630,RFC5305]) are outside the scope of this document.

Suggested:
 The Min/Max Unidirectional Link Delay metric as defined in
 [RFC8570] and [RFC8571], and Maximum (Link) Bandwidth as defined in
 [RFC3630] and [RFC5305], are not listed in Table 1 because they can
 be handled by applying the statistical operators defined in this
 document.  The metrics related to utilized bandwidth and reservable
 bandwidth (i.e., Maximum Reservable (Link) Bandwidth and Unreserved
 Bandwidth as defined in [RFC3630] and [RFC5305]) are outside the
 scope of this document. -->


7) <!-- [rfced] Section 3:  As this list indicated five items but only
contained four (i, ii, iv, and v), we renumbered it.  If a fifth item
is missing here, please let us know what it is.

Original:
 To address the issue and realize ALTO use cases, for metrics in
 Table 1, this document defines performance metric identifiers which
 can be used in the ALTO protocol with well-defined (i) Metric Name,
 (ii) Metric Description, (iv) Units of Measurement, and (v)
 Measurement Points, which are always specified by the specific ALTO
 services; for example, endpoint cost service is between the two
 endpoints.

Currently:
 To address the issue and realize ALTO use cases for the metrics
 listed in Table 1, this document defines performance metric
 identifiers that can be used in the ALTO protocol with the following
 well-defined items: (1) Metric Name, (2) Metric Description, (3)
 Units of Measurement, and (4) Measurement Points, which are always
 specified by the specific ALTO services; for example, the endpoint
 cost service is between the two endpoints.  Hence, the ALTO
 performance metric identifiers provide basic metric attributes. -->


8) <!-- [rfced] Please review each artwork element in this document.
Should any of the artwork elements be tagged as sourcecode (e.g.,
perhaps "http-message" or "json") or another type of element?
Please see
<https://www.rfc-editor.org/materials/sourcecode-types.txt; if the
list on that page does not contain an applicable type, please let us
know. -->


9) <!-- [rfced] Figure 1:  Please clarify the meaning of "estimation to".
Does it mean "estimation of" or something else?

Original:
 Figure 1. A framework to compute estimation to performance metrics -->


10) <!-- [rfced] Section 3.2:  "99% percentile" and "99.9% percentile"
read oddly, because they read as "99 percent percentile" and "99.9
percent percentile".  Should these be "99th percentile" and "99.9th
percentile", per "95th percentile" as used in Section 4?

Original:
 For example, delay-
 ow:p99 gives the 99% percentile of observed one-way delay; delay-
 ow:p99.9 gives the 99.9% percentile. -->


11) <!-- [rfced] Section 3.2:  Please note that because we did not see
any mention of "32", "character", or "metric" in RFC 7258 ("Pervasive
Monitoring Is an Attack"), we did a search and found the relevant
text in Section 10.6 of RFC 7285.  We updated the number accordingly.
Please let us know any concerns.

Original:
 Note that RFC 7258 limits the overall cost metric identifier to 32
 characters.

Currently:
 Note that [RFC7285] limits the overall cost metric identifier to 32
 characters. -->


12) <!-- [rfced] Section 4:  Should "set of delays {pkt.delay}" be
"set of (pkt.delay) entries" per "(pkt.delay)", "(pkt.dropped)", and
"(pkt.hopcount)" earlier in this paragraph?  We ask because we do not
see curly brackets used around "pkt." items anywhere else in this
document.

Original (the previous sentence is included for context):
 The measures of each individual packet (pkt) can include the delay from
 the time when the packet enters the network to the time when the
 packet leaves the network (pkt.delay); whether the packet is dropped
 before reaching the destination (pkt.dropped); the number of network
 hops that the packet traverses (pkt.hopcount).  The semantics of the
 performance metrics defined in this section are that they are
 statistics computed from these measures; for example, the
 x-percentile of the one-way delay is the x-percentile of the set of
 delays {pkt.delay} for the packets in the stream. -->


13) <!-- [rfced] Sections 4.1.3, 4.2.3, and 4.3.3:  We had trouble
following the meaning of these sentences.  If the suggested text is
not correct, please provide clarifying text.

Original:
 A non-normative reference definition of end-to-end one-way delay is
 [RFC7679].
...
 A non-normative reference definition of end-to-end
 round-trip delay is [RFC2681].
...
 A non-normative reference definition of end-to-end
 one-way delay variation is [RFC3393].

Suggested:
 A non-normative definition of the end-to-end one-way delay metric is
 provided in [RFC7679].
...
 A non-normative definition of the end-to-end round-trip delay metric
 is provided in [RFC2681].
...
 A non-normative definition of the one-way delay variation metric is
 provided in [RFC3393]. -->


14) <!-- [rfced] Examples 1 through 8 (Sections 4.1.3 and subsequent):

a) We moved the "Example" information (e.g., "Example 1") from the
body of the <artwork> elements to figure titles.  Please let us know
any concerns.

b) As the numbering scheme for the examples was misordered
(... 3, 5, 4, 5, 7, 8), we also ordered the numbering.

c) We found "value on" a bit confusing.  Does it mean "value for",
"value of", or something else?

Original numbering scheme:
 Example 1: Delay value on source-destination endpoint pairs
...
 Example 1a: Delay value on source-destination endpoint pairs for IPv6
...
 Example 2: Round-trip Delay of source-destination endpoint pairs
...
 Example 3: Delay variation value on source-destination endpoint pairs
...
 Example 5: Loss rate value on source-destination endpoint pairs
...
 Example 4: hopcount value on source-destination endpoint pairs
...
 Example 5: TCP throughput value on source-destination endpoint pairs
...
 Example 7: bw-residual value on source-destination endpoint pairs
...
 Example 8: bw-available value on source-destination endpoint pairs

Currently:
         Figure 2: Delay Value on Source-Destination Endpoint Pairs
                                (Example 1)
...
       Figure 3: Delay Value on Source-Destination Endpoint Pairs for
                             IPv6 (Example 1a)
...
      Figure 4: Round-Trip Delay of Source-Destination Endpoint Pairs
                               (Example 2)
...
       Figure 5: Delay Variation Value on Source-Destination Endpoint
                            Pairs (Example 3)
...
       Figure 6: Loss Rate Value on Source-Destination Endpoint Pairs
                                (Example 4)
...
       Figure 7: Hop Count Value on Source-Destination Endpoint Pairs
                                (Example 5)
...
       Figure 8: TCP Throughput Value on Source-Destination Endpoint
                             Pairs (Example 6)
...
     Figure 9: Residual Bandwidth Value on Source-Destination Endpoint
                             Pairs (Example 7)
...
         Figure 10: Available Bandwidth Value on Source-Destination
                         Endpoint Pairs (Example 8) -->


15) <!-- [rfced] Section 4.1.4:  Please clarify the meaning of
"the URI to the URI field" in this sentence.

Original ("RECOMMEDED" has been fixed):
 If the estimation is from the IPPM measurement framework, it is
 RECOMMEDED that the "parameters" field of an "estimation" one-way
 delay metric includes the following information: the URI to the URI
 field of the IPPM metric defined in the IPPM performance metric
 [IANA-IPPM] registry (e.g., https://www.iana.org/assignments/
 performance-metrics/OWDelay_Active_IP-UDP-Poisson-
 Payload250B_RFC8912sec7_Seconds_95Percentile).

Possibly (as we see that all other field names are quoted):
 If the estimation is from the IPPM measurement framework, it is
 RECOMMENDED that the "parameters" field of an "estimation" one-way
 delay metric include the URI in the "URI" field of the IPPM metric
 defined in the IPPM "Performance Metrics" registry [IANA-IPPM]
 (e.g., <https://www.iana.org/assignments/performance-metrics/
 OWDelay_Active_IP-UDP-Poisson-Payload250B_RFC8912sec7_Seconds_
 95Percentile>). -->


16) <!-- [rfced] Section 4.3.3:  To what does "the one" refer in this
sentence?  If the suggested text is not correct, please provide
clarifying text.

Original:
 This document defines the specific case that F selects as the "first"
 packet the one with the smallest one-way delay.

Suggested:
 This document defines the specific case where F selects the packet
 with the smallest one-way delay as the "first" packet. -->


17) <!-- [rfced] Section 4.3.3:  To what does "it" refer in this
sentence - the mean, or something else?

Original (the previous sentence is included for context):
 Note that in statistics, variations are typically evaluated by the
 distance from samples relative to the mean.  In networking context,
 it is more commonly defined from samples relative to the min. -->


18) <!-- [rfced] Section 4.4.4:  Does "of the average of loss rate" mean
"of the average loss rate" or something else?

Original ("link link" has been fixed):
 For estimation by aggregation of routing protocol link metrics, the
 default aggregation of the average of loss rate is the sum of the
 link link loss rates. -->


19) <!-- [rfced] Section 4.5.4:  Does "hop count" in these sentences mean
"the hop count", "the hop count metric", or something else?

Also, we see "estimating hopcounts" a couple lines later; does that
mean "hopcount values", "hop counts", or something else?

Original:
 "nominal": Typically hop count does not have a nominal value.
...
 "sla": Typically hop count does not have an SLA value.

 "estimation": The exact estimation method is out of the scope of this
 document.  An example of estimating hopcounts is by importing from
 IGP routing protocols. -->


20) <!-- [rfced] Section 5:  We could only find three metrics related to
throughput and bandwidth:  "tput", "bw-residual", and "bw-available".
What is the fourth metric?

Original:
 This section introduces four throughput/bandwidth related metrics. -->


21) <!-- [rfced] Section 5.1.3:  "a TCP congestion-control conforming
flow" reads a bit oddly.  Does it mean "a conformant TCP congestion
control flow" or something else?

Original:
 Intended Semantics: To give the throughput of a TCP congestion-
 control conforming flow from the specified source to the specified
 destination. -->


22) <!-- [rfced] Section 5.1.4: This sentence does not parse.  Please
clarify "provide estimation to", "set to
[I-D.ietf-tcpm-rfc8312bis]", and "for an ongoing congestion
control algorithm such as BBR, a a link to its specification".
Also, should "Cubic" be "CUBIC" per [I-D.ietf-tcpm-rfc8312bis]?

Original ("a a" has been fixed):
 To specify (1), it is RECOMMENDED that the "parameters"
 field (object) include a field named "congestion-control-algorithm",
 which provides a URI for the specification of the algorithm; for
 example, for an ALTO server to provide estimation to the throughput
 of a Cubic Congestion control flow, its "parameters" includes a field
 "congestion-control-algorithm", with value being set to
 [I-D.ietf-tcpm-rfc8312bis]; for an ongoing congestion control
 algorithm such as BBR, a a link to its specification. -->


23) <!-- [rfced] Section 5.2.3:  Should "residual bandwidth from the
specified source and the specified destination" be "residual
bandwidth from the specified source to the specified destination"?
We ask because we see "available bandwidth from the specified source
to the specified destination" in Section 5.3.3.

Also, we could not find the word "capacity" or the string "max"
used in the context of maximum bandwidth in RFC 8571.  Will this
particular citation be clear to readers?  (We see "Maximum
Bandwidth (i.e., the link capacity)" and "maximum bandwidth (i.e.,
the link capacity)" in RFCs 7471 and 8570, respectively, but nothing
similar in RFC 8571.)

Original:
 Intended Semantics: To specify temporal and spatial residual
 bandwidth from the specified source and the specified destination.
...
 When the max statistical operator is defined for
 the metric, it typically provides the minimum of the link capacities
 along the path, as the default value of the residual bandwidth of a
 link is its link capacity [RFC8571,8570,7471]. -->


24) <!-- [rfced] Sections 5.2.4 and 5.3.4:  "entry in Section 4.1.4 on
aggregation of routing protocol link metrics" in these two sentences
reads oddly and doesn't seem necessary.  If the suggested text is not
correct, please clarify.

Original:
 "estimation": See the "estimation" entry in Section 4.1.4 on
 aggregation of routing protocol link metrics.
...
 "estimation": See the "estimation" entry in Section 4.1.4 on
 aggregation of routing protocol link metrics.

Suggested:
 "estimation":  See the "estimation" entry in Section 4.1.4.
...
 "estimation":  See the "estimation" entry in Section 4.1.4. -->


25) <!-- [rfced] Section 6:  This sentence is not clear as written.
If the suggested text is not correct, please clarify the meaning of
"this document specifies common issues unless one metric has its
unique challenges".

Original (the previous sentence is included for context):
 Also, the performance metrics specified in this document are similar,
 in that they may use similar data sources and have similar issues in
 their calculation.  Hence, this document specifies common issues
 unless one metric has its unique challenges.

Suggested:
 Hence, this document specifies issues that the
 performance metrics might have in common and also discusses
 challenges regarding the computation of ALTO performance metrics
 (Section 6.4). -->


26) <!-- [rfced] Section 7:  Please review the following.  In the case of
item c), please let us know how this document should be updated.

a) We changed "ALTO Server" and "ALTO Client" to "ALTO server" and
"ALTO client" per usage in RFC 7285.

b) We removed the quotes in the second sentence, as the comparable
sentence in Section 8.3.5 of RFC 7285 cites RFCs 2818 and 5246, both
of which are obsolete.  From Section 8.3.5 of RFC 7285:

 ALTO server implementations as well as ALTO client implementations
 MUST support the "https" URI scheme [RFC2818] and Transport Layer
 Security (TLS) [RFC5246].

c) RFC 7230 has been obsoleted by RFCs 9110 and 9112.  In the case of
this document, it is not clear whether we should cite RFC 9110 and/or
RFC 9112.  Please let us know which RFC should be cited.

Original (this document):
 As specified in "Protection Strategies" (Section 15.3.2 of [RFC7285]),
 the ALTO Server should authenticate ALTO Clients when transmitting an
 ALTO information resource containing sensitive TE performance
 metrics.  "Authentication and Encryption" (Section 8.3.5 of
 [RFC7285]) specifies that "ALTO Server implementations as well as
 ALTO Client implementations MUST support the "https" URI scheme of
 [RFC7230] and Transport Layer Security (TLS) of [RFC8446]".

Currently (still cites obsoleted RFC 7230):
 As specified in Section 15.3.2 ("Protection Strategies") of [RFC7285],
 the ALTO server should authenticate ALTO clients when transmitting an
 ALTO information resource containing sensitive TE performance
 metrics.  Section 8.3.5 ("Authentication and Encryption") of
 [RFC7285] specifies that ALTO server implementations as well as ALTO
 client implementations MUST support the "https" URI scheme [RFC7230]
 and Transport Layer Security (TLS) [RFC8446]. -->


27) <!-- [rfced] We have included some specific questions about the IANA
text below. In addition to responding to those questions, please
review all of the IANA-related updates carefully and let us know
if any further updates are needed.

a) FYI - For clarity, we created subsections within Section 8.

Currently:
   8.1 ALTO Cost Metric Registry
   8.2 ALTO Cost Source Registry

b) Section 8.2. May we change '"ALTO Cost Source" registry'
here and in Section 3.1 to '"ALTO Cost Source Types" registry', to
match the plural style used for all other registries listed on
<https://www.iana.org/assignments/alto-protocol/>?  If you agree to
this change, we will ask IANA to update their page accordingly.

Original (Section 3.1 and this section):
 The "cost-source" field
 of the "cost-context" field MUST be one registered in "ALTO Cost
 Source" registry (Section 8).
...
 This document requests the creation of the "ALTO Cost Source"
 registry.

Suggested:
 The "cost-source" field
 of the "cost-context" field MUST be one that is registered in the
 "ALTO Cost Source Types" registry (Section 8).
...
 This document has created the "ALTO Cost Source Types" registry. 

c) Section 8.2. The last column in Table 3 does not match the last
column in the "ALTO Cost Source" registry at
<https://www.iana.org/assignments/alto-protocol/>. In Table 3, the
column is named "Security Considerations" and points to Section 7 of
this document whereas the column in the IANA registry is named
"References" and points to this document. Should the IANA registry be
updated to match Table 3? -->


28) <!-- [rfced] References:  We could not find a URL or DOI for
[Prophet].  Also, we couldn't locate it via ACM/IEEE Transactions on
Networking 2020.  Is this article available online somewhere and not
behind a paywall?

Original:
 [Prophet]  Gao, K., Zhang, J., and YR. Yang, "Prophet: Fast, Accurate
            Throughput Prediction with Reactive Flows", ACM/IEEE
            Transactions on Networking July, 2020. -->


29) <!-- [rfced] Please review the "Inclusive Language" portion of the
online Style Guide at
<https://www.rfc-editor.org/styleguide/part2/#inclusive_language>,
and let us know if any changes are needed.

Note that our script did not flag any words in particular, but this
should still be reviewed as a best practice. -->


30) <!-- [rfced] The following terms appear to be used inconsistently in
this document.  Please let us know which form is preferred.

 an "sla" value (Sections 5.2.4 and 5.3.4) /
   an SLA value (Sections 4.5.4 and 5.1.4)

 Available Bandwidth / available bandwidth (used generally in
   running text)

 Delay Variation / delay variation (in running text)*

 Hop Count / hop count (in running text)*

 Loss Rate / loss rate (in running text)*

 method of measurement or calculation /
   Method of Measurement or Calculation (e.g., "about the
    method of measurement or calculation",
    "such as Method of Measurement or Calculation")

 One-way Delay / one-way delay (in running text)*

 Residual Bandwidth / Residual bandwidth (metric)
   (in running text) (We also see "residual bandwidth".)

 Round-trip Delay / round-trip delay (in running text)*

 * For example, compare the sentence just after Table 1, the
   "These eight performance metrics can be classified" sentence, and
   the first sentence of Section 4. -->


Thank you.

RFC Editor/lb/kc


On Jun 23, 2023, at 4:50 PM, rfc-editor@rfc-editor.org wrote:

*****IMPORTANT*****

Updated 2023/06/23

RFC Author(s):
--------------

Instructions for Completing AUTH48

Your document has now entered AUTH48.  Once it has been reviewed and 
approved by you and all coauthors, it will be published as an RFC.  
If an author is no longer available, there are several remedies 
available as listed in the FAQ (https://www.rfc-editor.org/faq/).

You and you coauthors are responsible for engaging other parties 
(e.g., Contributors or Working Group) as necessary before providing 
your approval.

Planning your review 
---------------------

Please review the following aspects of your document:

*  RFC Editor questions

  Please review and resolve any questions raised by the RFC Editor 
  that have been included in the XML file as comments marked as 
  follows:

  <!-- [rfced] ... -->

  These questions will also be sent in a subsequent email.

*  Changes submitted by coauthors 

  Please ensure that you review any changes submitted by your 
  coauthors.  We assume that if you do not speak up that you 
  agree to changes submitted by your coauthors.

*  Content 

  Please review the full content of the document, as this cannot 
  change once the RFC is published.  Please pay particular attention to:
  - IANA considerations updates (if applicable)
  - contact information
  - references

*  Copyright notices and legends

  Please review the copyright notice and legends as defined in
  RFC 5378 and the Trust Legal Provisions 
  (TLP – https://trustee.ietf.org/license-info/).

*  Semantic markup

  Please review the markup in the XML file to ensure that elements of  
  content are correctly tagged.  For example, ensure that <sourcecode> 
  and <artwork> are set correctly.  See details at 
  <https://authors.ietf.org/rfcxml-vocabulary>.

*  Formatted output

  Please review the PDF, HTML, and TXT files to ensure that the 
  formatted output, as generated from the markup in the XML file, is 
  reasonable.  Please note that the TXT will have formatting 
  limitations compared to the PDF and HTML.


Submitting changes
------------------

To submit changes, please reply to this email using ‘REPLY ALL’ as all 
the parties CCed on this message need to see your changes. The parties 
include:

  *  your coauthors

  *  rfc-editor@rfc-editor.org (the RPC team)

  *  other document participants, depending on the stream (e.g., 
     IETF Stream participants are your working group chairs, the 
     responsible ADs, and the document shepherd).

  *  auth48archive@rfc-editor.org, which is a new archival mailing list 
     to preserve AUTH48 conversations; it is not an active discussion 
     list:

    *  More info:
       https://mailarchive.ietf.org/arch/msg/ietf-announce/yb6lpIGh-4Q9l2USxIAe6P8O4Zc

    *  The archive itself:
       https://mailarchive.ietf.org/arch/browse/auth48archive/

    *  Note: If only absolutely necessary, you may temporarily opt out 
       of the archiving of messages (e.g., to discuss a sensitive matter).
       If needed, please add a note at the top of the message that you 
       have dropped the address. When the discussion is concluded, 
       auth48archive@rfc-editor.org will be re-added to the CC list and 
       its addition will be noted at the top of the message. 

You may submit your changes in one of two ways:

An update to the provided XML file
— OR —
An explicit list of changes in this format

Section # (or indicate Global)

OLD:
old text

NEW:
new text

You do not need to reply with both an updated XML file and an explicit 
list of changes, as either form is sufficient.

We will ask a stream manager to review and approve any changes that seem
beyond editorial in nature, e.g., addition of new text, deletion of text, 
and technical changes.  Information about stream managers can be found in 
the FAQ.  Editorial changes do not require approval from a stream manager.


Approving for publication
--------------------------

To approve your RFC for publication, please reply to this email stating
that you approve this RFC for publication.  Please use ‘REPLY ALL’,
as all the parties CCed on this message need to see your approval.


Files 
-----

The files are available here:
  https://www.rfc-editor.org/authors/rfc9439.xml
  https://www.rfc-editor.org/authors/rfc9439.html
  https://www.rfc-editor.org/authors/rfc9439.pdf
  https://www.rfc-editor.org/authors/rfc9439.txt

Diff file of the text:
  https://www.rfc-editor.org/authors/rfc9439-diff.html
  https://www.rfc-editor.org/authors/rfc9439-rfcdiff.html (side by side)

Diff of the XML: 
  https://www.rfc-editor.org/authors/rfc9439-xmldiff1.html

The following files are provided to facilitate creation of your own 
diff files of the XML.  

Initial XMLv3 created using XMLv2 as input:
  https://www.rfc-editor.org/authors/rfc9439.original.v2v3.xml 

XMLv3 file that is a best effort to capture v3-related format updates 
only: 
  https://www.rfc-editor.org/authors/rfc9439.form.xml


Tracking progress
-----------------

The details of the AUTH48 status of your document are here:
  https://www.rfc-editor.org/auth48/rfc9439

Please let us know if you have any questions.  

Thank you for your cooperation,

RFC Editor

--------------------------------------
RFC9439 (draft-ietf-alto-performance-metrics-28)

Title            : ALTO Performance Cost Metrics
Author(s)        : Q. Wu, Y. Yang, Y. Lee, D. Dhody, S. Randriamasy, L. Contreras
WG Chair(s)      : Jan Seedorf, Mohamed Boucadair, Qin Wu
Area Director(s) : Martin Duke, Zaheduzzaman Sarker