Re: [auth48] AUTH48: RFC-to-be 9544 <draft-ietf-ippm-pam-09> for your review

[[e-mail]]

Authors,

While reviewing this document during AUTH48, please resolve (as necessary) the following questions, which are also in the XML file.


1) <!-- [rfced] FYI - We updated the abbreviated title (appears in the center of
the running header in the pdf output) to more closely align with the
document title. Please let us know any objections.

Original:
   Framework of PAM

Current:
   PAM for Services Governed by SLOs
-->


2) <!-- [rfced] How may we update the text after the comma to improve clarity?

Original:
   Hence it is not sufficient to measure service levels
   per se over time, but to assess the quality of the service being
   contextually provided (e.g., with the applicable SLO in mind).

Perhaps: 
   Hence, it is not sufficient to measure service levels
   per se over time; the quality of the service being
   contextually provided (e.g., with the applicable SLO in mind) must be also assessed.
-->


3) <!-- [rfced] May we update the text starting with ", and whether their
SLOs..." as follows for to create parallel structure?

Original:
   However, at this point, there are no standard metrics that can be
   used to account for the quality with which services are delivered
   relative to their SLOs, and whether their SLOs are being met at all
   times.

Perhaps: 
   However, at this point, there are no standard metrics that can be
   used to account for the quality with which services are delivered
   relative to their SLOs or to determine whether their SLOs are being met at all
   times.  
-->


4) <!-- [rfced] We see the term "Flow Record" in RFC 7011 but not in RFC 7012
(though "flow" does appear). Please review the citations and let us know
if any updates are needed.

Original:
   Flow records [RFC7011] and
   [RFC7012] maintain statistics about flows, including flow volume and
   flow duration, but again, contain very little information about
   service levels, let alone whether the service levels delivered meet
   their respective targets, i.e., their associated SLOs.
-->


5) <!-- [rfced] FYI - We updated the title of Section 2 from "Conventions and
Terminology" to "Conventions" because "Terminology" is one of the
subsections. Please review and let us know any concerns.

Original:
   2.  Conventions and Terminology
     2.1. Terminology
     2.2. Acronyms

Current:
   2.  Conventions
     2.1. Terminology
     2.2. Acronyms
-->


6) <!-- [rfced] Is "configurable optimal level threshold" correct here? Should
this read "configurable optimal threshold" (i.e., no "level") or "configurable
optimal level (i.e., no "threshold")?

Original:
   *  VI is a time interval during which at least one of the performance
      parameters degraded below its configurable optimal level
      threshold.

Perhaps:
   *  VI is a time interval during which at least one of the performance
      parameters degraded below its configurable optimal 
      threshold.

Or:
   *  VI is a time interval during which at least one of the performance
      parameters degraded below its configurable optimal 
      level.
-->


7) <!-- [rfced] Please confirm that "referred to for" is correct. Or should this
be updated to "referred to", "referred for", or something else?

Original:
   The monitoring of performance parameters to determine the quality of
   an interval is performed between the elements of the network that are
   referred to for the SLO corresponding to the performance parameter.
-->


8) <!-- [rfced] How may we update "this allows distinguishing" here? 

Original:
   For example, this allows distinguishing between cases in
   which violated intervals are caused by isolated violation occurrences
   (such as, a sporadic issue that may be caused by a temporary spike in
   a queue depth along the packet's path) or by broad violations across
   multiple packets (such as a problem with slow route convergence
   across the network or more foundational issues such as insufficient
   network resources).  

Perhaps ("this allows a service provider to distinguish"): 
   For example, this allows a service provider to distinguish between cases in
   which violated intervals are caused by isolated violation occurrences
   (such as a sporadic issue that may be caused by a temporary spike in
   a queue depth along the packet's path) or by broad violations across
   multiple packets (such as a problem with slow route convergence
   across the network or more foundational issues such as insufficient
   network resources).

Or ("this allows for distinguishing"):
   For example, this allows for distinguishing between cases in
   which violated intervals are caused by isolated violation occurrences
   (such as a sporadic issue that may be caused by a temporary spike in
   a queue depth along the packet's path) or by broad violations across
   multiple packets (such as a problem with slow route convergence
   across the network or more foundational issues such as insufficient
   network resources).  
-->


9) <!-- [rfced] This sentence appears in Section 3.2 and points to Section
3. Should "introduced in Section 3" either be removed or updated to "introduced
in this document"? Also, is this sentence introducing the bulleted list
that appear after the paragraph? If so, would adding "The following" be
helpful?

Original:
   A set of metrics can be created based on PAM introduced in Section 3.

Perhaps (omit section pointer):
   The following set of metrics can be created based on PAM.

Or (use "in this document" instead of section pointer):
   The following set of metrics can be created based on PAM as introduced
   in this document.
-->


10) <!-- [rfced] We updated the first two bulleted lists in Section 3.2 (created
parallel structure and removed the parentheses around sentences). Please
review these changes in the diff file and let us know any concerns.
-->


11) <!-- [rfced] May we change the first bulleted list in Sections 3.1 and all
the bulleted lists in Section 3.2 to definition lists? See
https://authors.ietf.org/en/rfcxml-vocabulary#dl.

One example:

Current:
   *  VI is a time interval during which at least one of the performance
      parameters degraded below its configurable optimal level
      threshold.

   *  SVI is a time interval during which at least one of the
      performance parameters degraded below its configurable critical
      threshold.

   *  Consequently, VFI is a time interval during which all performance
      parameters are at or better than their respective pre-defined
      optimal levels.

Perhaps:
   VI: A time interval during which at least one of the performance
      parameters degraded below its configurable optimal level
      threshold.

   SVI: A time interval during which at least one of the
      performance parameters degraded below its configurable critical
      threshold.

   VFI: A time interval during which all performance
      parameters are at or better than their respective pre-defined
      optimal levels.
-->


12) <!-- [rfced] Should "the following section" here read "this section"? This
sentence appears in Section 3.3; the following section is Section 4,
which does not mention a state model.

Original:
   While the definition of a service state model is outside the scope of
   this document, the following section provides some considerations for
   how such a state model and accompanying configuration settings could
   be defined.

Perhaps:
   While the definition of a service state model is outside the scope of
   this document, this section provides some considerations for
   how such a state model and accompanying configuration settings could
   be defined.
-->


13) <!-- [rfced] Would it be helpful to update "YANG/NETCONF/RESTCONF framework"
as follows? Or do you prefer the current?

Original:
   *  A YANG data model will allow PAM to be incorporated into
      monitoring applications based on the YANG/NETCONF/RESTCONF
      framework.  

Perhaps:
   *  A YANG data model will allow PAM to be incorporated into
      monitoring applications based on the YANG, NETCONF,
      and RESTCONF frameworks.  
-->


14) <!-- [rfced] FYI - We rephrased this as follows to form a complete
sentence. The other items in the list are complete sentences. Please
review.

Original:
   *  The definition of the metrics that represent histograms for
      service level parameters with buckets corresponding to individual
      service level objectives,

Updated:
   *  Metrics can be defined to represent histograms for
      service-level parameters with buckets corresponding to individual
      SLOs..
-->


15) <!-- [rfced] Please review "that should be maintained". Should this read "and
should be maintained", or is the current okay? Also, please confirm that
"violated time units" is correct here. We do not see this mentioned
elsewhere in the document.

Original:
   The same service levels that
   constitute SLO violations for one flow that should be maintained as
   part of the "violated time units" and related metrics, may be
   compliant for another flow.  

Perhaps:
   The same service levels that
   constitute SLO violations for one flow and should be maintained as
   part of the "violated time units" and related metrics may be
   compliant for another flow.  
-->


16) <!-- [rfced] FYI - We updated this fragment as follows to create a complete
sentence.

Original:
   By the same token, where the definition of what constitutes a
   "severe" or a "significant" violation depends on configuration
   settings or context.  

Perhaps:
   By the same token, the definition of what constitutes a
   "severe" or a "significant" violation depends on configuration
   settings or context.  
-->


17) <!-- [rfced] Abbreviations

a) We see both of the following as the expansion for PAM. 

  Precision Availability Metric (1 instance)
  Precision Availability Metrics (8 instances)

We updated to use the latter form (i.e., the form with "Metrics"). Please let
us know any objections.


b) We see two instances of "PAMs" in the document (see below). Since "Metrics"
in the expansion is already plural, is the "s" needed in "PAMs"? Please
review.

Original:
   To indicate a historic degree of precision availability, additional
   derived PAMs can be defined as follows:
   ...
   It might be useful for a service provider to determine the current
   condition of the service for which PAMs are maintained. 


c) OAM appears in the list of acronyms in Section 2.2 but is not mentioned
elsewhere in the document. May we delete this term from the list?


d) FYI - We have added an expansion for the following abbreviation
per Section 3.6 of RFC 7322 ("RFC Style Guide"). Please review each
expansion in the document carefully to ensure correctness.

  IP Flow Information Export (IPFIX)
-->


18) <!-- [rfced] Please review the "Inclusive Language" portion of the online 
Style Guide <https://www.rfc-editor.org/styleguide/part2/#inclusive_language>
and let us know if any changes are needed.

Note that our script did not flag any words in particular, but this should 
still be reviewed as a best practice.
-->


19) <!-- [rfced] We see a number of author-inserted comments in the .xml file for
this document. We are unsure if these have been resolved. Please review
and let us know if these can be deleted or if they need to be addressed.
-->


Thank you.

RFC Editor/rv



On Feb 15, 2024, at 5:34 PM, [e-mail] wrote:

*****IMPORTANT*****

Updated 2024/02/15

RFC Author(s):
--------------

Instructions for Completing AUTH48

Your document has now entered AUTH48.  Once it has been reviewed and 
approved by you and all coauthors, it will be published as an RFC.  
If an author is no longer available, there are several remedies 
available as listed in the FAQ (https://www.rfc-editor.org/faq/)

You and you coauthors are responsible for engaging other parties 
(e.g., Contributors or Working Group) as necessary before providing 
your approval.

Planning your review 
---------------------

Please review the following aspects of your document:

*  RFC Editor questions

  Please review and resolve any questions raised by the RFC Editor 
  that have been included in the XML file as comments marked as 
  follows:

  <!-- [rfced] ... -->

  These questions will also be sent in a subsequent email.

*  Changes submitted by coauthors 

  Please ensure that you review any changes submitted by your 
  coauthors.  We assume that if you do not speak up that you 
  agree to changes submitted by your coauthors.

*  Content 

  Please review the full content of the document, as this cannot 
  change once the RFC is published.  Please pay particular attention to:
  - IANA considerations updates (if applicable)
  - contact information
  - references

*  Copyright notices and legends

  Please review the copyright notice and legends as defined in
  RFC 5378 and the Trust Legal Provisions 
  (TLP – https://trustee.ietf.org/license-info/)

*  Semantic markup

  Please review the markup in the XML file to ensure that elements of  
  content are correctly tagged.  For example, ensure that <sourcecode> 
  and <artwork> are set correctly.  See details at 
  <https://authors.ietf.org/rfcxml-vocabulary>.

*  Formatted output

  Please review the PDF, HTML, and TXT files to ensure that the 
  formatted output, as generated from the markup in the XML file, is 
  reasonable.  Please note that the TXT will have formatting 
  limitations compared to the PDF and HTML.


Submitting changes
------------------

To submit changes, please reply to this email using ‘REPLY ALL’ as all 
the parties CCed on this message need to see your changes. The parties 
include:

  *  your coauthors

  *  [e-mail] (the RPC team)

  *  other document participants, depending on the stream (e.g., 
     IETF Stream participants are your working group chairs, the 
     responsible ADs, and the document shepherd).

  *  [e-mail], which is a new archival mailing list 
     to preserve AUTH48 conversations; it is not an active discussion 
     list:

    *  More info:
       https://mailarchive.ietf.org/arch/msg/ietf-announce/yb6lpIGh-4Q9l2USxIAe6P8O4Zc

    *  The archive itself:
       https://mailarchive.ietf.org/arch/browse/auth48archive/

    *  Note: If only absolutely necessary, you may temporarily opt out 
       of the archiving of messages (e.g., to discuss a sensitive matter).
       If needed, please add a note at the top of the message that you 
       have dropped the address. When the discussion is concluded, 
       [e-mail] will be re-added to the CC list and 
       its addition will be noted at the top of the message. 

You may submit your changes in one of two ways:

An update to the provided XML file
— OR —
An explicit list of changes in this format

Section # (or indicate Global)

OLD:
old text

NEW:
new text

You do not need to reply with both an updated XML file and an explicit 
list of changes, as either form is sufficient.

We will ask a stream manager to review and approve any changes that seem
beyond editorial in nature, e.g., addition of new text, deletion of text, 
and technical changes.  Information about stream managers can be found in 
the FAQ.  Editorial changes do not require approval from a stream manager.


Approving for publication
--------------------------

To approve your RFC for publication, please reply to this email stating
that you approve this RFC for publication.  Please use ‘REPLY ALL’,
as all the parties CCed on this message need to see your approval.


Files 
-----

The files are available here:
  https://www.rfc-editor.org/authors/rfc9544.xml
  https://www.rfc-editor.org/authors/rfc9544.html
  https://www.rfc-editor.org/authors/rfc9544.pdf
  https://www.rfc-editor.org/authors/rfc9544.txt

Diff file of the text:
  https://www.rfc-editor.org/authors/rfc9544-diff.html
  https://www.rfc-editor.org/authors/rfc9544-rfcdiff.html (side by side)

Alt-diff of the text (allows you to more easily view changes 
where text has been deleted or moved): 
  https://www.rfc-editor.org/authors/rfc9544-alt-diff.html

Diff of the XML: 
  https://www.rfc-editor.org/authors/rfc9544-xmldiff1.html


Tracking progress
-----------------

The details of the AUTH48 status of your document are here:
  https://www.rfc-editor.org/auth48/rfc9544

Please let us know if you have any questions.  

Thank you for your cooperation,

RFC Editor

--------------------------------------
RFC9544 (draft-ietf-ippm-pam-09)

Title            : Precision Availability Metrics for Services Governed by Service Level Objectives (SLOs)
Author(s)        : G. Mirsky, J. Halpern, X. Min, A. Clemm, J. Strassner, J. François
WG Chair(s)      : Marcus Ihlar, Tommy Pauly
Area Director(s) : Martin Duke, Zaheduzzaman Sarker