Re: [auth48] AUTH48: RFC-to-be 9544 <draft-ietf-ippm-pam-09> for your review

[Greg Mirsky <[e-mail]>]

Dear RFC Editor,
thank you for your thoughtful proposals helping in improving the document.
Please find my responses and notes below tagged GIM>>.

Regards,
Greg

On Thu, Feb 15, 2024 at 5:39 PM <[e-mail]> wrote:

> Authors,
>
> While reviewing this document during AUTH48, please resolve (as necessary)
> the following questions, which are also in the XML file.
>
>
> 1) <!-- [rfced] FYI - We updated the abbreviated title (appears in the
> center of
> the running header in the pdf output) to more closely align with the
> document title. Please let us know any objections.
>
> Original:
>    Framework of PAM
>
> Current:
>    PAM for Services Governed by SLOs
>
GIM>> I agree with the update with a minor modification

PAMs for Service Governed by SLOs

I've looked at the title itself and wanted to ask you and the co-authors a
question. The title introduces the SLO acronym, should the PAM acronym also
be introduced in the title as

 Precision Availability Metrics (PAMs) for Services Governed by Service
Level Objectives (SLOs)

-->
>
>
> 2) <!-- [rfced] How may we update the text after the comma to improve
> clarity?
>
> Original:
>    Hence it is not sufficient to measure service levels
>    per se over time, but to assess the quality of the service being
>    contextually provided (e.g., with the applicable SLO in mind).
>
> Perhaps:
>    Hence, it is not sufficient to measure service levels
>    per se over time; the quality of the service being
>    contextually provided (e.g., with the applicable SLO in mind) must be
> also assessed.
>
GIM>> I agree with the proposed update. It makes the text more readable.

> -->
>
>
> 3) <!-- [rfced] May we update the text starting with ", and whether their
> SLOs..." as follows for to create parallel structure?
>
> Original:
>    However, at this point, there are no standard metrics that can be
>    used to account for the quality with which services are delivered
>    relative to their SLOs, and whether their SLOs are being met at all
>    times.
>
> Perhaps:
>    However, at this point, there are no standard metrics that can be
>    used to account for the quality with which services are delivered
>    relative to their SLOs or to determine whether their SLOs are being met
> at all
>    times.

GIM>> I agree with the proposed update that improves the text. I wonder if
s/or/nor/ is acceptable in this sentence?

>
> -->
>
>
> 4) <!-- [rfced] We see the term "Flow Record" in RFC 7011 but not in RFC
> 7012
> (though "flow" does appear). Please review the citations and let us know
> if any updates are needed.
>
> Original:
>    Flow records [RFC7011] and
>    [RFC7012] maintain statistics about flows, including flow volume and
>    flow duration, but again, contain very little information about
>    service levels, let alone whether the service levels delivered meet
>    their respective targets, i.e., their associated SLOs.
>
GIM>> Thank you for your thoughtful question. I think that reference to RFC
7012 is justified as it refers, as you've noted, to 'Flow' and 'Flow
information'.

> -->
>
>
> 5) <!-- [rfced] FYI - We updated the title of Section 2 from "Conventions
> and
> Terminology" to "Conventions" because "Terminology" is one of the
> subsections. Please review and let us know any concerns.
>
> Original:
>    2.  Conventions and Terminology
>      2.1. Terminology
>      2.2. Acronyms
>
> Current:
>    2.  Conventions
>      2.1. Terminology
>      2.2. Acronyms
>
GIM>> I agree with the update

> -->
>
>
> 6) <!-- [rfced] Is "configurable optimal level threshold" correct here?
> Should
> this read "configurable optimal threshold" (i.e., no "level") or
> "configurable
> optimal level (i.e., no "threshold")?
>
> Original:
>    *  VI is a time interval during which at least one of the performance
>       parameters degraded below its configurable optimal level
>       threshold.
>
> Perhaps:
>    *  VI is a time interval during which at least one of the performance
>       parameters degraded below its configurable optimal
>       threshold.
>
GIM>> Thank you for pointing out that "threshold" and "level" are synonyms.
I slightly prefer the first option (above), but I can live with the other
proposal (below)

>
> Or:
>    *  VI is a time interval during which at least one of the performance
>       parameters degraded below its configurable optimal
>       level.
> -->
>
>
> 7) <!-- [rfced] Please confirm that "referred to for" is correct. Or
> should this
> be updated to "referred to", "referred for", or something else?
>
> Original:
>    The monitoring of performance parameters to determine the quality of
>    an interval is performed between the elements of the network that are
>    referred to for the SLO corresponding to the performance parameter.
>
GIM>> I think that it should be "referred to".

> -->
>
>
> 8) <!-- [rfced] How may we update "this allows distinguishing" here?
>
> Original:
>    For example, this allows distinguishing between cases in
>    which violated intervals are caused by isolated violation occurrences
>    (such as, a sporadic issue that may be caused by a temporary spike in
>    a queue depth along the packet's path) or by broad violations across
>    multiple packets (such as a problem with slow route convergence
>    across the network or more foundational issues such as insufficient
>    network resources).
>
> Perhaps ("this allows a service provider to distinguish"):
>    For example, this allows a service provider to distinguish between
> cases in
>    which violated intervals are caused by isolated violation occurrences
>    (such as a sporadic issue that may be caused by a temporary spike in
>    a queue depth along the packet's path) or by broad violations across
>    multiple packets (such as a problem with slow route convergence
>    across the network or more foundational issues such as insufficient
>    network resources).
>
> Or ("this allows for distinguishing"):
>    For example, this allows for distinguishing between cases in
>    which violated intervals are caused by isolated violation occurrences
>    (such as a sporadic issue that may be caused by a temporary spike in
>    a queue depth along the packet's path) or by broad violations across
>    multiple packets (such as a problem with slow route convergence
>    across the network or more foundational issues such as insufficient
>    network resources).

GIM>> I slightly prefer the latter option as the act of attributing
degradation to a cause could be performed by a function, not an operator.
But I can live with the former option.

>
> -->
>
>
> 9) <!-- [rfced] This sentence appears in Section 3.2 and points to Section
> 3. Should "introduced in Section 3" either be removed or updated to
> "introduced
> in this document"? Also, is this sentence introducing the bulleted list
> that appear after the paragraph? If so, would adding "The following" be
> helpful?
>
> Original:
>    A set of metrics can be created based on PAM introduced in Section 3.
>
> Perhaps (omit section pointer):
>    The following set of metrics can be created based on PAM.
>
> Or (use "in this document" instead of section pointer):
>    The following set of metrics can be created based on PAM as introduced
>    in this document.
>
GIM>> I slightly prefer the latter option, but I can live with the first
one.

> -->
>
>
> 10) <!-- [rfced] We updated the first two bulleted lists in Section 3.2
> (created
> parallel structure and removed the parentheses around sentences). Please
> review these changes in the diff file and let us know any concerns.
>
GIM>> I agree with the update that makes looks of lists in the document
consistent.

> -->
>
>
> 11) <!-- [rfced] May we change the first bulleted list in Sections 3.1 and
> all
> the bulleted lists in Section 3.2 to definition lists? See
> https://authors.ietf.org/en/rfcxml-vocabulary#dl.
>
> One example:
>
> Current:
>    *  VI is a time interval during which at least one of the performance
>       parameters degraded below its configurable optimal level
>       threshold.
>
>    *  SVI is a time interval during which at least one of the
>       performance parameters degraded below its configurable critical
>       threshold.
>
>    *  Consequently, VFI is a time interval during which all performance
>       parameters are at or better than their respective pre-defined
>       optimal levels.
>
> Perhaps:
>    VI: A time interval during which at least one of the performance
>       parameters degraded below its configurable optimal level
>       threshold.
>
>    SVI: A time interval during which at least one of the
>       performance parameters degraded below its configurable critical
>       threshold.
>
>    VFI: A time interval during which all performance
>       parameters are at or better than their respective pre-defined
>       optimal levels.
>
GIM>> I slightly prefer the current format, but I can live with the update.

> -->
>
>
> 12) <!-- [rfced] Should "the following section" here read "this section"?
> This
> sentence appears in Section 3.3; the following section is Section 4,
> which does not mention a state model.
>
> Original:
>    While the definition of a service state model is outside the scope of
>    this document, the following section provides some considerations for
>    how such a state model and accompanying configuration settings could
>    be defined.
>
> Perhaps:
>    While the definition of a service state model is outside the scope of
>    this document, this section provides some considerations for
>    how such a state model and accompanying configuration settings could
>    be defined.
>
GIM>> Thank you for your thoughtful consideration of the text. I agree with
the proposed update.

> -->
>
>
> 13) <!-- [rfced] Would it be helpful to update "YANG/NETCONF/RESTCONF
> framework"
> as follows? Or do you prefer the current?
>
> Original:
>    *  A YANG data model will allow PAM to be incorporated into
>       monitoring applications based on the YANG/NETCONF/RESTCONF
>       framework.
>
> Perhaps:
>    *  A YANG data model will allow PAM to be incorporated into
>       monitoring applications based on the YANG, NETCONF,
>       and RESTCONF frameworks.

GIM>> I agree with the proposed update.

>
> -->
>
>
> 14) <!-- [rfced] FYI - We rephrased this as follows to form a complete
> sentence. The other items in the list are complete sentences. Please
> review.
>
> Original:
>    *  The definition of the metrics that represent histograms for
>       service level parameters with buckets corresponding to individual
>       service level objectives,
>
> Updated:
>    *  Metrics can be defined to represent histograms for
>       service-level parameters with buckets corresponding to individual
>       SLOs..
>
GIM>> I agree with the proposed update.

> -->
>
>
> 15) <!-- [rfced] Please review "that should be maintained". Should this
> read "and
> should be maintained", or is the current okay? Also, please confirm that
> "violated time units" is correct here. We do not see this mentioned
> elsewhere in the document.
>
> Original:
>    The same service levels that
>    constitute SLO violations for one flow that should be maintained as
>    part of the "violated time units" and related metrics, may be
>    compliant for another flow.
>
> Perhaps:
>    The same service levels that
>    constitute SLO violations for one flow and should be maintained as
>    part of the "violated time units" and related metrics may be
>    compliant for another flow.

GIM>> I agree with the updated version.

>
> -->
>
>
> 16) <!-- [rfced] FYI - We updated this fragment as follows to create a
> complete
> sentence.
>
> Original:
>    By the same token, where the definition of what constitutes a
>    "severe" or a "significant" violation depends on configuration
>    settings or context.
>
> Perhaps:
>    By the same token, the definition of what constitutes a
>    "severe" or a "significant" violation depends on configuration
>    settings or context.

GIM>> I agree with the proposed update.

>
> -->
>
>
> 17) <!-- [rfced] Abbreviations
>
> a) We see both of the following as the expansion for PAM.
>
>   Precision Availability Metric (1 instance)
>   Precision Availability Metrics (8 instances)
>
> We updated to use the latter form (i.e., the form with "Metrics"). Please
> let
> us know any objections.
>
GIM>> Thank you for the question. I think that the use of singular, i.e.,
"Metric", in Acronyms is valid. Consequently, all acronyms in the document,
in my opinion, should be changed to "PAMs".

>
>
> b) We see two instances of "PAMs" in the document (see below). Since
> "Metrics"
> in the expansion is already plural, is the "s" needed in "PAMs"? Please
> review.
>
> Original:
>    To indicate a historic degree of precision availability, additional
>    derived PAMs can be defined as follows:
>    ...
>    It might be useful for a service provider to determine the current
>    condition of the service for which PAMs are maintained.

GIM>> If we agree that the acronym is expanded as "Precision Availability
Metric", then "PAMs" is the right form.

>
>
>
> c) OAM appears in the list of acronyms in Section 2.2 but is not mentioned
> elsewhere in the document. May we delete this term from the list?
>
GIM>> Yes, please remove it.

>
>
> d) FYI - We have added an expansion for the following abbreviation
> per Section 3.6 of RFC 7322 ("RFC Style Guide"). Please review each
> expansion in the document carefully to ensure correctness.
>
>   IP Flow Information Export (IPFIX)
>
GIM>> It seems that IPFIX is used only one time in the document. Do you
think that the expanded form without the acronym is sufficient? I'm okay
with any form.

> -->
>
>
> 18) <!-- [rfced] Please review the "Inclusive Language" portion of the
> online
> Style Guide <
> https://www.rfc-editor.org/styleguide/part2/#inclusive_language>
> and let us know if any changes are needed.
>
> Note that our script did not flag any words in particular, but this should
> still be reviewed as a best practice.
>
GIM>> I agree, don't find anything that requires rewording.

> -->
>
>
> 19) <!-- [rfced] We see a number of author-inserted comments in the .xml
> file for
> this document. We are unsure if these have been resolved. Please review
> and let us know if these can be deleted or if they need to be addressed.
>
GIM>> :-) My apologies for not cleaning our workspace properly. Thank you.

> -->
>
>
> Thank you.
>
> RFC Editor/rv
>
>
>
> On Feb 15, 2024, at 5:34 PM, [e-mail] wrote:
>
> *****IMPORTANT*****
>
> Updated 2024/02/15
>
> RFC Author(s):
> --------------
>
> Instructions for Completing AUTH48
>
> Your document has now entered AUTH48.  Once it has been reviewed and
> approved by you and all coauthors, it will be published as an RFC.
> If an author is no longer available, there are several remedies
> available as listed in the FAQ (https://www.rfc-editor.org/faq/).
>
> You and you coauthors are responsible for engaging other parties
> (e.g., Contributors or Working Group) as necessary before providing
> your approval.
>
> Planning your review
> ---------------------
>
> Please review the following aspects of your document:
>
> *  RFC Editor questions
>
>   Please review and resolve any questions raised by the RFC Editor
>   that have been included in the XML file as comments marked as
>   follows:
>
>   <!-- [rfced] ... -->
>
>   These questions will also be sent in a subsequent email.
>
> *  Changes submitted by coauthors
>
>   Please ensure that you review any changes submitted by your
>   coauthors.  We assume that if you do not speak up that you
>   agree to changes submitted by your coauthors.
>
> *  Content
>
>   Please review the full content of the document, as this cannot
>   change once the RFC is published.  Please pay particular attention to:
>   - IANA considerations updates (if applicable)
>   - contact information
>   - references
>
> *  Copyright notices and legends
>
>   Please review the copyright notice and legends as defined in
>   RFC 5378 and the Trust Legal Provisions
>   (TLP – https://trustee.ietf.org/license-info/).
>
> *  Semantic markup
>
>   Please review the markup in the XML file to ensure that elements of
>   content are correctly tagged.  For example, ensure that <sourcecode>
>   and <artwork> are set correctly.  See details at
>   <https://authors.ietf.org/rfcxml-vocabulary>.
>
> *  Formatted output
>
>   Please review the PDF, HTML, and TXT files to ensure that the
>   formatted output, as generated from the markup in the XML file, is
>   reasonable.  Please note that the TXT will have formatting
>   limitations compared to the PDF and HTML.
>
>
> Submitting changes
> ------------------
>
> To submit changes, please reply to this email using ‘REPLY ALL’ as all
> the parties CCed on this message need to see your changes. The parties
> include:
>
>   *  your coauthors
>
>   *  [e-mail] (the RPC team)
>
>   *  other document participants, depending on the stream (e.g.,
>      IETF Stream participants are your working group chairs, the
>      responsible ADs, and the document shepherd).
>
>   *  [e-mail], which is a new archival mailing list
>      to preserve AUTH48 conversations; it is not an active discussion
>      list:
>
>     *  More info:
>
> https://mailarchive.ietf.org/arch/msg/ietf-announce/yb6lpIGh-4Q9l2USxIAe6P8O4Zc
>
>     *  The archive itself:
>        https://mailarchive.ietf.org/arch/browse/auth48archive/
>
>     *  Note: If only absolutely necessary, you may temporarily opt out
>        of the archiving of messages (e.g., to discuss a sensitive matter).
>        If needed, please add a note at the top of the message that you
>        have dropped the address. When the discussion is concluded,
>        [e-mail] will be re-added to the CC list and
>        its addition will be noted at the top of the message.
>
> You may submit your changes in one of two ways:
>
> An update to the provided XML file
> — OR —
> An explicit list of changes in this format
>
> Section # (or indicate Global)
>
> OLD:
> old text
>
> NEW:
> new text
>
> You do not need to reply with both an updated XML file and an explicit
> list of changes, as either form is sufficient.
>
> We will ask a stream manager to review and approve any changes that seem
> beyond editorial in nature, e.g., addition of new text, deletion of text,
> and technical changes.  Information about stream managers can be found in
> the FAQ.  Editorial changes do not require approval from a stream manager.
>
>
> Approving for publication
> --------------------------
>
> To approve your RFC for publication, please reply to this email stating
> that you approve this RFC for publication.  Please use ‘REPLY ALL’,
> as all the parties CCed on this message need to see your approval.
>
>
> Files
> -----
>
> The files are available here:
>   https://www.rfc-editor.org/authors/rfc9544.xml
>   https://www.rfc-editor.org/authors/rfc9544.html
>   https://www.rfc-editor.org/authors/rfc9544.pdf
>   https://www.rfc-editor.org/authors/rfc9544.txt
>
> Diff file of the text:
>   https://www.rfc-editor.org/authors/rfc9544-diff.html
>   https://www.rfc-editor.org/authors/rfc9544-rfcdiff.html (side by side)
>
> Alt-diff of the text (allows you to more easily view changes
> where text has been deleted or moved):
>   https://www.rfc-editor.org/authors/rfc9544-alt-diff.html
>
> Diff of the XML:
>   https://www.rfc-editor.org/authors/rfc9544-xmldiff1.html
>
>
> Tracking progress
> -----------------
>
> The details of the AUTH48 status of your document are here:
>   https://www.rfc-editor.org/auth48/rfc9544
>
> Please let us know if you have any questions.
>
> Thank you for your cooperation,
>
> RFC Editor
>
> --------------------------------------
> RFC9544 (draft-ietf-ippm-pam-09)
>
> Title            : Precision Availability Metrics for Services Governed by
> Service Level Objectives (SLOs)
> Author(s)        : G. Mirsky, J. Halpern, X. Min, A. Clemm, J. Strassner,
> J. François
> WG Chair(s)      : Marcus Ihlar, Tommy Pauly
> Area Director(s) : Martin Duke, Zaheduzzaman Sarker
>
>