Re: [OPSAWG] New I-D -> Guidelines for Charactering "OAM"

[Carlos Pignataro <[e-mail]>]

Dear Greg,

Thank you for your interest in our draft, and the associated extensive
comments! All welcome!

*CMP: Please find some additional follow-ups.*

On Sat, Jan 13, 2024 at 7:52 PM Greg Mirsky <[e-mail]> wrote:

> Hi Adrian,
> thank you for your kind consideration of my notes. Please find my
> follow-up notes below tagged by GIM>>.
>
> Regards,
> Greg
>
> On Fri, Jan 12, 2024 at 11:25 AM Adrian Farrel <[e-mail]>
> wrote:
>
>> Hi Greg,
>>
>>
>>
>> Thanks for your thoughtful inspection of our draft.
>>
>>
>>
>> Carlos and I wanted to be sure that all of the discussions of this draft
>> were indexed on one list, and we wanted to avoid multiple copies going to
>> people who are subscribed to multiple lists. So we asked that follow-ups
>> went only to OPSAWG. I have moved IPPM, MPLS, SPRING, and DetNet to BCC on
>> this email.
>>
> GIM>> Thank you.
>
>>
>>
>> It was certainly not our intent to disparage any work that was being done
>> in any other working group, and I understand the effort that has gone into
>> the DetNet OAM Framework to ensure that the terminology is clear and
>> unencumbered in the DetNet context.
>>
>>
>>
>> Our concern was, however, that different contexts are applying different
>> definitions of the terms “in-band” and “out-of-band”. Those definitions are
>> (often) clear and precise, but they are not consistent across contexts.
>> Thus, a water-tight definition in the DetNet context is not universally
>> applicable, and a reader coming to DetNet from another context may bring
>> with them their own understanding of the terms.
>>
> GIM>> Although the wording in the DetNet OAM Framework is indeed specific
> for DetNet environment, for example, the reference to DetNet service
> sub-layer and PREOF, the authors and the WG strived to make the definitions
> generic. I believe that we achieved a reasonable level of generality
> because the interpretation of "in-band/out-of-band" terminology in RAW OAM
> was based on our work in DetNet. I believe that it could be a reasonable
> way of building common understanding through a discussion of existing terms
> instead of fork-lifting and trying to invent something different that has
> not been used by any WG.
>

CMP: The set of challenges continues to be that
*(1) these are still context-dependent re-definitions of
"in-band". draft-ietf-detnet-oam-framework says "*In-band OAM is [...]
within the monitored DetNet OAM domain [... long set of conditions ...]*".
Clearly, these are only within a DetNet context. These are not portable nor
generic or generalizable.*
*(2) the terms "in-band"/"out-of-band" are already tainted with a multitude
of potential meanings, and have interpretations attached to them. *
*(3) IOAM saw the same confusion with the I for "in-band", and instead of
redefining it with a narrower scope, it changed the term to a finer
resolution of the defintion and uniqueness (there is no band)*
CMP: Thus far, this does not show to me that the terms are confusion-safe.



>
>>
>> Our intent, therefore, is to select a finer-grained set of terms that
>> have universal applicability and that can be selected within a context
>> without loss of generality.
>>
> GIM>> I agree with that wholeheartedly.
>
>>
>>
>> This is a tricky little subject and I know that Carlos and I expected it
>> to generate more than a little discussion. If we end up with “everything is
>> OK and nothing needs to change” that will be OK with us. If we discover
>> that some work is using terms too generally, while others already have
>> perfect definitions, that will lead to something similar to this document
>> to bring the good into the light.
>>
>>
>>
>> Further comments in line…
>>
>>
>>
>>
>>
>> *From:* Greg Mirsky <[e-mail]>
>> *Sent:* 12 January 2024 00:09
>> *To:* Carlos Pignataro <[e-mail]>; Adrian Farrel <
>> [e-mail]>
>> *Cc:* Ops Area WG <[e-mail]>; IETF IPPM WG <[e-mail]>; mpls <
>> [e-mail]>; spring <[e-mail]>; DetNet WG <[e-mail]>
>> *Subject:* Re: [OPSAWG] New I-D -> Guidelines for Charactering "OAM"
>>
>>
>>
>>    - Hi Carlos and Adrian,
>>
>> thank you for starting this work. I believe that having a common
>> dictionary helps in having more productive discussions. I took the liberty
>> of inviting all the WGs which you invited to review the document and added
>> DetNet WG. Please feel free to trim the distribution list.
>>
>> I've read the document and have several general questions to start our
>> discussion:
>>
>>    - It seems that the motivation of this document is the assumption
>>    that "in-band OAM" and "out-of-band OAM" are not representative or cannot
>>    be understood or correctly attributed, interpreted by the IETF community.
>>    Is that correct?
>>
>> I think the wording here would be “cannot be reliably understood
>> consistently”. That is, without looking at a context-specific definition
>> (such as that which you supply in the DetNet OAM Framework), the use of the
>> terms may be misinterpreted.
>>
>> This is an assertion, but one (we think) is founded on observation of
>> recent conversations on mailing lists, and also of witnessing many years of
>> people talking passed each other.
>>
>
CMP: Further:
(1) Cannot be understood unambiguously.
(2) Cannot be interpreted by someone who first hears the terms (i.e.,
"there is no band")
CMP: I added some of this text to the draft.


>>    - As we discuss and try to establish (change) the IETF dictionary, it
>>    is important to analyze the terminology used by other SDOs. I believe that
>>    it is beneficial to maintain consistent terminology which will minimize
>>    misunderstandings among experts with different experiences of working at
>>    different centers of technological expertise.
>>
>> This is a good point. It is certainly true that if other SDOs working
>> with packet networks have established terminology that we can agree with
>> and which is not, itself, subject to context-specific definitions, then
>> there is no reason to choose other terms. Do you have any suggested sources?
>>
> IEEE 802.1Q 2014 uses in-band/out-of-band
>

CMP: Unless somehow I missed it, this (paid, 1832 page) document does not
use the terms "in-band OAM" or "out-of-band OAM". It mentions in-band
loopback, and in-band management. As such, I do not think they are
relevant, nor solve the IETF confusion.



> It is notable that the ITU-T has long worked with non-packet transport
>> networks and has used the terms in-band and out-of-band. But even there we
>> see some fragmentation with terms such as “in-fibre, but out-of-band”
>> becoming necessary.
>>
>>    - I that DetNet OAM Framework
>>    <https://datatracker.ietf.org/doc/draft-ietf-detnet-oam-framework/>
>>    provides sufficiently clear interpretation of terms that can be generalized
>>    for non-DetNet networks:
>>
>>    *  In-band OAM is an active OAM that is in-band within the monitored
>>
>>       DetNet OAM domain when it traverses the same set of links and
>>
>>       interfaces receiving the same QoS and Packet Replication,
>>
>>       Elimination, and Ordering Functions (PREOF) treatment as the
>>
>>       monitored DetNet flow.
>>
>>
>>
>>
CMP: I strongly disagree with this definition gening easy to generalize.
This is not only 'within a DetNet OAM domain', but also:
(1) "same set of links and interfaces" --> how are 'links' and 'interfaces'
defined here, include 'tunnels' or not, etc... looking at rfc 8343...
(2) "...receiving the same QoS and PREOF treatment..." --> that also seems
specific and not generalizable. What if you want to define OAM that follows
the same topology but not same QoS treatment? Or an "or" of those...
*(3) how does this generalize to in-situ?*


> This, of course, does not distinguish between “in-packet” (such as IOAM),
>> and having its own packet (such as ping).
>>
>> GIM>> The definition is for active OAM. Hybrid OAM, which, as I
> understand it, is referred to in the document as "in-packet", is inherently
> "in-band" with the monitored data flow. Looking back, perhaps we could have
> noted that in DetNet OAM Framework. Furthermore, I note that what is
> referred to as "in-packet" is identified as "on-path telemetry". What do
> you think about that term?
>

CMP: Thanks for agreeing that the DetNet OAM framework has a gap in regards
to in-situ.
CMP: I would not use "telemetry", but "on-path OAM" can work -- so long as
it is not "in-band" due to the confusion experienced.
CMP: If there is a path defined, then "on-path" or "in-path" can work
perfectly. If there were a "band" we could use "in-band", but...


>
>>
>>    *  Out-of-band OAM is an active OAM whose path through the DetNet
>>
>>       domain is not topologically identical to the path of the monitored
>>
>>       DetNet flow, or its test packets receive different QoS and/or
>>
>>       PREOF treatment, or both.
>>
>> As can be seen, the interpretation of "in-band" accepted by the DetNet WG
>> includes not only topological equivalence between the monitored data flow
>> and path traversed by active OAM but also QoS equivalence between them. I
>> believe that is essential in differentiating in-band OAM from out-of-band
>> OAM.
>>
>>
>>
>> Right. But is there terminology to talk about a packet that does follow
>> the topology, but does not receive the same QoS treatment?
>>
>> GIM>> Is there a case of useful information that an operator obtains in
> that scenario, i.e., when a test packet is topologically with the monitored
> flow but is marked by a different CoS and, as a result, receives a
> different QoS treatment? In other words, can topology and CoS marking be
> different between the monitored data flow and specially constructed test
> packets that are aimed to monitor that data flow? Personally, I am not
> aware of any useful information (except for direct loss measurement) that
> can be obtained using that setup and I conclude that active OAM must repeat
> topology of the data paths and use the same CoS marking.
>

CMP: Greg, I feel you have not answered the question -- which is the same
one I posed above. What would you call OAM following topology but not
receiving (necessarily) the same QoS treatment? I believe that "in-band
type 27" is not descriptive, and finer granularity.

>
>>
>> It is perhaps a little strong to say this, but what you have done is
>> define two classes of OAM (both good things and meaningful in the DetNet
>> context) and then assigned existing names to those classes. What we are
>> suggesting is that you have some finer granularity categorisation of OAM
>> available, and that for the purposes of your DetNet work, you are
>> collecting those granularities into two different sets.
>>
>> GIM>> As I noted above, I believe that topology and CoS are both
> requirements for an active OAM method and separating them "throws baby out
> of water".
>

CMP: I understood something different in Adrian's point. You have created a
new definition, within Detnet, one that amalgamates a number of criteria
together, but then assigned them an existing term, "in-band OAM"...

>
>>    - I find the use of "path congruence" in inherently meshed
>>    packet-switched networks confusing if not misleading. (Note that RFC
>>    6669 <https://www.rfc-editor.org/rfc/rfc6669> explains congruence by
>>    using in-band term.) Is there evidence of the term being used besides a
>>    single case in RFC 6669?
>>
>> Well, I would say that 6669 is an example of how “in-band” has been used,
>> and I’d point out that it does not match your DetNet OAM Framework
>> definition (as there is no mention of identical treatment). Note that the
>> text from RFC 6669 is replicated into RFC 7276 (same authors, same subject).
>>
>> You don’t say what you find misleading or confusing. Is it that, in a
>> meshed packet network, each individual packet might be forwarded
>> differently so that congruence cannot be guaranteed? That could be true,
>> but we hope for greater stability than that, I think.
>>
>> If “path congruence” was a new term (with only one previous use) that
>> might make it a really neat term to use (because it would lack all previous
>> meanings). However, it is not. It has been used (to mean the same set of
>> links, ports, and nodes) in our more path-oriented work such as RFC 5828,
>> RFC 6373, right up to RFC 9059.
>>
>> Perhaps “congruent” is overloaded given that we are not talking about
>> “topological congruence”, a term that is also quite widely used (e.g., RFC
>> 2796, RFC 4258, RFC 5059, RFC 6549, RFC 8795)
>>
> GIM>> My concern with using "congruence" is most likely caused by how the
> term is used in geometry.And saying that "congruent paths" are paths that
> cross the same set of nodes and links seems like too narrow compared to the
> definition in geometry. And that raises my next question: Why not simply
> define the relationship between the data path and the path traversed by a
> test packet without introducing, what seems, an unnecessary term?
>

CMP: A more fitting term would most certainly be welcomed! The objective is
that it does not redefine "in-band" for example (RFC 6669 and DetNet
already have different definitions for the same term!)


>
>>    - Similarly, "in-packet" vs. "dedicated packet". I believe that RFC
>>    7799 <https://www.rfc-editor.org/rfc/rfc7799> has that addressed by
>>    using "active", "passive", and "hybrid" terminology. Although these terms
>>    applied to measurement methods, i.e., performance monitoring component of
>>    OAM, but, in my opinion, can be extended to fault management OAM.
>>
>> Well, we agree that RFC 7799 can be used to generate the terms "active
>> OAM", "passive OAM", and "hybrid OAM". Although we think, for the benefit
>> of clarity, the reader should not be left to examine RFC 7799 and project
>> meaning from performance monitoring to OAM in general: they should be
>> presented with a clear set of definitions (per our section 3).
>>
>> It is further our belief that the definitions of active and passive OAM
>> do not match with “in-packet” and “dedicated-packet”. Indeed, possibly, the
>> closest is that “active OAM” is “dedicated-packet”, and “hybrid OAM” is
>> “in-packet” leaving “passive OAM” to be just observation.
>>
> GIM>> I consider OAM to define a toolbox that contains tools based on
> active, passive, and hybrid methods in support of OAM functions
> (connectivity check, continuity verification, automatic protection
> switchover, and performance monitoring among others). Thus, I prefer to use
> the terminology of RFC 7799 that classifies measurement methods, not OAMs.
>

CMP: Greg, this does not seem to drive us towards a path of more clarity.
Let me explain.
*(1) First, RFC 7799 defines Hybrid Type I and Hybrid Type II (methods and
metrics). This in itself is already overloading when you are using only
"hybrid".*
*(2) Second, let's look at this text from RFC7799:*
3.7.  *Passive Metric*
   Passive Metrics apply to observations of packet traffic (traffic
   flows in [RFC7011]).
   Passive performance metrics are assessed independently of the packets
   or traffic flows, and solely through observation.
*Some refer to such   assessments as "out of band".*

CMP: So based on that trajectory, *passive can be out-of-band*, but your
DetNet document says "*Out-of-band OAM is an active OAM*".

CMP: Frankly, the more we look at it, the clearer it becomes that *-band
can lead to confusions...


>>    - It seems like the definition of Compound/Combined misses the point
>>    that RFC 7799 already defines a hybrid measurement method (not OAM but
>>    measurement methods) as a method in which elements of active and
>>    passive measurement methods are used. Hence, hybrid is already a
>>    combination of active and passive measurement methodologies and the
>>    introduction of compound or combined terms is unnecessary, a duplication of
>>    the existing and accepted terminology (at least in IPPM WG). And
>>    "Active-Hybrid-Passive OAM" is the result of that omission because,
>>    according to the definition in RFC 7799, Active-Passive is Hybrid. Thus,
>>    Active-Hybrid-Passive is nothing else but Hybrid-Hybrid. Does that make
>>    sense?
>>
>> I should certainly have preferred it had RFC 7799 not used the term
>> “hybrid” to actually refer to a third category that is not a hybrid of the
>> first two categories. For the definitions of active OAM and passive OAM, I
>> don’t think the combination matches the definition of hybrid OAM.
>>
>> So, perhaps, let’s stop referring to RFC 7799’s definitions of
>> not-actually-OAM-packets, and nail down our own definitions. That will tell
>> us whether we need two, three, four, or more terms.
>>
> GIM>> I would note that the terminology introduced in RFC 7799 has been
> accepted and broadly used not only in the documents produced by IPPM WG but
> also many groups in the Routing area.
>

CMP: Even more reason to readjust trajectory and prevent further confusion.

>
>>    - I cannot agree that RFC 7799 "adds to the confusion" by pointing
>>    that
>>
>>    Passive performance metrics are assessed independently of the packets
>>
>>    or traffic flows, and solely through observation.  Some refer to such
>>
>>    assessments as "out of band".
>>
>> Indeed, passive measurement methods are not required to use packets that
>> are in-band with the monitored data flow. Usually, the management plane
>> protocol is used to collect, to perform the observation function. In some
>> cases, in-band active OAM packets may be used, e.g., direct loss
>> measurement in ETH-OAM.
>>
>>
>>
>> Yes, but where is this “in-band with the monitored data flow” defined for
>> a packet network? And you say “are not required to” rather than “MUST NOT”.
>> That means that the passive methods might send their packets with the
>> monitored data flow or might not.
>>
>> We live in a world where there is not necessarily a distinction between
>> the MCN and DCN.
>>
>> GIM>> Although there might be no topological distinction between MCN and
> DCN, it is more likely that they use different CoS markings. If that is the
> case, from the point of the definitions in DetNet OAM Framework, MCN is
> out-of-band relative to DCN.
>

CMP: Do you feel that is a clear unambiguous use of terms?

>
>>
>> I find that throw-away sentence in RFC 7799 both helpful and unhelpful.
>> It is helpful to know that some people call this “out of band”. It is
>> unhelpful to refer to an assessment method as “out of band” as there is no
>> message or packet involved to be in or out of band.
>>
>>
>>
>> FWIW, I believe that RFC 7799 and DetNet OAM Requirements already provide
>> a clear terminology for OAM in general and its elements, i.e., Fault
>> Management and Performance Monitoring.
>>
>>
>>
>
CMP: Thanks for sharing your beliefs. Our observations show otherwise.


> OK. I suspect that we are going to have to come up with a set of OAM
>> techniques and ask you to categorise them according to your terminology to
>> see whether all bases are covered.
>>
>>
>>
>> But I am also going to have to review your text from the DetNet OAM
>> Framework because it contains phrases that are not clear (to me)…
>>
>>
>>
>>       In-band OAM is an active OAM that is in-band within the monitored
>>
>>       DetNet OAM domain when it traverses the same set of links and
>>
>>       interfaces receiving the same QoS and Packet Replication,
>>
>>       Elimination, and Ordering Functions (PREOF) treatment as the
>>
>>       monitored DetNet flow.
>>
>>
>>
>> There is something broken here. Maybe too many words. Perhaps you mean…
>>
>>
>>
>>       In-band OAM is an active OAM that traverses the same set of links
>> and
>>
>>       interfaces receiving the same QoS and Packet Replication,
>>
>>       Elimination, and Ordering Functions (PREOF) treatment as the
>>
>>       monitored DetNet flow within the monitored DetNet OAM domain
>>
>
>>
>> …and…
>>
>>
>>
>>       Out-of-band OAM is an active OAM whose path through the DetNet
>>
>>       domain is not topologically identical to the path of the monitored
>>
>>       DetNet flow, or its test packets receive different QoS and/or
>>
>>       PREOF treatment, or both.
>>
> GIM>> Many thanks for your thoughtful suggestion. I'll make sure that we
> apply this change in the course of AUTH48.
>
>>
>>
>> As noted before, this leaves a few gaps.
>>
>>    - Active OAM that follows the same path, but does not receive the
>>    same QoS treatment
>>
>> GIM>> That would be out-of-band
>
>>
>>    -
>>    - There is no distinction between instrumentation of data packets and
>>    dedicated instrumentation packets
>>
>> GIM>> That is the distinction between hybrid and active OAM methods.
>

CMP: I would qualify this as retrofitting existing terms into
narrowscoped contexts, instead of building unambiguous terms for the future.

Thanks,

Carlos..


>
>>    -
>>
>>
>>
>> Cheers,
>>
>> Adrian
>>
>>
>>
>> On Fri, Jan 5, 2024 at 12:39 PM Carlos Pignataro <[e-mail]>
>> wrote:
>>
>> Hi, Ops Area WG,
>>
>>
>>
>> Every now and again, there are discussions on how to best characterize or
>> qualify a particular kind of "OAM", as well as misunderstandings due to
>> having different definitions and contexts for a given term. A case in point
>> is "in-band" or "out-of-band" OAM, as recently surfaced at
>> https://mailarchive.ietf.org/arch/msg/opsawg/jREEH1sFOZ-uxZNky-RTggpxkuk/
>> .
>>
>>
>>
>> To alleviate this issue, Adrian and I wrote a short I-D to provide
>> forward-looking guidance on "foobar OAM".
>>
>>
>>
>> We would appreciate feedback and input on this position, which aims at
>> updating the guidelines for the "OAM" acronym, with unambiguous guidelines
>> for their modifiers.
>>
>>
>>
>> Guidelines for Charactering "OAM":
>>
>>
>> https://datatracker.ietf.org/doc/draft-pignataro-opsawg-oam-whaaat-question-mark/
>>
>>
>>
>> Look forward to input and comments to make this more clear and effective!
>>
>>
>>
>> Adrian & Carlos.
>>
>>
>>
>>
>>
>> _______________________________________________
>> OPSAWG mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/opsawg
>>
>>