Re: [MBONED] WGLC for draft-ietf-mboned-multicast-telemetry

[Hitoshi Asaeda <[e-mail]>]

Hi Toerless,

I reply for your Mtrace2’s comment.

> 352 5.1.  Mtrace verson 2
> 
> 354    Mtrace version 2 (Mtrace2) [RFC8487] is a protocol that allows the
> 355    tracing of an IP multicast routing path.  Mtrace2 provides additional
> 356    information such as the packet rates and losses, as well as other
> 357    diagnostic information.  Unlike unicast traceroute, Mtrace2 traces
> 358    the path that a packet would take from a source to a receiver.  It is
> 
> Not really, right ? Mtrace2 traces the path that the tree building messages
> follow from receiver to source. The reverse of this traced path can then be
> assumed to be the path taken by multicast packets on the branch of the tree
> to the receiver fdrom which the Mtrace2 was started.

You are right. I don’t see any contradiction of your statement and the statement written in the draft.

Mtrace2 request packets are forwarded from a receiver to a source, while it enables the receiver to trace the path for a packet forwarded from the source to the receiver after receiving the Mtrace2 reply as the forwarding path is constructed with the RPF method.

Regards,

Hitoshi


> On Sep 21, 2023, at 10:47, Toerless Eckert <[e-mail]> wrote:
> 
> On Thu, Sep 14, 2023 at 08:01:31AM -0700, Leonard Giuliano wrote:
>> MBONED (and IPPM),
>> 
>> We would like to officially begin working group last call for Multicast
>> On-path Telemetry using IOAM.  We had a previous WGLC for this doc earlier
>> this year, and at the time there was not enough response from the WG to
>> support advancement, but the draft has since been updated and in the meeting
>> in IETF117 there was support in the room for another WGLC.  It was also
>> suggested that IPPM be included for their input and review.
>> 
>> Please post whether you support/oppose the advancement of this draft as well
>> as any comments you may have to the list by Sept 29.
>> 
>> Most recent version of the draft can be found here:
>> https://datatracker.ietf.org/doc/draft-ietf-mboned-multicast-telemetry/
>> 
>> -MBONED Chairs
> 
> Thanks a lot folks for the work. Some comments, mostly textual inline below in
> idnits output. I did not check the idnits header created. One terminology / functional
> suggestion about the use of pre-existing rfc9197 terminology (ingress/egress interfaces)
> as well as tracing the ingress interface as well.
> 
> 
> idnits 2.17.1 
> 
> draft-ietf-mboned-multicast-telemetry-07.txt:
> 
>  Checking boilerplate required by RFC 5378 and the IETF Trust (see
>  https://trustee.ietf.org/license-info):
>  ----------------------------------------------------------------------------
> 
>     No issues found here.
> 
>  Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt:
>  ----------------------------------------------------------------------------
> 
>     No issues found here.
> 
>  Checking nits according to https://www.ietf.org/id-info/checklist :
>  ----------------------------------------------------------------------------
> 
>  == There are 1 instance of lines with multicast IPv4 addresses in the
>     document.  If these are generic example addresses, they should be changed
>     to use the 233.252.0.x range defined in RFC 5771
> 
> 
>  Miscellaneous warnings:
>  ----------------------------------------------------------------------------
> 
>  -- The document date (6 September 2023) is 15 days in the past.  Is this
>     intentional?
> 
> 
>  Checking references for intended status: Proposed Standard
>  ----------------------------------------------------------------------------
> 
>     (See RFCs 3967 and 4897 for information about using normative references
>     to lower-maturity documents in RFCs)
> 
>  == Missing Reference: 'A' is mentioned on line 220, but not defined
> 
>  -- Looks like a reference, but probably isn't: '0' on line 220
> 
>  -- Looks like a reference, but probably isn't: '1' on line 220
> 
>  == Missing Reference: 'RFC XXXX' is mentioned on line 452, but not defined
> 
>  == Unused Reference: 'RFC4687' is defined on line 469, but no explicit
>     reference was found in the text
> 
>  == Unused Reference: 'RFC6037' is defined on line 482, but no explicit
>     reference was found in the text
> 
>  == Unused Reference: 'I-D.xie-bier-ipv6-encapsulation' is defined on line
>     590, but no explicit reference was found in the text
> 
>  ** Downref: Normative reference to an Informational RFC: RFC 4687
> 
>  ** Downref: Normative reference to an Historic RFC: RFC 6037
> 
> 
>     Summary: 2 errors (**), 0 flaws (~~), 6 warnings (==), 3 comments (--).
> 
>     Run idnits with the --verbose option for more detailed information about
>     the items above.
> 
> --------------------------------------------------------------------------------
> 
> 
> 2 MBONED                                                           H. Song
> 3 Internet-Draft                                                M. McBride
> 4 Intended status: Standards Track                  Futurewei Technologies
> 5 Expires: 9 March 2024                                          G. Mirsky
> 6                                                                ZTE Corp.
> 7                                                                G. Mishra
> 8                                                             Verizon Inc.
> 9                                                                H. Asaeda
> 10                                                                     NICT
> 11                                                                  T. Zhou
> 12                                                      Huawei Technologies
> 13                                                         6 September 2023
> 
> 15                  Multicast On-path Telemetry using IOAM
> 16                 draft-ietf-mboned-multicast-telemetry-07
> 
> 18 Abstract
> 
> 20    This document specifies the requirements of on-path telemetry for
> 21    multicast traffic using In-situ OAM.  While In-situ OAM is
> 22    advantageous for multicast traffic telemetry, some unique challenges
> 23    present.  This document provides the solutions based on the In-situ
> 
> "some unique challenges present" - does not parse.
> 
> 24    OAM trace option and direct export option to support the telemetry
> 25    data correlation and the multicast tree reconstruction without
> 26    incurring data redundancy.
> 
> 28 Requirements Language
> 
> 30    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
> 31    "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
> 32    "OPTIONAL" in this document are to be interpreted as described in BCP
> 33    14 [RFC2119][RFC8174] when, and only when, they appear in all
> 34    capitals, as shown here.
> 
> 36 Status of This Memo
> 
> 38    This Internet-Draft is submitted in full conformance with the
> 39    provisions of BCP 78 and BCP 79.
> 
> 41    Internet-Drafts are working documents of the Internet Engineering
> 42    Task Force (IETF).  Note that other groups may also distribute
> 43    working documents as Internet-Drafts.  The list of current Internet-
> 44    Drafts is at https://datatracker.ietf.org/drafts/current/.
> 
> 46    Internet-Drafts are draft documents valid for a maximum of six months
> 47    and may be updated, replaced, or obsoleted by other documents at any
> 48    time.  It is inappropriate to use Internet-Drafts as reference
> 49    material or to cite them other than as "work in progress."
> 
> 51    This Internet-Draft will expire on 9 March 2024.
> 
> 53 Copyright Notice
> 
> 55    Copyright (c) 2023 IETF Trust and the persons identified as the
> 56    document authors.  All rights reserved.
> 
> 58    This document is subject to BCP 78 and the IETF Trust's Legal
> 59    Provisions Relating to IETF Documents (https://trustee.ietf.org/
> 60    license-info) in effect on the date of publication of this document.
> 61    Please review these documents carefully, as they describe your rights
> 62    and restrictions with respect to this document.  Code Components
> 63    extracted from this document must include Revised BSD License text as
> 64    described in Section 4.e of the Trust Legal Provisions and are
> 65    provided without warranty as described in the Revised BSD License.
> 
> 67 Table of Contents
> 
> 69    1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
> 70    2.  Requirements for Multicast Traffic Telemetry  . . . . . . . .   4
> 71    3.  Issues of Existing Techniques . . . . . . . . . . . . . . . .   4
> 72    4.  Modifications to Existing Solutions . . . . . . . . . . . . .   5
> 73      4.1.  Per-hop postcard using IOAM DEX . . . . . . . . . . . . .   5
> 74      4.2.  Per-section postcard for IOAM Trace . . . . . . . . . . .   7
> 75    5.  Application Considerations for Multicast Protocols  . . . . .   8
> 76      5.1.  Mtrace verson 2 . . . . . . . . . . . . . . . . . . . . .   8
> 77      5.2.  Application in PIM  . . . . . . . . . . . . . . . . . . .   9
> 78      5.3.  Application of MVPN X-PMSI Tunnel Encapsulation
> 79            Attribute . . . . . . . . . . . . . . . . . . . . . . . .   9
> 80      5.4.  Application in BIER . . . . . . . . . . . . . . . . . . .  10
> 81    6.  Security Considerations . . . . . . . . . . . . . . . . . . .  10
> 82    7.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  10
> 83    8.  Acknowledgments . . . . . . . . . . . . . . . . . . . . . . .  10
> 84    9.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  10
> 85      9.1.  Normative References  . . . . . . . . . . . . . . . . . .  10
> 86      9.2.  Informative References  . . . . . . . . . . . . . . . . .  12
> 87    Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  13
> 
> 89 1.  Introduction
> 
> 91    Multicast is used by residential broadband customers across operator
> 
> suggest /used by/used for example by, but not limited to/
> aka: there's a lot more.
> 
> 92    networks, private MPLS customers, and internal customers within
> 93    corporate intranet.  Multicast provides real time interactive online
> 94    meetings or podcasts, IPTV, and financial markets real-time data,
> 95    which all have a reliance on UDP's unreliable transport.  End-to-end
> 96    QOS, therefore, should be a critical component of multicast
> 97    deployment in order to provide a good end user experience.  In
> 98    multicast video streaming, if a packet is dropped, and that packet
> 99    happens to be a reference frame (i.e., I-Frame) in the video feed,
> 100    the client receiver of the multicast feed goes into buffering mode
> 101    resulting in a frozen window. 
> 
> Suggest rewording of last sentence:
> 
> In multicast real-time media streaming, loss of a single packet containing a
> reference frame can result in the inability of all thousands of receivers to
> decode a whole so-called Group-of-Picture sequence of packets, introducing
> black picture for periods often as much as 4 seconds. Unexpected long delay
> in propagation of a packet in such real-time media streaming may equally result
> oin the packet not being received and cxreating the same results.
> 
> 101    Multicast packet drops and delay can
> 
> ...therefore
> 
> 102    severely affect the application performance and user experience.
> 
> 104    It is important to monitor the performance of the multicast traffic.
> 105    New on-path telemetry techniques such as In-situ OAM (IOAM)
> 106    [RFC9197], IOAM Direct Export (DEX) [RFC9326] IOAM Marking-based
> 107    Postcard (PBT-M) [I-D.song-ippm-postcard-based-telemetry], and Hybrid
> 108    Two-Step (HTS) [I-D.mirsky-ippm-hybrid-two-step] are useful and
> 109    complementary to the existing active OAM performance monitoring
> 110    methods, provide promising means to directly monitor the network
> 
> "existing active OAM performance monitoring" - such as ? One or two references
> would help.
> 
> 111    experience of multicast traffic.  However, multicast traffic has some
> 112    unique characteristics which pose some challenges on applying such
> 113    techniques in an efficient way.
> 
> 115    The IP multicast packet data for a particular (S, G) state is
> 116    identical from one branch to another on its way to multiple
> 117    receivers.  When adding IOAM trace data to multicast packets, each
> 118    replicated packet would keep the telemetry data for its entire
> 119    forwarding path.  Since the replicated packets all share common path
> 120    segments, redundant data will be collected for the same original
> 121    multicast packet.  Such redundancy consumes extra network bandwidth
> 122    unnecessarily.  For a large multicast tree, such redundancy is
> 123    considerable.  Alternatively, it could be more efficient to collect
> 124    the telemetry data using solutions such as IOAM DEX to eliminate the
> 125    data redundancy.  However, IOAM DEX is lack of a branch identifier,
> 
> s/is lack of/lacks/
> 
> 126    making telemetry data correlation and multicast-tree reconstruction
> 127    difficult.
> 
> 129    This draft provides two solutions to the IOAM data redundancy problem
> 130    based on the IOAM standards.  The requirements for multicast traffic
> 131    telemetry are discussed along with the issues of the existing on-path
> 132    telemetry techniques.  We propose modifications to make these
> 133    techniques adapt to multicast in order for the original multicast
> 134    tree to be correctly reconstructed while eliminating redundant data.
> 
> 136 2.  Requirements for Multicast Traffic Telemetry
> 
> 138    Multicast traffic is forwarded through a multicast tree.  With PIM
> 139    and P2MP (e.g., MLDP, RSVP-TE) the forwarding tree is established and
> 
> Please provide expansion of terms and references for them on first use, aka: for
> MLDP, RSVP-TE here (not later). Please check for all abbreviations in the document,
> i may overlook them as i am not doing a thorough search.
> 
> 140    maintained by the multicast routing protocol.  With BIER, no state is
> 141    created in the network to establish a forwarding tree; instead, a
> 142    bier header provides the necessary information for each packet to
> 143    know the egress points.  Multicast packets are only replicated at
> 
> And with BIER-TE the replication points too ;-))
> 
> 144    each tree branch fork node for efficiency.
> 
> 146    There are several requirements for multicast traffic telemetry, a few
> 147    of which are:
> 
> 149    *  Reconstruct and visualize the multicast tree through data plane
> 150       monitoring.
> 
> 152    *  Gather the multicast packet delay and jitter performance on each
> 153       path.
> 
> 155    *  Find the multicast packet drop location and reason.
> 
> 157    *  Gather the VPN state and tunnel information in case of P2MP
> 158       multicast.
> 
> 160    In order to meet these requirements, we need the ability to directly
> 161    monitor the multicast traffic and derive data from the multicast
> 162    packets.  The conventional OAM mechanisms, such as multicast ping and
> 163    trace, are not sufficient to meet these requirements.
> 
> references for ping and trace here please.
> 
> 165 3.  Issues of Existing Techniques
> 
> 167    On-path Telemetry techniques that directly retrieve data from
> 168    multicast traffic's live network experience are ideal for addressing
> 169    the aforementioned requirements.  The representative techniques
> 170    include In-situ OAM (IOAM) Trace option [RFC9197], IOAM Direct Export
> 171    (DEX) option [RFC9326], and PBT-M
> 172    [I-D.song-ippm-postcard-based-telemetry].  However, unlike unicast,
> 173    multicast poses some unique challenges to applying these techniques.
> 
> 175    Multicast packets are replicated at each branch fork node in the
> 176    corresponding multicast tree.  Therefore, there are multiple copies
> 177    of the original multicast packet in the network.
> 
> 179    If the IOAM trace option is used for on-path data collection, the
> 180    partial trace data will also be replicated into the copy for each
> 181    branch.  The end result is that, at the multicast tree leaves, each
> 182    copy of the multicast packet has a complete trace.  Most of the data
> 183    (except data from the last leaf branch), however, has redundant
> 184    copies.  Data redundancy introduces unnecessary header overhead,
> 185    wastes network bandwidth, and complicates the data processing.  The
> 186    larger the multicast tree is or the longer the multicast path is, the
> 187    more severe the redundancy problem becomes.
> 
> It is not clear to me if i am incorrectly understanding how IAM trace is supposed to work.
> I thought that it would allow to collect trace information from more than one hop along the
> path. So if i assume there is replication on every hop with multicast, and collection of
> trace data on every hop, then the packet on every receiver would still have different
> trace data, and only initial parts of the trace would be the same. But if lets
> say i am multicast to 100 receivers, i am still saving a huge amount of data on
> the wire by multicasting as compared to the source sending 100 unicast packets.
> 
> Whether this is true or false, it seems such a simple example to summarize how
> IOAM trace works would help the text.
> 
> 189    The postcard-based solutions (e.g., IOAM DEX), can be used to
> 190    eliminate such data redundancy, because each node on the tree only
> 191    sends a postcard covering local data.  However, they cannot track and
> 192    correlate the tree branches properly due to the lack of branching
> 193    information, so they can bring confusion about the multicast tree
> 194    topology.  For example, in a multicast tree, Node A has two branches,
> 195    one to Node B and the other to node C; further, Node B leads to Node
> 196    D and Node C leads to Node E.  When applying postcard-based methods,
> 197    one cannot tell whether or not Node D(E) is the next hop of Node B(C)
> 198    from the received postcards.
> 
> Unless one correlates the exporting nodes with knowledge about the tree collected
> by other means, such as mtrace or SNMP/YANG. Aka: May be useful to include an
> argument why such correlation is undesirable or inferior to what's proposed here.
> 
> May i also note that BIER-TE, RFC9262 does contain the tree information in the
> packet header, so it would be possible to directly deduce the tree a packet was
> passed over when correlating received IOAM DEX information.
> 
> 200    The fundamental reason for this problem is that there is not an
> 201    identifier (either implicit or explicit) to correlate the data on
> 202    each branch.
> 
> 204 4.  Modifications to Existing Solutions
> 
> 206    We provide two solutions to address the above issues.  One is based
> 207    on IOAM DEX and requires an extension to the instruction header of
> 208    the IOAM DEX Option.  The second solution combines the IOAM trace
> 209    option and postcards for redundancy removal.
> 
> 211 4.1.  Per-hop postcard using IOAM DEX
> 
> 213    One way to mitigate the postcard-based telemetry's tree tracking
> 214    weakness is to augment it with a branch identifier field.  Note that
> 215    this works for the IOAM DEX option but not for PBT-M because the IOAM
> 216    DEX option has an instruction header which can be used to hold the
> 217    branch identifier.  To make the branch identifier globally unique,
> 218    the branch fork node ID plus an index is used.  For example, Node A
> 219    has two branches: one to Node B and the other to Node C.  Node A will
> 220    use [A, 0] as the branch identifier for the branch to B, and [A, 1]
> 221    for the branch to C.  The identifier is carried with the multicast
> 222    packet until the next branch fork node.  Each node MUST export the
> 223    branch identifier in the received IOAM DEX header in the postcards it
> 224    sends.  The branch identifier, along with the other fields such as
> 225    flow ID and sequence number, is sufficient for the data collector to
> 226    reconstruct the topology of the multicast tree.
> 
> 228    Figure 1 shows an example of this solution.  "P" stands for the
> 229    postcard packet.  The square brackets contains the branch identifier.
> 230    The curly brace contains the telemetry data about a specific node.
> 
> 232      P:[A,0]{A}  P:[A,0]{B} P:[B,1]{D}  P:[B,0]{C}   P:[B,0]{E}
> 233           ^            ^          ^        ^           ^
> 234           :            :          :        :           :
> 235           :            :          :        :           :
> 236           :            :          :      +-:-+       +-:-+
> 237           :            :          :      |   |       |   |
> 238           :            :      +---:----->| C |------>| E |-...
> 239         +-:-+        +-:-+    |   :      |   |       |   |
> 240         |   |        |   |----+   :      +---+       +---+
> 241         | A |------->| B |        :
> 242         |   |        |   |--+   +-:-+
> 243         +---+        +---+  |   |   |
> 244                             +-->| D |--...
> 245                                 |   |
> 246                                 +---+
> 
> 248                          Figure 1: Per-hop Postcard
> 
> 250    Each branch fork node needs to generate a unique branch identifier
> 251    (i.e., branch ID) for each branch in its multicast tree instance and
> 252    include it in the IOAM DEX option header.  The branch ID remains
> 253    unchanged until the next branch fork node.  The branch ID contains
> 254    two parts: the branch fork node ID and an interface index.
> 
> 256    Conforming to the node ID specification in IOAM [RFC9197], the node
> 257    ID is a 3-octet unsigned integer.  The interface index is a two-octet
> 258    unsigned integer.  As shown in Figure 2, the branch ID consumes 8
> 259    octets in total.  The three unused octets MUST be set to 0.
> 
> 261        0                   1                   2                   3
> 262        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> 263       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 264       |                 node_id                       |     unused    |
> 265       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 266       |       Interface Index         |           unused              |
> 267       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 
> 269                     Figure 2: Multicast Branch ID format
> 
> 271    Figure 3 shows that the branch ID is carried as an optional field
> 272    after the flow ID and sequence number optional fields in the IOAM DEX
> 273    option header.  Two bits "N" and "I" (i.e., the third and fourth bits
> 274    in the Extension-Flags field) are reserved to indicate the presence
> 275    of the optional branch ID field.  "N" stands for the Node ID and "I"
> 276    stands for the interface index.  If "N" and "I" are both set to 1,
> 277    the optional multicast branch ID field is present; otherwise it is
> 278    absent.
> 
> 280         0                   1                   2                   3
> 281         0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
> 282        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 283        |        Namespace-ID           |     Flags     |F|S|N|I|E-Flags|
> 284        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 285        |               IOAM-Trace-Type                 |   Reserved    |
> 286        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 287        |                         Flow ID (optional)                    |
> 288        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 289        |                     Sequence Number  (Optional)               |
> 290        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 291        |                       Multicast Branch ID                     |
> 292        |                            (optional)                         |
> 293        +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
> 
> 295           Figure 3: Carry Branch ID in IOAM DEX option header
> 
> 297    Once a node gets the branch ID information from the upstream, it MUST
> 298    carry this information in its telemetry data export postcards, so the
> 299    original multicast tree can be correctly reconstructed based on the
> 300    postcards.
> 
> Couple of considerations coming to mind for this mechanism:
> 
> 1. It looks as if the trace information is done on egress, but this is not said.
> Is it obvious to anybody who has read throughall IOAM specs that ONLY egress trace
> data is generated ? Aka: in IPFIX we did define both ingress and egress data capture.
> If in doubt, then maybe add some sentence in this section explaining that this tracing
> is done on egress, aka: after replication - and hence the branch is assumed to be known.
> 
> 2. I do think it would be extremely useful to also
> know the incoming interface, not only the brach ID. And you have the 16 bit
> "Unused" where the incoming interface would nicely fit into without increasing the
> exported payload. For example when using MP2MP in MLDP or Bidir-PIM, the incoming
> interface is not a given, aka: can be different.
> 
> 3. Of couse, if/when you have the incoming interface, then you could also state
> that you do want to optionally do trace on ingress, namely when the packet is received
> by discarded by RPF-check or Bidir-PIM Acceptance check. Aka: Packets that are being
> discarded will have no output branch, but are a great feature to trace. And of course
> there could be the simple logic of setting the branch-id to 0 to indicate the packet
> was discarded, and to imply this was ingress trace output.
> 
> 4. RFC9197 already uses terms ingress_if_id and egress_if_id. If its not too much
> trouble i would suggest to avoid inventing new terms such as "branch_id" but instead
> reuse those pre-existing terms.
> 
> 5. There is another reason, why i do not like branch-id as a term: Together with
> the examples that use branch-id 0, 1 do make me worry that implementers could mis-interpret
> this to mean that the branch-id is allocated starting from 0 independently for
> every multicast tree and renumbered. Aka: In one tree, the OIF to router C is the
> first branch, so it gets branch_id 0, and on another tree, it is the second branch,
> so it gets branch_id 1. Aka: There should be text reconfirming that the same outoing
> interface needs to use the same identifier across all trees, and the examples
> ideally should use non-consecutive identifiers, not starting at 0.
> 
> 302 4.2.  Per-section postcard for IOAM Trace
> 
> 304    The second solution is a combination of the IOAM trace option and the
> 305    postcard-based telemetry.  To avoid data redundancy, at each branch
> 306    fork node, the trace data accumulated up to this node is exported by
> 307    a postcard before the packet is replicated.  In this solution, each
> 308    branch also needs to maintain some identifier to help correlate the
> 309    postcards for each tree section.  The natural way to accomplish this
> 310    is to simply carry the branch fork node's data (including its ID) in
> 311    the trace of each branch.  This is also necessary because each
> 312    replicated multicast packet can have different telemetry data
> 313    pertaining to this particular copy (e.g., node delay, egress
> 314    timestamp, and egress interface).  As a consequence, the local data
> 315    exported by each branch fork node can only contain partial data
> 316    (e.g., ingress interface and ingress timestamp).
> 
> 318    Figure 4 shows an example in a segment of a multicast tree.  Node B
> 319    and D are two branch fork nodes and they will export a postcard
> 320    covering the trace data for the previous section.  The end node of
> 321    each path will also need to export the data of the last section as a
> 322    postcard.
> 
> 324                 P:{A,B'}            P:{B1,C,D'}
> 325                    ^                     ^
> 326                    :                     :
> 327                    :                     :
> 328                    :                     :    {D1}
> 329                    :                     :    +--...
> 330                    :        +---+      +---+  |
> 331                    :   {B1} |   |{B1,C}|   |--+ {D2}
> 332                    :    +-->| C |----->| D |-----...
> 333        +---+     +---+  |   |   |      |   |--+
> 334        |   | {A} |   |--+   +---+      +---+  |
> 335        | A |---->| B |                        +--...
> 336        |   |     |   |--+   +---+             {D3}
> 337        +---+     +---+  |   |   |{B2,E}
> 338                         +-->| E |--...
> 339                        {B2} |   |
> 340                             +---+
> 
> 342                        Figure 4: Per-section Postcard
> 
> 344    There is no need to modify the IOAM trace option header format as
> 345    specified in [RFC9197].  We just need to configure the branch fork
> 346    nodes to export the postcards and refresh the IOAM header and data
> 347    (e.g., clear the node data list and reset the Remaining Length
> 348    field).
> 
> Not quite clear: If you do not apply thre section 4.1 proposal here, how
> would you know from the P:{B1,C,D'} postcard that it should be stitched below
> the P:{A,B'} postcard ? Wouldn't you need the same type of ingress/egress
> interface information to do the stitching ?
> 
> 350 5.  Application Considerations for Multicast Protocols
> 
> 352 5.1.  Mtrace verson 2
> 
> 354    Mtrace version 2 (Mtrace2) [RFC8487] is a protocol that allows the
> 355    tracing of an IP multicast routing path.  Mtrace2 provides additional
> 356    information such as the packet rates and losses, as well as other
> 357    diagnostic information.  Unlike unicast traceroute, Mtrace2 traces
> 358    the path that a packet would take from a source to a receiver.  It is
> 
> Not really, right ? Mtrace2 traces the path that the tree building messages
> follow from receiver to source. The reverse of this traced path can then be
> assumed to be the path taken by multicast packets on the branch of the tree
> to the receiver fdrom which the Mtrace2 was started.
> 
> 359    usually initiated from an Mtrace2 client by sending an Mtrace2 Query
> 360    to a Last-Hop Router (LHR) or to a Rendezvous Point (RP).  The LHR/RP
> 361    turns the Query packet into an Mtrace2 Request, appends a Standard
> 362    Response Block containing its interface addresses and packet
> 363    statistics to the Request packet, and forwards the packet towards the
> 364    source/RP.  In a similar fashion, each router along the path to the
> 365    source/RP appends a Standard Response Block to the end of the Request
> 366    packet and forwards it to its upstream router.  When the First-Hop
> 367    Router (FHR) receives the Request packet, it appends its own Standard
> 368    Response Block, turns the Request packet into a Reply, and unicasts
> 369    the Reply back to the Mtrace2 client.
> 
> 371    New on-path telemetry techniques will enhance Mtrace2, and other
> 372    existing OAM solutions, with more granular and realtime network
> 373    status data through direct measurements.  There are various multicast
> 374    protocols that are used to forward the multicast data.  Each will
> 375    require their own unique on-path telemetry solution.
> 
> At this point in time, readers like me will wonder:
> 
> Is there actually any integration between Mtrace2 and IOAM, and how does this
> document itself help.
> 
> Maybe replace last paragraph by something like: At this point, Mtrace2 does not
> integrate with IOAM2 directly, but network management systems may use Mtrace2
> to learn about routers of interest, on which to instantiate for example the
> IOM mechanisms introdcued in 4.1 and 4.2 ?
> 
> (just guessing. Some practical reason why / how Mtrace can be relevant in a solution
> applying this documents work would be lovely. Else only explicit notion that
> additional work is needed for integration).
> 
> 
> 377 5.2.  Application in PIM
> 
> 379    PIM-SM [RFC7761] is the most widely used multicast routing protocol
> 380    deployed today.  Of the various PIM modes (PIM-SM, PIM-DM, BIDIR-PIM,
> 381    PIM-SSM), PIM-SSM is the preferred method due to its simplicity and
> 382    removal of network source discovery complexity.  With all PIM modes,
> 383    control plane state is established in the network in order to forward
> 384    multicast UDP data packets.  All PIM modes utilize network based
> 385    source discovery except for PIM-SSM, which utilizes application based
> 386    source discovery.  IP Multicast packets fall within the range of
> 
> I don't think we qualify Bidir-PIM as "source-discovery".
> 
> 387    224.0.0.0 through 239.255.255.255.  The telemetry solution will need
> 388    to work within this address range and provide telemetry data for this
> 389    UDP traffic.
> 
> 391    A proposed solution for encapsulating the telemetry instruction
> 392    header and metadata in IPv6 packets is described in
> 393    [I-D.ietf-ippm-ioam-ipv6-options].
> 
> So... is there any IPv4 proposal, or do we only have IPv6 options ? 387-389
> does not mention an IPv4 encapsulation. Would be useful to be explicit. And
> maybe start with IPv6, and then only mention IPv4. 
> 
> 395 5.3.  Application of MVPN X-PMSI Tunnel Encapsulation Attribute
> 
> 397    Multipoint Label Distribution Protocol (mLDP), P2MP RSVP-TE, Ingress
> 398    Replication (IR), PIM MDT SAFI with GRE Transport, are commonly used
> 399    within a Multicast VPN (MVPN) environment utilizing MVPN procedures
> 400    Multicast in MPLS/BGP IP VPNs [RFC6513] and BGP Encoding and
> 401    Procedures for Multicast in MPLS/BGP IP VPNs [RFC6514].  MLDP LDP
> 402    Extension for P2MP and MP2MP LSPs [RFC6388] provides extensions to
> 403    LDP to establish point-to-multipoint (P2MP) and multipoint-to-
> 404    multipoint (MP2MP) label switched paths (LSPs) in MPLS networks.
> 405    P2MP RSVP-TE provides extensions to RSVP-TE RSVP-TE for P2MP LSPs
> 406    [RFC4875] for establish traffic-engineered P2MP LSPs in MPLS
> 407    networks.  Ingress Replication (IR) P2MP Trees Ingress Replication
> 408    Tunnels in Multicast VPNs [RFC7988] using unicast replication from
> 409    parent node to child node over MPLS Unicast Tunnel.  PIM MDT SAFI
> 410    Multicast in BGP/MPLS IP VPNs [RFC6037]utilizes PIM modes PIM-SSM,
> 411    PIM-SM, PIM-BIDIR control plane with GRE transport data plane in the
> 412    core for X-PMSI P-Tree using MVPN procedures.  Replication SID SR
> 413    Replication Segment for Multi-point Service Delivery
> 414    [I-D.ietf-spring-sr-replication-segment] replication segments for
> 415    P2MP multicast service delivery in Segment Routing SR-MPLS networks.
> 416    The telemetry solution will need to be able to follow these P2MP and
> 417    MP2MP paths.  The telemetry instruction header and data should be
> 418    encapsulated into MPLS packets on P2MP and MP2MP paths.  A
> 419    corresponding proposal is described in
> 420    [I-D.song-mpls-extension-header].
> 
> This is unreadable.
> 
> Correct me if i am wrong, but IAM even with the extensions
> proposed here is primarily a forwarding plane thing. So i would suggest
> to start the section by stating that IOAM and the recommendations of this
> document (4.1, 4.2) are equally applicable to multicast MPLS forwarded packets,
> and include Hayou's draft as a possible forwarding plane mechanism.
> 
> Then a paragraph listing the relevant forwarding plane RFCs.
> 
> Then another paragraph stating that network management will need to correlate
> and potentially use/expand equally the multicast MPLS control plane - and list
> the according RFCs for control-plane.
> 
> 422 5.4.  Application in BIER
> 
> 424    BIER [RFC8279] adds a new header to multicast packets and allows the
> 425    multicast packets to be forwarded according to the header only.  By
> 426    eliminating the requirement of maintaining per multicast group state,
> 427    BIER is more scalable than the traditional multicast solutions.
> 
> 429    OAM Requirements for BIER [I-D.ietf-bier-oam-requirements] lists many
> 430    of the requirements for OAM at the BIER layer which will help in the
> 431    forming of on-path telemetry requirements as well.
> 
> 433    Depending on how the BIER header is encapsulated into packets with
> 434    different transport protocols, the method to encapsulate the
> 435    telemetry instruction header and metadata also varies.  It is also
> 436    possible to make the instruction header and metadata a part of the
> 437    BIER header itself, such as in a TLV.
> 
> If you can fit my prior note about BIER-TE, rfc9262 here, that would be lovely.
> 
> Cheers
>    Toerless
> 
> 439 6.  Security Considerations
> 
> 441    No new security issues are identified other than those discovered by
> 442    the IOAM trace and DEX options.
> 
> 444 7.  IANA Considerations
> 
> 446    The document requests two new extension flag registrations in the
> 447    "IOAM DEX Extension-Flags" registry, as described in Section 4.1.
> 
> 449    Bit 2 "Multicast Branching Node ID [RFC XXXX] [RFC Editor: please
> 450    replace with the RFC number of the current document]".
> 
> 452    Bit 3 "Multicast Branching Interface Index [RFC XXXX] [RFC Editor:
> 453    please replace with the RFC number of the current document]".
> 
> 455 8.  Acknowledgments
> 
> 457    The authors would like to thank Frank Brockners, Nils Warnke, Jake
> 458    Holland, and Dino Farinacci for the comments and suggestions.
> 
> 460 9.  References
> 
> 462 9.1.  Normative References
> 
> 464    [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
> 465               Requirement Levels", BCP 14, RFC 2119,
> 466               DOI 10.17487/RFC2119, March 1997,
> 467               <https://www.rfc-editor.org/info/rfc2119>.
> 
> 469    [RFC4687]  Yasukawa, S., Farrel, A., King, D., and T. Nadeau,
> 470               "Operations and Management (OAM) Requirements for Point-
> 471               to-Multipoint MPLS Networks", RFC 4687,
> 472               DOI 10.17487/RFC4687, September 2006,
> 473               <https://www.rfc-editor.org/info/rfc4687>.
> 
> 475    [RFC4875]  Aggarwal, R., Ed., Papadimitriou, D., Ed., and S.
> 476               Yasukawa, Ed., "Extensions to Resource Reservation
> 477               Protocol - Traffic Engineering (RSVP-TE) for Point-to-
> 478               Multipoint TE Label Switched Paths (LSPs)", RFC 4875,
> 479               DOI 10.17487/RFC4875, May 2007,
> 480               <https://www.rfc-editor.org/info/rfc4875>.
> 
> 482    [RFC6037]  Rosen, E., Ed., Cai, Y., Ed., and IJ. Wijnands, "Cisco
> 483               Systems' Solution for Multicast in BGP/MPLS IP VPNs",
> 484               RFC 6037, DOI 10.17487/RFC6037, October 2010,
> 485               <https://www.rfc-editor.org/info/rfc6037>.
> 
> 487    [RFC6388]  Wijnands, IJ., Ed., Minei, I., Ed., Kompella, K., and B.
> 488               Thomas, "Label Distribution Protocol Extensions for Point-
> 489               to-Multipoint and Multipoint-to-Multipoint Label Switched
> 490               Paths", RFC 6388, DOI 10.17487/RFC6388, November 2011,
> 491               <https://www.rfc-editor.org/info/rfc6388>.
> 
> 493    [RFC6513]  Rosen, E., Ed. and R. Aggarwal, Ed., "Multicast in MPLS/
> 494               BGP IP VPNs", RFC 6513, DOI 10.17487/RFC6513, February
> 495               2012, <https://www.rfc-editor.org/info/rfc6513>.
> 
> 497    [RFC6514]  Aggarwal, R., Rosen, E., Morin, T., and Y. Rekhter, "BGP
> 498               Encodings and Procedures for Multicast in MPLS/BGP IP
> 499               VPNs", RFC 6514, DOI 10.17487/RFC6514, February 2012,
> 500               <https://www.rfc-editor.org/info/rfc6514>.
> 
> 502    [RFC7761]  Fenner, B., Handley, M., Holbrook, H., Kouvelas, I.,
> 503               Parekh, R., Zhang, Z., and L. Zheng, "Protocol Independent
> 504               Multicast - Sparse Mode (PIM-SM): Protocol Specification
> 505               (Revised)", STD 83, RFC 7761, DOI 10.17487/RFC7761, March
> 506               2016, <https://www.rfc-editor.org/info/rfc7761>.
> 
> 508    [RFC7988]  Rosen, E., Ed., Subramanian, K., and Z. Zhang, "Ingress
> 509               Replication Tunnels in Multicast VPN", RFC 7988,
> 510               DOI 10.17487/RFC7988, October 2016,
> 511               <https://www.rfc-editor.org/info/rfc7988>.
> 
> 513    [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
> 514               2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
> 515               May 2017, <https://www.rfc-editor.org/info/rfc8174>.
> 
> 517    [RFC8279]  Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A.,
> 518               Przygienda, T., and S. Aldrin, "Multicast Using Bit Index
> 519               Explicit Replication (BIER)", RFC 8279,
> 520               DOI 10.17487/RFC8279, November 2017,
> 521               <https://www.rfc-editor.org/info/rfc8279>.
> 
> 523    [RFC8487]  Asaeda, H., Meyer, K., and W. Lee, Ed., "Mtrace Version 2:
> 524               Traceroute Facility for IP Multicast", RFC 8487,
> 525               DOI 10.17487/RFC8487, October 2018,
> 526               <https://www.rfc-editor.org/info/rfc8487>.
> 
> 528    [RFC9197]  Brockners, F., Ed., Bhandari, S., Ed., and T. Mizrahi,
> 529               Ed., "Data Fields for In Situ Operations, Administration,
> 530               and Maintenance (IOAM)", RFC 9197, DOI 10.17487/RFC9197,
> 531               May 2022, <https://www.rfc-editor.org/info/rfc9197>.
> 
> 533    [RFC9326]  Song, H., Gafni, B., Brockners, F., Bhandari, S., and T.
> 534               Mizrahi, "In Situ Operations, Administration, and
> 535               Maintenance (IOAM) Direct Exporting", RFC 9326,
> 536               DOI 10.17487/RFC9326, November 2022,
> 537               <https://www.rfc-editor.org/info/rfc9326>.
> 
> 539 9.2.  Informative References
> 
> 541    [I-D.ietf-bier-oam-requirements]
> 542               Mirsky, G., Nainar, N. K., Chen, M., and S. Pallagatti,
> 543               "Operations, Administration and Maintenance (OAM)
> 544               Requirements for Bit Index Explicit Replication (BIER)
> 545               Layer", Work in Progress, Internet-Draft, draft-ietf-bier-
> 546               oam-requirements-13, 10 August 2023,
> 547               <https://datatracker.ietf.org/doc/html/draft-ietf-bier-
> 548               oam-requirements-13>.
> 
> 550    [I-D.ietf-ippm-ioam-ipv6-options]
> 551               Bhandari, S. and F. Brockners, "In-situ OAM IPv6 Options",
> 552               Work in Progress, Internet-Draft, draft-ietf-ippm-ioam-
> 553               ipv6-options-12, 7 May 2023,
> 554               <https://datatracker.ietf.org/doc/html/draft-ietf-ippm-
> 555               ioam-ipv6-options-12>.
> 
> 557    [I-D.ietf-spring-sr-replication-segment]
> 558               Voyer, D., Filsfils, C., Parekh, R., Bidgoli, H., and Z.
> 559               J. Zhang, "SR Replication segment for Multi-point Service
> 560               Delivery", Work in Progress, Internet-Draft, draft-ietf-
> 561               spring-sr-replication-segment-19, 28 August 2023,
> 562               <https://datatracker.ietf.org/doc/html/draft-ietf-spring-
> 563               sr-replication-segment-19>.
> 
> 565    [I-D.mirsky-ippm-hybrid-two-step]
> 566               Mirsky, G., Lingqiang, W., Zhui, G., Song, H., and P.
> 567               Thubert, "Hybrid Two-Step Performance Measurement Method",
> 568               Work in Progress, Internet-Draft, draft-mirsky-ippm-
> 569               hybrid-two-step-15, 15 June 2023,
> 570               <https://datatracker.ietf.org/doc/html/draft-mirsky-ippm-
> 571               hybrid-two-step-15>.
> 
> 573    [I-D.song-ippm-postcard-based-telemetry]
> 574               Song, H., Mirsky, G., Zhou, T., Li, Z., Graf, T., Mishra,
> 575               G. S., Shin, J., and K. Lee, "On-Path Telemetry using
> 576               Packet Marking to Trigger Dedicated OAM Packets", Work in
> 577               Progress, Internet-Draft, draft-song-ippm-postcard-based-
> 578               telemetry-16, 2 June 2023,
> 579               <https://datatracker.ietf.org/doc/html/draft-song-ippm-
> 580               postcard-based-telemetry-16>.
> 
> 582    [I-D.song-mpls-extension-header]
> 583               Song, H., Zhou, T., Andersson, L., Zhang, Z. J., and R.
> 584               Gandhi, "MPLS Network Actions using Post-Stack Extension
> 585               Headers", Work in Progress, Internet-Draft, draft-song-
> 586               mpls-extension-header-12, 14 April 2023,
> 587               <https://datatracker.ietf.org/doc/html/draft-song-mpls-
> 588               extension-header-12>.
> 
> 590    [I-D.xie-bier-ipv6-encapsulation]
> 591               Xie, J., Geng, L., McBride, M., Asati, R., Dhanaraj, S.,
> 592               Zhu, Y., Qin, Z., Shin, M., Mishra, G. S., and X. Geng,
> 593               "Encapsulation for BIER in Non-MPLS IPv6 Networks", Work
> 594               in Progress, Internet-Draft, draft-xie-bier-ipv6-
> 595               encapsulation-10, 22 February 2021,
> 596               <https://datatracker.ietf.org/doc/html/draft-xie-bier-
> 597               ipv6-encapsulation-10>.
> 
> 599 Authors' Addresses
> 
> 601    Haoyu Song
> 602    Futurewei Technologies
> 603    2330 Central Expressway
> 604    Santa Clara,
> 605    United States of America
> 606    Email: [e-mail]
> 607    Mike McBride
> 608    Futurewei Technologies
> 609    2330 Central Expressway
> 610    Santa Clara,
> 611    United States of America
> 612    Email: [e-mail]
> 
> 614    Greg Mirsky
> 615    ZTE Corp.
> 616    Email: [e-mail]
> 
> 618    Gyan Mishra
> 619    Verizon Inc.
> 620    Email: [e-mail]
> 
> 622    Hitoshi Asaeda
> 623    National Institute of Information and Communications Technology
> 624    4-2-1 Nukui-Kitamachi, Tokyo
> 625    184-8795
> 626    Japan
> 627    Email: [e-mail]
> 
> 629    Tianran Zhou
> 630    Huawei Technologies
> 631    Beijing
> 632    China
> 633    Email: [e-mail]
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> MBONED mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/mboned