### Re: [ippm] [MBONED] WGLC for draft-ietf-mboned-multicast-telemetry

Toerless Eckert <tte@cs.fau.de> Thu, 21 September 2023 15:53 UTC

Return-Path: <eckert@i4.informatik.uni-erlangen.de>

X-Original-To: ippm@ietfa.amsl.com

Delivered-To: ippm@ietfa.amsl.com

Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C2355C151080; Thu, 21 Sep 2023 08:53:15 -0700 (PDT)

X-Virus-Scanned: amavisd-new at amsl.com

X-Spam-Flag: NO

X-Spam-Score: -1.658

X-Spam-Level:

X-Spam-Status: No, score=-1.658 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.249, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=no autolearn_force=no

Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id idsitPyMbhXJ; Thu, 21 Sep 2023 08:53:11 -0700 (PDT)

Received: from faui40.informatik.uni-erlangen.de (faui40.informatik.uni-erlangen.de [IPv6:2001:638:a000:4134::ffff:40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9F469C151994; Thu, 21 Sep 2023 08:53:10 -0700 (PDT)

Received: from faui48e.informatik.uni-erlangen.de (faui48e.informatik.uni-erlangen.de [IPv6:2001:638:a000:4134::ffff:51]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by faui40.informatik.uni-erlangen.de (Postfix) with ESMTPS id 4Rs0Kt4G0gznkcF; Thu, 21 Sep 2023 17:53:06 +0200 (CEST)

Received: by faui48e.informatik.uni-erlangen.de (Postfix, from userid 10463) id 4Rs0Kt3SvgzkZQ4; Thu, 21 Sep 2023 17:53:06 +0200 (CEST)

Date: Thu, 21 Sep 2023 17:53:06 +0200

From: Toerless Eckert <tte@cs.fau.de>

To: Hitoshi Asaeda <asaeda@ieee.org>

Cc: Leonard Giuliano <lenny@juniper.net>, draft-ietf-mboned-multicast-telemetry@ietf.org, MBONED WG <mboned@ietf.org>, ippm@ietf.org

Message-ID: <ZQxm4q1E8QgXjR4w@faui48e.informatik.uni-erlangen.de>

References: <c67ed24d-8a01-98f0-7ad3-bbf90ecff9dc@juniper.net> <ZQugx8cIj/j18pvj@faui48e.informatik.uni-erlangen.de> <440C0F10-E6CA-4AF5-B81C-A7C798E30AC6@ieee.org>

MIME-Version: 1.0

Content-Type: text/plain; charset="utf-8"

Content-Disposition: inline

Content-Transfer-Encoding: 8bit

In-Reply-To: <440C0F10-E6CA-4AF5-B81C-A7C798E30AC6@ieee.org>

Archived-At: <https://mailarchive.ietf.org/arch/msg/ippm/OK9TI76cRuqiutpUzfWb0vfNGuQ>

Subject: Re: [ippm] [MBONED] WGLC for draft-ietf-mboned-multicast-telemetry

X-BeenThere: ippm@ietf.org

X-Mailman-Version: 2.1.39

Precedence: list

List-Id: IETF IP Performance Metrics Working Group <ippm.ietf.org>

List-Unsubscribe: <https://www.ietf.org/mailman/options/ippm>, <mailto:ippm-request@ietf.org?subject=unsubscribe>

List-Archive: <https://mailarchive.ietf.org/arch/browse/ippm/>

List-Post: <mailto:ippm@ietf.org>

List-Help: <mailto:ippm-request@ietf.org?subject=help>

List-Subscribe: <https://www.ietf.org/mailman/listinfo/ippm>, <mailto:ippm-request@ietf.org?subject=subscribe>

X-List-Received-Date: Thu, 21 Sep 2023 15:53:15 -0000

Thanks, Hitoshi I guess i was irritated by skipping the step that explains how mtrace traces a control-plane path, which to me is a core difference of interest. Cheers Toerless On Thu, Sep 21, 2023 at 01:16:18PM +0900, Hitoshi Asaeda wrote: > Hi Toerless, > > I reply for your Mtrace2’s comment. > > > 352 5.1. Mtrace verson 2 > > > > 354 Mtrace version 2 (Mtrace2) [RFC8487] is a protocol that allows the > > 355 tracing of an IP multicast routing path. Mtrace2 provides additional > > 356 information such as the packet rates and losses, as well as other > > 357 diagnostic information. Unlike unicast traceroute, Mtrace2 traces > > 358 the path that a packet would take from a source to a receiver. It is > > > > Not really, right ? Mtrace2 traces the path that the tree building messages > > follow from receiver to source. The reverse of this traced path can then be > > assumed to be the path taken by multicast packets on the branch of the tree > > to the receiver fdrom which the Mtrace2 was started. > > You are right. I don’t see any contradiction of your statement and the statement written in the draft. > > Mtrace2 request packets are forwarded from a receiver to a source, while it enables the receiver to trace the path for a packet forwarded from the source to the receiver after receiving the Mtrace2 reply as the forwarding path is constructed with the RPF method. > > Regards, > > Hitoshi > > > > On Sep 21, 2023, at 10:47, Toerless Eckert <tte@cs.fau.de> wrote: > > > > On Thu, Sep 14, 2023 at 08:01:31AM -0700, Leonard Giuliano wrote: > >> MBONED (and IPPM), > >> > >> We would like to officially begin working group last call for Multicast > >> On-path Telemetry using IOAM. We had a previous WGLC for this doc earlier > >> this year, and at the time there was not enough response from the WG to > >> support advancement, but the draft has since been updated and in the meeting > >> in IETF117 there was support in the room for another WGLC. It was also > >> suggested that IPPM be included for their input and review. > >> > >> Please post whether you support/oppose the advancement of this draft as well > >> as any comments you may have to the list by Sept 29. > >> > >> Most recent version of the draft can be found here: > >> https://datatracker.ietf.org/doc/draft-ietf-mboned-multicast-telemetry/ > >> > >> -MBONED Chairs > > > > Thanks a lot folks for the work. Some comments, mostly textual inline below in > > idnits output. I did not check the idnits header created. One terminology / functional > > suggestion about the use of pre-existing rfc9197 terminology (ingress/egress interfaces) > > as well as tracing the ingress interface as well. > > > > > > idnits 2.17.1 > > > > draft-ietf-mboned-multicast-telemetry-07.txt: > > > > Checking boilerplate required by RFC 5378 and the IETF Trust (see > > https://trustee.ietf.org/license-info) > > ---------------------------------------------------------------------------- > > > > No issues found here. > > > > Checking nits according to https://www.ietf.org/id-info/1id-guidelines.txt: > > ---------------------------------------------------------------------------- > > > > No issues found here. > > > > Checking nits according to https://www.ietf.org/id-info/checklist : > > ---------------------------------------------------------------------------- > > > > == There are 1 instance of lines with multicast IPv4 addresses in the > > document. If these are generic example addresses, they should be changed > > to use the 233.252.0.x range defined in RFC 5771 > > > > > > Miscellaneous warnings: > > ---------------------------------------------------------------------------- > > > > -- The document date (6 September 2023) is 15 days in the past. Is this > > intentional? > > > > > > Checking references for intended status: Proposed Standard > > ---------------------------------------------------------------------------- > > > > (See RFCs 3967 and 4897 for information about using normative references > > to lower-maturity documents in RFCs) > > > > == Missing Reference: 'A' is mentioned on line 220, but not defined > > > > -- Looks like a reference, but probably isn't: '0' on line 220 > > > > -- Looks like a reference, but probably isn't: '1' on line 220 > > > > == Missing Reference: 'RFC XXXX' is mentioned on line 452, but not defined > > > > == Unused Reference: 'RFC4687' is defined on line 469, but no explicit > > reference was found in the text > > > > == Unused Reference: 'RFC6037' is defined on line 482, but no explicit > > reference was found in the text > > > > == Unused Reference: 'I-D.xie-bier-ipv6-encapsulation' is defined on line > > 590, but no explicit reference was found in the text > > > > ** Downref: Normative reference to an Informational RFC: RFC 4687 > > > > ** Downref: Normative reference to an Historic RFC: RFC 6037 > > > > > > Summary: 2 errors (**), 0 flaws (~~), 6 warnings (==), 3 comments (--). > > > > Run idnits with the --verbose option for more detailed information about > > the items above. > > > > -------------------------------------------------------------------------------- > > > > > > 2 MBONED H. Song > > 3 Internet-Draft M. McBride > > 4 Intended status: Standards Track Futurewei Technologies > > 5 Expires: 9 March 2024 G. Mirsky > > 6 ZTE Corp. > > 7 G. Mishra > > 8 Verizon Inc. > > 9 H. Asaeda > > 10 NICT > > 11 T. Zhou > > 12 Huawei Technologies > > 13 6 September 2023 > > > > 15 Multicast On-path Telemetry using IOAM > > 16 draft-ietf-mboned-multicast-telemetry-07 > > > > 18 Abstract > > > > 20 This document specifies the requirements of on-path telemetry for > > 21 multicast traffic using In-situ OAM. While In-situ OAM is > > 22 advantageous for multicast traffic telemetry, some unique challenges > > 23 present. This document provides the solutions based on the In-situ > > > > "some unique challenges present" - does not parse. > > > > 24 OAM trace option and direct export option to support the telemetry > > 25 data correlation and the multicast tree reconstruction without > > 26 incurring data redundancy. > > > > 28 Requirements Language > > > > 30 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", > > 31 "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and > > 32 "OPTIONAL" in this document are to be interpreted as described in BCP > > 33 14 [RFC2119][RFC8174] when, and only when, they appear in all > > 34 capitals, as shown here. > > > > 36 Status of This Memo > > > > 38 This Internet-Draft is submitted in full conformance with the > > 39 provisions of BCP 78 and BCP 79. > > > > 41 Internet-Drafts are working documents of the Internet Engineering > > 42 Task Force (IETF). Note that other groups may also distribute > > 43 working documents as Internet-Drafts. The list of current Internet- > > 44 Drafts is at https://datatracker.ietf.org/drafts/current/. > > > > 46 Internet-Drafts are draft documents valid for a maximum of six months > > 47 and may be updated, replaced, or obsoleted by other documents at any > > 48 time. It is inappropriate to use Internet-Drafts as reference > > 49 material or to cite them other than as "work in progress." > > > > 51 This Internet-Draft will expire on 9 March 2024. > > > > 53 Copyright Notice > > > > 55 Copyright (c) 2023 IETF Trust and the persons identified as the > > 56 document authors. All rights reserved. > > > > 58 This document is subject to BCP 78 and the IETF Trust's Legal > > 59 Provisions Relating to IETF Documents (https://trustee.ietf.org/ > > 60 license-info) in effect on the date of publication of this document. > > 61 Please review these documents carefully, as they describe your rights > > 62 and restrictions with respect to this document. Code Components > > 63 extracted from this document must include Revised BSD License text as > > 64 described in Section 4.e of the Trust Legal Provisions and are > > 65 provided without warranty as described in the Revised BSD License. > > > > 67 Table of Contents > > > > 69 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 > > 70 2. Requirements for Multicast Traffic Telemetry . . . . . . . . 4 > > 71 3. Issues of Existing Techniques . . . . . . . . . . . . . . . . 4 > > 72 4. Modifications to Existing Solutions . . . . . . . . . . . . . 5 > > 73 4.1. Per-hop postcard using IOAM DEX . . . . . . . . . . . . . 5 > > 74 4.2. Per-section postcard for IOAM Trace . . . . . . . . . . . 7 > > 75 5. Application Considerations for Multicast Protocols . . . . . 8 > > 76 5.1. Mtrace verson 2 . . . . . . . . . . . . . . . . . . . . . 8 > > 77 5.2. Application in PIM . . . . . . . . . . . . . . . . . . . 9 > > 78 5.3. Application of MVPN X-PMSI Tunnel Encapsulation > > 79 Attribute . . . . . . . . . . . . . . . . . . . . . . . . 9 > > 80 5.4. Application in BIER . . . . . . . . . . . . . . . . . . . 10 > > 81 6. Security Considerations . . . . . . . . . . . . . . . . . . . 10 > > 82 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 10 > > 83 8. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 10 > > 84 9. References . . . . . . . . . . . . . . . . . . . . . . . . . 10 > > 85 9.1. Normative References . . . . . . . . . . . . . . . . . . 10 > > 86 9.2. Informative References . . . . . . . . . . . . . . . . . 12 > > 87 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 13 > > > > 89 1. Introduction > > > > 91 Multicast is used by residential broadband customers across operator > > > > suggest /used by/used for example by, but not limited to/ > > aka: there's a lot more. > > > > 92 networks, private MPLS customers, and internal customers within > > 93 corporate intranet. Multicast provides real time interactive online > > 94 meetings or podcasts, IPTV, and financial markets real-time data, > > 95 which all have a reliance on UDP's unreliable transport. End-to-end > > 96 QOS, therefore, should be a critical component of multicast > > 97 deployment in order to provide a good end user experience. In > > 98 multicast video streaming, if a packet is dropped, and that packet > > 99 happens to be a reference frame (i.e., I-Frame) in the video feed, > > 100 the client receiver of the multicast feed goes into buffering mode > > 101 resulting in a frozen window. > > > > Suggest rewording of last sentence: > > > > In multicast real-time media streaming, loss of a single packet containing a > > reference frame can result in the inability of all thousands of receivers to > > decode a whole so-called Group-of-Picture sequence of packets, introducing > > black picture for periods often as much as 4 seconds. Unexpected long delay > > in propagation of a packet in such real-time media streaming may equally result > > oin the packet not being received and cxreating the same results. > > > > 101 Multicast packet drops and delay can > > > > ...therefore > > > > 102 severely affect the application performance and user experience. > > > > 104 It is important to monitor the performance of the multicast traffic. > > 105 New on-path telemetry techniques such as In-situ OAM (IOAM) > > 106 [RFC9197], IOAM Direct Export (DEX) [RFC9326] IOAM Marking-based > > 107 Postcard (PBT-M) [I-D.song-ippm-postcard-based-telemetry], and Hybrid > > 108 Two-Step (HTS) [I-D.mirsky-ippm-hybrid-two-step] are useful and > > 109 complementary to the existing active OAM performance monitoring > > 110 methods, provide promising means to directly monitor the network > > > > "existing active OAM performance monitoring" - such as ? One or two references > > would help. > > > > 111 experience of multicast traffic. However, multicast traffic has some > > 112 unique characteristics which pose some challenges on applying such > > 113 techniques in an efficient way. > > > > 115 The IP multicast packet data for a particular (S, G) state is > > 116 identical from one branch to another on its way to multiple > > 117 receivers. When adding IOAM trace data to multicast packets, each > > 118 replicated packet would keep the telemetry data for its entire > > 119 forwarding path. Since the replicated packets all share common path > > 120 segments, redundant data will be collected for the same original > > 121 multicast packet. Such redundancy consumes extra network bandwidth > > 122 unnecessarily. For a large multicast tree, such redundancy is > > 123 considerable. Alternatively, it could be more efficient to collect > > 124 the telemetry data using solutions such as IOAM DEX to eliminate the > > 125 data redundancy. However, IOAM DEX is lack of a branch identifier, > > > > s/is lack of/lacks/ > > > > 126 making telemetry data correlation and multicast-tree reconstruction > > 127 difficult. > > > > 129 This draft provides two solutions to the IOAM data redundancy problem > > 130 based on the IOAM standards. The requirements for multicast traffic > > 131 telemetry are discussed along with the issues of the existing on-path > > 132 telemetry techniques. We propose modifications to make these > > 133 techniques adapt to multicast in order for the original multicast > > 134 tree to be correctly reconstructed while eliminating redundant data. > > > > 136 2. Requirements for Multicast Traffic Telemetry > > > > 138 Multicast traffic is forwarded through a multicast tree. With PIM > > 139 and P2MP (e.g., MLDP, RSVP-TE) the forwarding tree is established and > > > > Please provide expansion of terms and references for them on first use, aka: for > > MLDP, RSVP-TE here (not later). Please check for all abbreviations in the document, > > i may overlook them as i am not doing a thorough search. > > > > 140 maintained by the multicast routing protocol. With BIER, no state is > > 141 created in the network to establish a forwarding tree; instead, a > > 142 bier header provides the necessary information for each packet to > > 143 know the egress points. Multicast packets are only replicated at > > > > And with BIER-TE the replication points too ;-)) > > > > 144 each tree branch fork node for efficiency. > > > > 146 There are several requirements for multicast traffic telemetry, a few > > 147 of which are: > > > > 149 * Reconstruct and visualize the multicast tree through data plane > > 150 monitoring. > > > > 152 * Gather the multicast packet delay and jitter performance on each > > 153 path. > > > > 155 * Find the multicast packet drop location and reason. > > > > 157 * Gather the VPN state and tunnel information in case of P2MP > > 158 multicast. > > > > 160 In order to meet these requirements, we need the ability to directly > > 161 monitor the multicast traffic and derive data from the multicast > > 162 packets. The conventional OAM mechanisms, such as multicast ping and > > 163 trace, are not sufficient to meet these requirements. > > > > references for ping and trace here please. > > > > 165 3. Issues of Existing Techniques > > > > 167 On-path Telemetry techniques that directly retrieve data from > > 168 multicast traffic's live network experience are ideal for addressing > > 169 the aforementioned requirements. The representative techniques > > 170 include In-situ OAM (IOAM) Trace option [RFC9197], IOAM Direct Export > > 171 (DEX) option [RFC9326], and PBT-M > > 172 [I-D.song-ippm-postcard-based-telemetry]. However, unlike unicast, > > 173 multicast poses some unique challenges to applying these techniques. > > > > 175 Multicast packets are replicated at each branch fork node in the > > 176 corresponding multicast tree. Therefore, there are multiple copies > > 177 of the original multicast packet in the network. > > > > 179 If the IOAM trace option is used for on-path data collection, the > > 180 partial trace data will also be replicated into the copy for each > > 181 branch. The end result is that, at the multicast tree leaves, each > > 182 copy of the multicast packet has a complete trace. Most of the data > > 183 (except data from the last leaf branch), however, has redundant > > 184 copies. Data redundancy introduces unnecessary header overhead, > > 185 wastes network bandwidth, and complicates the data processing. The > > 186 larger the multicast tree is or the longer the multicast path is, the > > 187 more severe the redundancy problem becomes. > > > > It is not clear to me if i am incorrectly understanding how IAM trace is supposed to work. > > I thought that it would allow to collect trace information from more than one hop along the > > path. So if i assume there is replication on every hop with multicast, and collection of > > trace data on every hop, then the packet on every receiver would still have different > > trace data, and only initial parts of the trace would be the same. But if lets > > say i am multicast to 100 receivers, i am still saving a huge amount of data on > > the wire by multicasting as compared to the source sending 100 unicast packets. > > > > Whether this is true or false, it seems such a simple example to summarize how > > IOAM trace works would help the text. > > > > 189 The postcard-based solutions (e.g., IOAM DEX), can be used to > > 190 eliminate such data redundancy, because each node on the tree only > > 191 sends a postcard covering local data. However, they cannot track and > > 192 correlate the tree branches properly due to the lack of branching > > 193 information, so they can bring confusion about the multicast tree > > 194 topology. For example, in a multicast tree, Node A has two branches, > > 195 one to Node B and the other to node C; further, Node B leads to Node > > 196 D and Node C leads to Node E. When applying postcard-based methods, > > 197 one cannot tell whether or not Node D(E) is the next hop of Node B(C) > > 198 from the received postcards. > > > > Unless one correlates the exporting nodes with knowledge about the tree collected > > by other means, such as mtrace or SNMP/YANG. Aka: May be useful to include an > > argument why such correlation is undesirable or inferior to what's proposed here. > > > > May i also note that BIER-TE, RFC9262 does contain the tree information in the > > packet header, so it would be possible to directly deduce the tree a packet was > > passed over when correlating received IOAM DEX information. > > > > 200 The fundamental reason for this problem is that there is not an > > 201 identifier (either implicit or explicit) to correlate the data on > > 202 each branch. > > > > 204 4. Modifications to Existing Solutions > > > > 206 We provide two solutions to address the above issues. One is based > > 207 on IOAM DEX and requires an extension to the instruction header of > > 208 the IOAM DEX Option. The second solution combines the IOAM trace > > 209 option and postcards for redundancy removal. > > > > 211 4.1. Per-hop postcard using IOAM DEX > > > > 213 One way to mitigate the postcard-based telemetry's tree tracking > > 214 weakness is to augment it with a branch identifier field. Note that > > 215 this works for the IOAM DEX option but not for PBT-M because the IOAM > > 216 DEX option has an instruction header which can be used to hold the > > 217 branch identifier. To make the branch identifier globally unique, > > 218 the branch fork node ID plus an index is used. For example, Node A > > 219 has two branches: one to Node B and the other to Node C. Node A will > > 220 use [A, 0] as the branch identifier for the branch to B, and [A, 1] > > 221 for the branch to C. The identifier is carried with the multicast > > 222 packet until the next branch fork node. Each node MUST export the > > 223 branch identifier in the received IOAM DEX header in the postcards it > > 224 sends. The branch identifier, along with the other fields such as > > 225 flow ID and sequence number, is sufficient for the data collector to > > 226 reconstruct the topology of the multicast tree. > > > > 228 Figure 1 shows an example of this solution. "P" stands for the > > 229 postcard packet. The square brackets contains the branch identifier. > > 230 The curly brace contains the telemetry data about a specific node. > > > > 232 P:[A,0]{A} P:[A,0]{B} P:[B,1]{D} P:[B,0]{C} P:[B,0]{E} > > 233 ^ ^ ^ ^ ^ > > 234 : : : : : > > 235 : : : : : > > 236 : : : +-:-+ +-:-+ > > 237 : : : | | | | > > 238 : : +---:----->| C |------>| E |-... > > 239 +-:-+ +-:-+ | : | | | | > > 240 | | | |----+ : +---+ +---+ > > 241 | A |------->| B | : > > 242 | | | |--+ +-:-+ > > 243 +---+ +---+ | | | > > 244 +-->| D |--... > > 245 | | > > 246 +---+ > > > > 248 Figure 1: Per-hop Postcard > > > > 250 Each branch fork node needs to generate a unique branch identifier > > 251 (i.e., branch ID) for each branch in its multicast tree instance and > > 252 include it in the IOAM DEX option header. The branch ID remains > > 253 unchanged until the next branch fork node. The branch ID contains > > 254 two parts: the branch fork node ID and an interface index. > > > > 256 Conforming to the node ID specification in IOAM [RFC9197], the node > > 257 ID is a 3-octet unsigned integer. The interface index is a two-octet > > 258 unsigned integer. As shown in Figure 2, the branch ID consumes 8 > > 259 octets in total. The three unused octets MUST be set to 0. > > > > 261 0 1 2 3 > > 262 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > > 263 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > 264 | node_id | unused | > > 265 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > 266 | Interface Index | unused | > > 267 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > > > 269 Figure 2: Multicast Branch ID format > > > > 271 Figure 3 shows that the branch ID is carried as an optional field > > 272 after the flow ID and sequence number optional fields in the IOAM DEX > > 273 option header. Two bits "N" and "I" (i.e., the third and fourth bits > > 274 in the Extension-Flags field) are reserved to indicate the presence > > 275 of the optional branch ID field. "N" stands for the Node ID and "I" > > 276 stands for the interface index. If "N" and "I" are both set to 1, > > 277 the optional multicast branch ID field is present; otherwise it is > > 278 absent. > > > > 280 0 1 2 3 > > 281 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 > > 282 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > 283 | Namespace-ID | Flags |F|S|N|I|E-Flags| > > 284 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > 285 | IOAM-Trace-Type | Reserved | > > 286 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > 287 | Flow ID (optional) | > > 288 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > 289 | Sequence Number (Optional) | > > 290 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > 291 | Multicast Branch ID | > > 292 | (optional) | > > 293 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ > > > > 295 Figure 3: Carry Branch ID in IOAM DEX option header > > > > 297 Once a node gets the branch ID information from the upstream, it MUST > > 298 carry this information in its telemetry data export postcards, so the > > 299 original multicast tree can be correctly reconstructed based on the > > 300 postcards. > > > > Couple of considerations coming to mind for this mechanism: > > > > 1. It looks as if the trace information is done on egress, but this is not said. > > Is it obvious to anybody who has read throughall IOAM specs that ONLY egress trace > > data is generated ? Aka: in IPFIX we did define both ingress and egress data capture. > > If in doubt, then maybe add some sentence in this section explaining that this tracing > > is done on egress, aka: after replication - and hence the branch is assumed to be known. > > > > 2. I do think it would be extremely useful to also > > know the incoming interface, not only the brach ID. And you have the 16 bit > > "Unused" where the incoming interface would nicely fit into without increasing the > > exported payload. For example when using MP2MP in MLDP or Bidir-PIM, the incoming > > interface is not a given, aka: can be different. > > > > 3. Of couse, if/when you have the incoming interface, then you could also state > > that you do want to optionally do trace on ingress, namely when the packet is received > > by discarded by RPF-check or Bidir-PIM Acceptance check. Aka: Packets that are being > > discarded will have no output branch, but are a great feature to trace. And of course > > there could be the simple logic of setting the branch-id to 0 to indicate the packet > > was discarded, and to imply this was ingress trace output. > > > > 4. RFC9197 already uses terms ingress_if_id and egress_if_id. If its not too much > > trouble i would suggest to avoid inventing new terms such as "branch_id" but instead > > reuse those pre-existing terms. > > > > 5. There is another reason, why i do not like branch-id as a term: Together with > > the examples that use branch-id 0, 1 do make me worry that implementers could mis-interpret > > this to mean that the branch-id is allocated starting from 0 independently for > > every multicast tree and renumbered. Aka: In one tree, the OIF to router C is the > > first branch, so it gets branch_id 0, and on another tree, it is the second branch, > > so it gets branch_id 1. Aka: There should be text reconfirming that the same outoing > > interface needs to use the same identifier across all trees, and the examples > > ideally should use non-consecutive identifiers, not starting at 0. > > > > 302 4.2. Per-section postcard for IOAM Trace > > > > 304 The second solution is a combination of the IOAM trace option and the > > 305 postcard-based telemetry. To avoid data redundancy, at each branch > > 306 fork node, the trace data accumulated up to this node is exported by > > 307 a postcard before the packet is replicated. In this solution, each > > 308 branch also needs to maintain some identifier to help correlate the > > 309 postcards for each tree section. The natural way to accomplish this > > 310 is to simply carry the branch fork node's data (including its ID) in > > 311 the trace of each branch. This is also necessary because each > > 312 replicated multicast packet can have different telemetry data > > 313 pertaining to this particular copy (e.g., node delay, egress > > 314 timestamp, and egress interface). As a consequence, the local data > > 315 exported by each branch fork node can only contain partial data > > 316 (e.g., ingress interface and ingress timestamp). > > > > 318 Figure 4 shows an example in a segment of a multicast tree. Node B > > 319 and D are two branch fork nodes and they will export a postcard > > 320 covering the trace data for the previous section. The end node of > > 321 each path will also need to export the data of the last section as a > > 322 postcard. > > > > 324 P:{A,B'} P:{B1,C,D'} > > 325 ^ ^ > > 326 : : > > 327 : : > > 328 : : {D1} > > 329 : : +--... > > 330 : +---+ +---+ | > > 331 : {B1} | |{B1,C}| |--+ {D2} > > 332 : +-->| C |----->| D |-----... > > 333 +---+ +---+ | | | | |--+ > > 334 | | {A} | |--+ +---+ +---+ | > > 335 | A |---->| B | +--... > > 336 | | | |--+ +---+ {D3} > > 337 +---+ +---+ | | |{B2,E} > > 338 +-->| E |--... > > 339 {B2} | | > > 340 +---+ > > > > 342 Figure 4: Per-section Postcard > > > > 344 There is no need to modify the IOAM trace option header format as > > 345 specified in [RFC9197]. We just need to configure the branch fork > > 346 nodes to export the postcards and refresh the IOAM header and data > > 347 (e.g., clear the node data list and reset the Remaining Length > > 348 field). > > > > Not quite clear: If you do not apply thre section 4.1 proposal here, how > > would you know from the P:{B1,C,D'} postcard that it should be stitched below > > the P:{A,B'} postcard ? Wouldn't you need the same type of ingress/egress > > interface information to do the stitching ? > > > > 350 5. Application Considerations for Multicast Protocols > > > > 352 5.1. Mtrace verson 2 > > > > 354 Mtrace version 2 (Mtrace2) [RFC8487] is a protocol that allows the > > 355 tracing of an IP multicast routing path. Mtrace2 provides additional > > 356 information such as the packet rates and losses, as well as other > > 357 diagnostic information. Unlike unicast traceroute, Mtrace2 traces > > 358 the path that a packet would take from a source to a receiver. It is > > > > Not really, right ? Mtrace2 traces the path that the tree building messages > > follow from receiver to source. The reverse of this traced path can then be > > assumed to be the path taken by multicast packets on the branch of the tree > > to the receiver fdrom which the Mtrace2 was started. > > > > 359 usually initiated from an Mtrace2 client by sending an Mtrace2 Query > > 360 to a Last-Hop Router (LHR) or to a Rendezvous Point (RP). The LHR/RP > > 361 turns the Query packet into an Mtrace2 Request, appends a Standard > > 362 Response Block containing its interface addresses and packet > > 363 statistics to the Request packet, and forwards the packet towards the > > 364 source/RP. In a similar fashion, each router along the path to the > > 365 source/RP appends a Standard Response Block to the end of the Request > > 366 packet and forwards it to its upstream router. When the First-Hop > > 367 Router (FHR) receives the Request packet, it appends its own Standard > > 368 Response Block, turns the Request packet into a Reply, and unicasts > > 369 the Reply back to the Mtrace2 client. > > > > 371 New on-path telemetry techniques will enhance Mtrace2, and other > > 372 existing OAM solutions, with more granular and realtime network > > 373 status data through direct measurements. There are various multicast > > 374 protocols that are used to forward the multicast data. Each will > > 375 require their own unique on-path telemetry solution. > > > > At this point in time, readers like me will wonder: > > > > Is there actually any integration between Mtrace2 and IOAM, and how does this > > document itself help. > > > > Maybe replace last paragraph by something like: At this point, Mtrace2 does not > > integrate with IOAM2 directly, but network management systems may use Mtrace2 > > to learn about routers of interest, on which to instantiate for example the > > IOM mechanisms introdcued in 4.1 and 4.2 ? > > > > (just guessing. Some practical reason why / how Mtrace can be relevant in a solution > > applying this documents work would be lovely. Else only explicit notion that > > additional work is needed for integration). > > > > > > 377 5.2. Application in PIM > > > > 379 PIM-SM [RFC7761] is the most widely used multicast routing protocol > > 380 deployed today. Of the various PIM modes (PIM-SM, PIM-DM, BIDIR-PIM, > > 381 PIM-SSM), PIM-SSM is the preferred method due to its simplicity and > > 382 removal of network source discovery complexity. With all PIM modes, > > 383 control plane state is established in the network in order to forward > > 384 multicast UDP data packets. All PIM modes utilize network based > > 385 source discovery except for PIM-SSM, which utilizes application based > > 386 source discovery. IP Multicast packets fall within the range of > > > > I don't think we qualify Bidir-PIM as "source-discovery". > > > > 387 224.0.0.0 through 239.255.255.255. The telemetry solution will need > > 388 to work within this address range and provide telemetry data for this > > 389 UDP traffic. > > > > 391 A proposed solution for encapsulating the telemetry instruction > > 392 header and metadata in IPv6 packets is described in > > 393 [I-D.ietf-ippm-ioam-ipv6-options]. > > > > So... is there any IPv4 proposal, or do we only have IPv6 options ? 387-389 > > does not mention an IPv4 encapsulation. Would be useful to be explicit. And > > maybe start with IPv6, and then only mention IPv4. > > > > 395 5.3. Application of MVPN X-PMSI Tunnel Encapsulation Attribute > > > > 397 Multipoint Label Distribution Protocol (mLDP), P2MP RSVP-TE, Ingress > > 398 Replication (IR), PIM MDT SAFI with GRE Transport, are commonly used > > 399 within a Multicast VPN (MVPN) environment utilizing MVPN procedures > > 400 Multicast in MPLS/BGP IP VPNs [RFC6513] and BGP Encoding and > > 401 Procedures for Multicast in MPLS/BGP IP VPNs [RFC6514]. MLDP LDP > > 402 Extension for P2MP and MP2MP LSPs [RFC6388] provides extensions to > > 403 LDP to establish point-to-multipoint (P2MP) and multipoint-to- > > 404 multipoint (MP2MP) label switched paths (LSPs) in MPLS networks. > > 405 P2MP RSVP-TE provides extensions to RSVP-TE RSVP-TE for P2MP LSPs > > 406 [RFC4875] for establish traffic-engineered P2MP LSPs in MPLS > > 407 networks. Ingress Replication (IR) P2MP Trees Ingress Replication > > 408 Tunnels in Multicast VPNs [RFC7988] using unicast replication from > > 409 parent node to child node over MPLS Unicast Tunnel. PIM MDT SAFI > > 410 Multicast in BGP/MPLS IP VPNs [RFC6037]utilizes PIM modes PIM-SSM, > > 411 PIM-SM, PIM-BIDIR control plane with GRE transport data plane in the > > 412 core for X-PMSI P-Tree using MVPN procedures. Replication SID SR > > 413 Replication Segment for Multi-point Service Delivery > > 414 [I-D.ietf-spring-sr-replication-segment] replication segments for > > 415 P2MP multicast service delivery in Segment Routing SR-MPLS networks. > > 416 The telemetry solution will need to be able to follow these P2MP and > > 417 MP2MP paths. The telemetry instruction header and data should be > > 418 encapsulated into MPLS packets on P2MP and MP2MP paths. A > > 419 corresponding proposal is described in > > 420 [I-D.song-mpls-extension-header]. > > > > This is unreadable. > > > > Correct me if i am wrong, but IAM even with the extensions > > proposed here is primarily a forwarding plane thing. So i would suggest > > to start the section by stating that IOAM and the recommendations of this > > document (4.1, 4.2) are equally applicable to multicast MPLS forwarded packets, > > and include Hayou's draft as a possible forwarding plane mechanism. > > > > Then a paragraph listing the relevant forwarding plane RFCs. > > > > Then another paragraph stating that network management will need to correlate > > and potentially use/expand equally the multicast MPLS control plane - and list > > the according RFCs for control-plane. > > > > 422 5.4. Application in BIER > > > > 424 BIER [RFC8279] adds a new header to multicast packets and allows the > > 425 multicast packets to be forwarded according to the header only. By > > 426 eliminating the requirement of maintaining per multicast group state, > > 427 BIER is more scalable than the traditional multicast solutions. > > > > 429 OAM Requirements for BIER [I-D.ietf-bier-oam-requirements] lists many > > 430 of the requirements for OAM at the BIER layer which will help in the > > 431 forming of on-path telemetry requirements as well. > > > > 433 Depending on how the BIER header is encapsulated into packets with > > 434 different transport protocols, the method to encapsulate the > > 435 telemetry instruction header and metadata also varies. It is also > > 436 possible to make the instruction header and metadata a part of the > > 437 BIER header itself, such as in a TLV. > > > > If you can fit my prior note about BIER-TE, rfc9262 here, that would be lovely. > > > > Cheers > > Toerless > > > > 439 6. Security Considerations > > > > 441 No new security issues are identified other than those discovered by > > 442 the IOAM trace and DEX options. > > > > 444 7. IANA Considerations > > > > 446 The document requests two new extension flag registrations in the > > 447 "IOAM DEX Extension-Flags" registry, as described in Section 4.1. > > > > 449 Bit 2 "Multicast Branching Node ID [RFC XXXX] [RFC Editor: please > > 450 replace with the RFC number of the current document]". > > > > 452 Bit 3 "Multicast Branching Interface Index [RFC XXXX] [RFC Editor: > > 453 please replace with the RFC number of the current document]". > > > > 455 8. Acknowledgments > > > > 457 The authors would like to thank Frank Brockners, Nils Warnke, Jake > > 458 Holland, and Dino Farinacci for the comments and suggestions. > > > > 460 9. References > > > > 462 9.1. Normative References > > > > 464 [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate > > 465 Requirement Levels", BCP 14, RFC 2119, > > 466 DOI 10.17487/RFC2119, March 1997, > > 467 <https://www.rfc-editor.org/info/rfc2119>. > > > > 469 [RFC4687] Yasukawa, S., Farrel, A., King, D., and T. Nadeau, > > 470 "Operations and Management (OAM) Requirements for Point- > > 471 to-Multipoint MPLS Networks", RFC 4687, > > 472 DOI 10.17487/RFC4687, September 2006, > > 473 <https://www.rfc-editor.org/info/rfc4687>. > > > > 475 [RFC4875] Aggarwal, R., Ed., Papadimitriou, D., Ed., and S. > > 476 Yasukawa, Ed., "Extensions to Resource Reservation > > 477 Protocol - Traffic Engineering (RSVP-TE) for Point-to- > > 478 Multipoint TE Label Switched Paths (LSPs)", RFC 4875, > > 479 DOI 10.17487/RFC4875, May 2007, > > 480 <https://www.rfc-editor.org/info/rfc4875>. > > > > 482 [RFC6037] Rosen, E., Ed., Cai, Y., Ed., and IJ. Wijnands, "Cisco > > 483 Systems' Solution for Multicast in BGP/MPLS IP VPNs", > > 484 RFC 6037, DOI 10.17487/RFC6037, October 2010, > > 485 <https://www.rfc-editor.org/info/rfc6037>. > > > > 487 [RFC6388] Wijnands, IJ., Ed., Minei, I., Ed., Kompella, K., and B. > > 488 Thomas, "Label Distribution Protocol Extensions for Point- > > 489 to-Multipoint and Multipoint-to-Multipoint Label Switched > > 490 Paths", RFC 6388, DOI 10.17487/RFC6388, November 2011, > > 491 <https://www.rfc-editor.org/info/rfc6388>. > > > > 493 [RFC6513] Rosen, E., Ed. and R. Aggarwal, Ed., "Multicast in MPLS/ > > 494 BGP IP VPNs", RFC 6513, DOI 10.17487/RFC6513, February > > 495 2012, <https://www.rfc-editor.org/info/rfc6513>. > > > > 497 [RFC6514] Aggarwal, R., Rosen, E., Morin, T., and Y. Rekhter, "BGP > > 498 Encodings and Procedures for Multicast in MPLS/BGP IP > > 499 VPNs", RFC 6514, DOI 10.17487/RFC6514, February 2012, > > 500 <https://www.rfc-editor.org/info/rfc6514>. > > > > 502 [RFC7761] Fenner, B., Handley, M., Holbrook, H., Kouvelas, I., > > 503 Parekh, R., Zhang, Z., and L. Zheng, "Protocol Independent > > 504 Multicast - Sparse Mode (PIM-SM): Protocol Specification > > 505 (Revised)", STD 83, RFC 7761, DOI 10.17487/RFC7761, March > > 506 2016, <https://www.rfc-editor.org/info/rfc7761>. > > > > 508 [RFC7988] Rosen, E., Ed., Subramanian, K., and Z. Zhang, "Ingress > > 509 Replication Tunnels in Multicast VPN", RFC 7988, > > 510 DOI 10.17487/RFC7988, October 2016, > > 511 <https://www.rfc-editor.org/info/rfc7988>. > > > > 513 [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC > > 514 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, > > 515 May 2017, <https://www.rfc-editor.org/info/rfc8174>. > > > > 517 [RFC8279] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., > > 518 Przygienda, T., and S. Aldrin, "Multicast Using Bit Index > > 519 Explicit Replication (BIER)", RFC 8279, > > 520 DOI 10.17487/RFC8279, November 2017, > > 521 <https://www.rfc-editor.org/info/rfc8279>. > > > > 523 [RFC8487] Asaeda, H., Meyer, K., and W. Lee, Ed., "Mtrace Version 2: > > 524 Traceroute Facility for IP Multicast", RFC 8487, > > 525 DOI 10.17487/RFC8487, October 2018, > > 526 <https://www.rfc-editor.org/info/rfc8487>. > > > > 528 [RFC9197] Brockners, F., Ed., Bhandari, S., Ed., and T. Mizrahi, > > 529 Ed., "Data Fields for In Situ Operations, Administration, > > 530 and Maintenance (IOAM)", RFC 9197, DOI 10.17487/RFC9197, > > 531 May 2022, <https://www.rfc-editor.org/info/rfc9197>. > > > > 533 [RFC9326] Song, H., Gafni, B., Brockners, F., Bhandari, S., and T. > > 534 Mizrahi, "In Situ Operations, Administration, and > > 535 Maintenance (IOAM) Direct Exporting", RFC 9326, > > 536 DOI 10.17487/RFC9326, November 2022, > > 537 <https://www.rfc-editor.org/info/rfc9326>. > > > > 539 9.2. Informative References > > > > 541 [I-D.ietf-bier-oam-requirements] > > 542 Mirsky, G., Nainar, N. K., Chen, M., and S. Pallagatti, > > 543 "Operations, Administration and Maintenance (OAM) > > 544 Requirements for Bit Index Explicit Replication (BIER) > > 545 Layer", Work in Progress, Internet-Draft, draft-ietf-bier- > > 546 oam-requirements-13, 10 August 2023, > > 547 <https://datatracker.ietf.org/doc/html/draft-ietf-bier- > > 548 oam-requirements-13>. > > > > 550 [I-D.ietf-ippm-ioam-ipv6-options] > > 551 Bhandari, S. and F. Brockners, "In-situ OAM IPv6 Options", > > 552 Work in Progress, Internet-Draft, draft-ietf-ippm-ioam- > > 553 ipv6-options-12, 7 May 2023, > > 554 <https://datatracker.ietf.org/doc/html/draft-ietf-ippm- > > 555 ioam-ipv6-options-12>. > > > > 557 [I-D.ietf-spring-sr-replication-segment] > > 558 Voyer, D., Filsfils, C., Parekh, R., Bidgoli, H., and Z. > > 559 J. Zhang, "SR Replication segment for Multi-point Service > > 560 Delivery", Work in Progress, Internet-Draft, draft-ietf- > > 561 spring-sr-replication-segment-19, 28 August 2023, > > 562 <https://datatracker.ietf.org/doc/html/draft-ietf-spring- > > 563 sr-replication-segment-19>. > > > > 565 [I-D.mirsky-ippm-hybrid-two-step] > > 566 Mirsky, G., Lingqiang, W., Zhui, G., Song, H., and P. > > 567 Thubert, "Hybrid Two-Step Performance Measurement Method", > > 568 Work in Progress, Internet-Draft, draft-mirsky-ippm- > > 569 hybrid-two-step-15, 15 June 2023, > > 570 <https://datatracker.ietf.org/doc/html/draft-mirsky-ippm- > > 571 hybrid-two-step-15>. > > > > 573 [I-D.song-ippm-postcard-based-telemetry] > > 574 Song, H., Mirsky, G., Zhou, T., Li, Z., Graf, T., Mishra, > > 575 G. S., Shin, J., and K. Lee, "On-Path Telemetry using > > 576 Packet Marking to Trigger Dedicated OAM Packets", Work in > > 577 Progress, Internet-Draft, draft-song-ippm-postcard-based- > > 578 telemetry-16, 2 June 2023, > > 579 <https://datatracker.ietf.org/doc/html/draft-song-ippm- > > 580 postcard-based-telemetry-16>. > > > > 582 [I-D.song-mpls-extension-header] > > 583 Song, H., Zhou, T., Andersson, L., Zhang, Z. J., and R. > > 584 Gandhi, "MPLS Network Actions using Post-Stack Extension > > 585 Headers", Work in Progress, Internet-Draft, draft-song- > > 586 mpls-extension-header-12, 14 April 2023, > > 587 <https://datatracker.ietf.org/doc/html/draft-song-mpls- > > 588 extension-header-12>. > > > > 590 [I-D.xie-bier-ipv6-encapsulation] > > 591 Xie, J., Geng, L., McBride, M., Asati, R., Dhanaraj, S., > > 592 Zhu, Y., Qin, Z., Shin, M., Mishra, G. S., and X. Geng, > > 593 "Encapsulation for BIER in Non-MPLS IPv6 Networks", Work > > 594 in Progress, Internet-Draft, draft-xie-bier-ipv6- > > 595 encapsulation-10, 22 February 2021, > > 596 <https://datatracker.ietf.org/doc/html/draft-xie-bier- > > 597 ipv6-encapsulation-10>. > > > > 599 Authors' Addresses > > > > 601 Haoyu Song > > 602 Futurewei Technologies > > 603 2330 Central Expressway > > 604 Santa Clara, > > 605 United States of America > > 606 Email: hsong@futurewei.com > > 607 Mike McBride > > 608 Futurewei Technologies > > 609 2330 Central Expressway > > 610 Santa Clara, > > 611 United States of America > > 612 Email: mmcbride@futurewei.com > > > > 614 Greg Mirsky > > 615 ZTE Corp. > > 616 Email: gregimirsky@gmail.com > > > > 618 Gyan Mishra > > 619 Verizon Inc. > > 620 Email: gyan.s.mishra@verizon.com > > > > 622 Hitoshi Asaeda > > 623 National Institute of Information and Communications Technology > > 624 4-2-1 Nukui-Kitamachi, Tokyo > > 625 184-8795 > > 626 Japan > > 627 Email: asaeda@nict.go.jp > > > > 629 Tianran Zhou > > 630 Huawei Technologies > > 631 Beijing > > 632 China > > 633 Email: zhoutianran@huawei.com > > > > > > > > > > > > > > > > > > > > > > _______________________________________________ > > MBONED mailing list > > MBONED@ietf.org > > https://www.ietf.org/mailman/listinfo/mboned > > > -- --- tte@cs.fau.de

- [ippm] WGLC for draft-ietf-mboned-multicast-telem… Leonard Giuliano
- Re: [ippm] [MBONED] WGLC for draft-ietf-mboned-mu… Jingrong Xie
- Re: [ippm] [MBONED] WGLC for draft-ietf-mboned-mu… Linda Dunbar
- Re: [ippm] WGLC for draft-ietf-mboned-multicast-t… Giuseppe Fioccola
- Re: [ippm] [MBONED] WGLC for draft-ietf-mboned-mu… Greg Mirsky
- Re: [ippm] WGLC for draft-ietf-mboned-multicast-t… duzongpeng@foxmail.com
- Re: [ippm] WGLC for draft-ietf-mboned-multicast-t… Tianran Zhou
- Re: [ippm] [MBONED] WGLC for draft-ietf-mboned-mu… Xuewei Wang
- Re: [ippm] [MBONED] WGLC for draft-ietf-mboned-mu… Toerless Eckert
- Re: [ippm] [MBONED] WGLC for draft-ietf-mboned-mu… Gyan Mishra
- Re: [ippm] [MBONED] WGLC for draft-ietf-mboned-mu… Hitoshi Asaeda
- Re: [ippm] WGLC for draft-ietf-mboned-multicast-t… Thomas.Graf
- Re: [ippm] WGLC for draft-ietf-mboned-multicast-t… Ernesto Ruffini
- Re: [ippm] [MBONED] WGLC for draft-ietf-mboned-mu… Toerless Eckert
- Re: [ippm] [MBONED] WGLC for draft-ietf-mboned-mu… Haoyu Song
- Re: [ippm] WGLC for draft-ietf-mboned-multicast-t… Henrik Nydell
- Re: [ippm] WGLC for draft-ietf-mboned-multicast-t… 庞冉(联通集团本部)
- Re: [ippm] WGLC for draft-ietf-mboned-multicast-t… Chongfeng Xie
- Re: [ippm] WGLC for draft-ietf-mboned-multicast-t… Aijun Wang
- Re: [ippm] WGLC for draft-ietf-mboned-multicast-t… Leonard Giuliano
- Re: [ippm] WGLC for draft-ietf-mboned-multicast-t… Haoyu Song
- Re: [ippm] [MBONED] WGLC for draft-ietf-mboned-mu… Leonard Giuliano