Re: [ippm] 回复: FW: WG Adoption Call for draft-tgraf-opsawg-ipfix-on-path-telemetry-01

[Tianran Zhou <[e-mail]>]

Hi Zhenqiang and Thomas,

Thanks very much for sharing your experience and the discussion in the mailing list.
IMO, whether to use gRPC or IPFIX is not a technique problem, but different choice/preference from users.
I agree with Zhenqiang our intention is not to have many options for the same feature.
But the world is diverse. Like we always have the same IGP extension for both OSPF and ISIS. At present I cannot see the dominance of one protocol.
I have practice on both gRPC and IPFIX. I believe both approaches can achieve this case.
Because this is neither large volume nor frequent.
The fast path will process delay per-packet, but the export will only work on mean/min/max delay.

Best,
Tianran

From: [e-mail] [mailto:[e-mail]]
Sent: Wednesday, January 11, 2023 4:23 PM
To: Benoit Claise <[e-mail]>; [e-mail]; Tianran Zhou <[e-mail]>; ippm <[e-mail]>
Cc: [e-mail]
Subject: Re: Re: [ippm] 回复: FW: WG Adoption Call for draft-tgraf-opsawg-ipfix-on-path-telemetry-01

Hello Mr. Claise,

please see my comments beginning with [ZQ].

________________________________
Best Regards,
Zhenqiang Li

[e-mail]<mailto:[e-mail]>

发件人： Benoit Claise<mailto:[e-mail]>
发送时间： 2023-01-10 05:15
收件人： [e-mail]<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; zhoutianran=[e-mail]<mailto:zhoutianran=[e-mail]>; [e-mail]<mailto:[e-mail]>
抄送： [e-mail]<mailto:[e-mail]>
主题： Re: [ippm] 回复: FW: WG Adoption Call for draft-tgraf-opsawg-ipfix-on-path-telemetry-01
Dear all,

I agree with Thomas on a couple of points.

What is important is that, for accounting information, monitored at high frequency, such as IPFIX flow records metering data plane traffic, should be exported directly from line cards, without proxying to the route processor. As such, UDP is the way to go (*). On top of that, losing an export packet or two is not the end of the world.
In this case (draft-tgraf-opsawg-ipfix-on-path-telemetry-01), where we monitor delay over (all packets of) the flow record lifetime, it's the same principle: we need the UDP-based export. So the IPFIX protocol.
[ZQ] I don't think it is apropriate to limit the implementation especailly the internal software and hardware architecture when we define protocol. Our lab tests show gRPC works well for exporting the iOAM metrics and gRPC is TCP based. I am not sure whether or not  our vendors implement this by  proxying the collected data plane information to the route processor and then packeting and exporting all the relevant information to the colloctor. Why do we need to tell the vendors how to do it? From the specification point, IPFIX does not require UDP,  TCP based SCTP is a valid option.

[ZQ] I agree with you that the packets of iOAM flow should not be sampled, i.e. all packets of the flow need to be monitored.

Btw, the flow record lifetime is configurable, based on timers... in case you want to act on the delay (in the flow record) faster... at the cost of exporting more flow records.

[ZQ] iOAM metrics should be exported in seconds, in minutes is not acceptable. IPFIX exporting timer is usually in minutes.

Now, if we speak about events, this is a different story, as events needs to acted upon. So a reliable transport is required here.

Let's keep in mind that we speak about hybrid type I (see https://www.rfc-editor.org/rfc/rfc7799#section-3.8), so the issue of isolating the specific packets to look at at intermediate node is trivial.

(*) and this is the reason why SCTP, the compulsory export protocol in RFC 7011 was never a success. See https://www.claise.be/from-netflow-to-ipfix-via-psamp-13-years-of-standardization-explained-2/

Regards, Benoit
On 1/9/2023 11:06 AM, [e-mail]<mailto:[e-mail]> wrote:
Dear Zhenqiang,

See response inline.

Best wishes
Thomas

From: [e-mail]<mailto:[e-mail]> <[e-mail]><mailto:[e-mail]>
Sent: Monday, January 9, 2023 8:43 AM
To: Graf Thomas, INI-NET-VNC-HCS <[e-mail]><mailto:[e-mail]>; Tianran Zhou <zhoutianran=[e-mail]><mailto:zhoutianran=[e-mail]>; ippm <[e-mail]><mailto:[e-mail]>
Cc: [e-mail]<mailto:[e-mail]>
Subject: Re: RE: [ippm] 回复: FW: WG Adoption Call for draft-tgraf-opsawg-ipfix-on-path-telemetry-01

DearThomas,

Please see my further discussion beginning with [ZQ].

________________________________
Best Regards,
Zhenqiang Li

[e-mail]<mailto:[e-mail]>

发件人： [e-mail]<mailto:[e-mail]>
发送时间： 2023-01-07 01:23
收件人： [e-mail]<mailto:[e-mail]>; zhoutianran=[e-mail]<mailto:zhoutianran=[e-mail]>; [e-mail]<mailto:[e-mail]>
抄送： [e-mail]<mailto:[e-mail]>
主题： RE: RE: [ippm] 回复: FW: WG Adoption Call for draft-tgraf-opsawg-ipfix-on-path-telemetry-01
Dear Zhenqiang,

Thanks a lot for the feedback. Much appreciated.

I do not disagree that YANG push isn't capable of exporting control and forwarding plane metrics. However it is not the best choice in terms of scale. Table 1 of RFC 9232 gives a good summary. It even makes the distinction between network processor and route processor export. What it doesn't show however is that gRPC isn't suitable for network processors. To date, no vendor implementations are present. From my research with network and ASIC vendors as being the co-author of draft-ietf-netconf-distributed (https://datatracker.ietf.org/doc/html/draft-ietf-netconf-distributed-notif<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-netconf-distributed-notif&data=05%7C01%7CThomas.Graf%40swisscom.com%7C502beb5de8ee458ffbcb08daf214f30f%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C638088469032680658%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=sA%2FvpsWKZOlBaLZEvBESmZZ8WFMKsw2Az7Wf54Og8KQ%3D&reserved=0>) I understood that gRPC won't be implementable in its current form. One of the main reason is the lack of backpressure support on network processors.

I fully agree that export for probing metrics such a TWAMP, TWAMP light or STAMP makes perfectly sense. With draft-ietf-ippm-stamp-yang and RFC 8913 we have well defined YANG modules. Speaking as network operator with approx. 7000 YANG publishers I can confirm that this works well. Either when both probing and export runs on the route processor or both on the network processor directly.



[ZQ] For export implementaion of data plane iOAM, we'd better hear the voice from vendors. Some co-authors of this doc are from vendors. As I stated in the previous mail,  almost all our vendors have already supported exporting passport iOAM metrics by gRPC with GPB encoding. We have finished lab tests with good results and have plan to deploy this kind of data plane iOAM in our field network this year. As for the implementation, I don't know the details. There are several other processors on the line card besides network processor.

[TG] The network processor is the processor which forwards the packet and export the forwarding plane metrics. Ideally directly without proxying over the route processor. We are in contact with major vendors since over a year and have two working implementations and working on another technical feasibility. The results will be presented at IETF 116 hackathon and documented in the next version.


•         In my understanding, the aggregation benifit of IPFIX is not applicable for iOAM metrics, such as delay and packet loss, because iOAM metrics are different for different flows.

This I don't understand fully. Could you describe more detailed please.

To my understanding and maybe this helps to establish a clearer picture. IOAM is applied to existing or replicated packets and is classified as One-Way Delay Hybrid Type I according to Section 1 of draft-ietf-ippm-ioam-deployment (https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-deployment#section-1<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-ippm-ioam-deployment%23section-1&data=05%7C01%7CThomas.Graf%40swisscom.com%7C502beb5de8ee458ffbcb08daf214f30f%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C638088469032680658%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=94g93GZq4TB7diC2nK5TcXCLBESnS6mUfITwKyq08Q8%3D&reserved=0>). There are two modes. Active and Passive. In passive, where draft-tgraf-opsawg-ipfix-on-path-telemetry focuses on, existing packets are being used. Where in active mode, existing packets are replicated. IPFIX takes the packets and accounts them into flows. In flows we account bytes and packets over a given time period. With draft-tgraf-opsawg-ipfix-on-path-telemetry we also account for delay.



[ZQ] I am mainly talking about passive mode. Bytes and packets can be accumulated for flows. But I can not imagine how to accumulate delays for flows? For my understanding, aggregation is different from accumulation. For example, Bytes and packets accumulated for different flows can be further aggregated on destination prefix if configured and if the destinations of the flows belong to one common route entry in the routing talbe. Since delay metric may be different for different flows, it is not appropriate to aggregate this metric based on destionation, AS, community etc.

[TG] As mentioned in my previous email. The delay is accounted per flow. Section 1 (https://datatracker.ietf.org/doc/html/draft-tgraf-opsawg-ipfix-on-path-telemetry-01#section-1) describes how the delay is being measured. I am quoting:

The delay is measured by calculating the difference between the timestamp imposed with On-Path Telemetry in the packet at the IOAM encapsulation node and the timestamp exported in the IPFIX flow record from the IOAM transit and decapsulation nodes. Depending on the IE, the lowest, highest or the sum of measured path delay is being exported.

                                IOAM-Domain
              .........................................
              .                                       .
              .    D1                                 .
              . <------>                              .
              .                                       .
              .          D2                           .
              . <-------------------->                .
              .                                       .
              .                  D3                   .
              . <-----------------------------------> .
              .                                       .
(H1) ------ (R1) ------- (R2) ------- (R3) -------- (R4) ------ (H2)
Host 1  Encapsulation   Transit      Transit   Decapsulation  Host 2
            Node         Node 1       Node 2        Node
              .                                       .
              .                                       .
              .........................................

Figure 1: IOAM Delay use case. Packets flow from host 1 to host 2.

On the usecase showed in Figure 1 using IOAM to export the delay metrics, the node R2 exports the delay D1, the node R3 exports the delay D2 and the decapsulation node R4 exports the total delay D3 using IPFIX.

[TG] The same as IPFIX aggregates for packets and bytes, the sum of the delay for all flows is accounted in PathDelaySumDeltaMicroseconds/PathDelaySumDeltaNanoseconds. The PathDelayMeanDeltaMicroseconds/PathDelayMeanDeltaNanoseconds is calculated as described in Section 7.2 https://datatracker.ietf.org/doc/html/draft-tgraf-opsawg-ipfix-on-path-telemetry-01#section-7.2.


•         Neither sampling, how can you insure that the flow you want its iOAM metrics will be sampled?

Packets have to be selected in the IOAM encapsulation node as described in Section 7.4 of draft-ietf-ippm-ioam-deployment https://datatracker.ietf.org/doc/html/draft-ietf-ippm-ioam-deployment#section-7.4<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-ietf-ippm-ioam-deployment%23section-7.4&data=05%7C01%7CThomas.Graf%40swisscom.com%7C502beb5de8ee458ffbcb08daf214f30f%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C638088469032680658%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=GgV98jt9iibX3ctGw02wMBl81nbjabGxkDg6GL%2F8QhE%3D&reserved=0>. IOAM doesn't specify how the packets are being selected. To my opinion it would make sense to refer in Section 7.4 of draft-ietf-ippm-ioam-deployment to RFC 5474 and RFC 5475 which are describing packet selection and sampling techniques, both widely used with IPFIX.

The benefit of sampling at the IOAM encapsulation node and flag with IOAM-DEX versus sampling on all nodes is that the amount of flows being exported is the same, but in case of IOAM-DEX we know the forwarding path in addition without additional overhead in the data export.



[ZQ] Yes, agree that packets have to be selected for iOAM but must be flow based and must not be further sampled for the selected flows in my understanding because some underlay supporting mechanisms such as alternative marking require consecutive packets. Usually we configure sampling rate for IPFIX, 1:5000 for example. But we can not insure that the packet sampled on node1 will be exactly sampled at the next node on its traverse path. Thus it is difficult to get the delay metric when working in this way.

[TG] Exactly. The difference to traditional IPFIX is that in IOAM only samples/selects once, at the edge on the IOAM encapsulation node.


•         The iOAM metrics, especially the delay and packet loss, should be exported in time, for example in one second. I don't think IPFIX is the right protocol.

For packet loss, you are correct, IPFIX is not the right protocol since packet loss is observed at the edges of the observation domain (black box monitoring). Packet loss is observed with probing where YANG push is the best choice for data export. IPFIX is being used to visualize where the packets are being dropped for which flow with IE89 ForwardingStatus.

[ZQ] Again, it is better to export all the data plane iOAM metrics by one protocol if possible.

IPFIX can export metrics without aggregation where flows are immediately being exported at ingress or egress. It is correct that aggregation introduces delay. Therefore it should only be applied to use cases which make sense https://www.rfc-editor.org/rfc/rfc7015#section-3<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.rfc-editor.org%2Frfc%2Frfc7015%23section-3&data=05%7C01%7CThomas.Graf%40swisscom.com%7C502beb5de8ee458ffbcb08daf214f30f%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C638088469032680658%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=D2o7UpmX5sG5cvM9auSL%2F5%2BiENpfkZuNICY4UzRBZis%3D&reserved=0>. What I like to highlight is that in IPFIX supports an inactice timeout window. This allows the data export to be exported as quickly as possibly without waiting to finish the active timeout window.

[ZQ] As I know, there is no immediate export working mode for IPFIX. IPFIX exports flow information when the flow is aged or over, when the configured export interval expires. At present, the export interval is in minute which can not satisfy the time requirement of iOAM.

[TQ] As described previously described IPFIX indeed supports it by default. RFC 7011 doesn't mandate aggregation. RFC 7015 describes this in section 4.2 figure 4 https://www.rfc-editor.org/rfc/rfc7015#section-4.2. Here an example how this is configured in Cisco IOS XR (https://www.cisco.com/c/en/us/td/docs/routers/asr9000/software/netflow/command/reference/b-netflow-cr-asr9k/netflow-commands.html#wp1580058501). There are many other vendor implementations.

Therefore using aggregation depends on the use case, the objective. Speaking for a network operator with a large scale Network Telemetry pipeline one of the key objectives is to have a wide view across the network while minimizing the amount of data as much and as early in the pipeline as possible. For automated traffic verification of several thousands of L3 VPN's a highly aggregated data set makes more sense then when troubleshooting of a particular customer flow across the network is needed. Here cardinality counts. Therefore we use with RFC7015 a two step data aggregation concept. One where the aggregation is performed on the network node without loosing cardinality. Where at data-collection through replication a second aggregation is performed where cardinality is reduced to a minimum. Allowing latency queries on the data set.

Looking forward to your comments.

Best wishes
Thomas

From: [e-mail]<mailto:[e-mail]> <[e-mail]<mailto:[e-mail]>>
Sent: Friday, January 6, 2023 3:51 PM
To: Graf Thomas, INI-NET-VNC-HCS <[e-mail]<mailto:[e-mail]>>; Tianran Zhou <zhoutianran=[e-mail]<mailto:zhoutianran=[e-mail]>>; ippm <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>
Subject: Re: RE: [ippm] 回复: FW: WG Adoption Call for draft-tgraf-opsawg-ipfix-on-path-telemetry-01

Hello Thomas,

As summarized in table 1 in RFC9232, gRPC is an application protocol, which can be used to export all the telemetry metrics for Management Plane, Control Plane, Forwarding Plane and External Data. By using GPB encoding, gRPC is widely used to export  real time performace metrics, such as link delay, jitter, packet loss measured by TWAMP or STAMP. Almost all our vendors support this way to export passport iOAM metrics and the test results in our lab are good. We have plan to deploy it in our field network soon.

In my understanding, the aggregation benifit of IPFIX is not applicable for iOAM metrics, such as delay and packet loss, because iOAM metrics are different for different flows. Neither sampling, how can you insure that the flow you want its iOAM metrics will be sampled? The iOAM metrics, especially the delay and packet loss, should be exported in time, for example in one second. I don't think IPFIX is the right protocol.

I do not object this doc to move forward if we can reach this consensus.

________________________________
Best Regards,
Zhenqiang Li

[e-mail]<mailto:[e-mail]>

发件人： [e-mail]<mailto:[e-mail]>
发送时间： 2023-01-04 00:20
收件人： [e-mail]<mailto:[e-mail]>; zhoutianran=[e-mail]<mailto:zhoutianran=[e-mail]>; [e-mail]<mailto:[e-mail]>
抄送： [e-mail]<mailto:[e-mail]>
主题： RE: [ippm] 回复: FW: WG Adoption Call for draft-tgraf-opsawg-ipfix-on-path-telemetry-01
Dear Zhenqiang,

Thanks a lot for your feedback.

I presume with gRPC you are referring to YANG push (RFC 8639, RFC 8641, draft-ietf-netconf-udp-notif, draft-ietf-netconf-https-notif). gNMI (gRPC is the transport of gNMI) has been proposed (draft-openconfig-rtgwg-gnmi-spec) in 2018 but not standardized within IETF as a transport protocol for YANG push. It did not find enough traction. A good overview about the state of the union about YANG push in the industry is a presentation I gave at the IEPG, slide 3-6, http://www.iepg.org/2022-11-06-ietf115/slides-115-iepg-sessa-network-operator-challenges-in-network-telemetry-data-mesh-integration-thomas-graf-00.pdf<https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.iepg.org%2F2022-11-06-ietf115%2Fslides-115-iepg-sessa-network-operator-challenges-in-network-telemetry-data-mesh-integration-thomas-graf-00.pdf&data=05%7C01%7CThomas.Graf%40swisscom.com%7C502beb5de8ee458ffbcb08daf214f30f%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C638088469032680658%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=9usAacXd2s7rhe1h2xxMnjdXVzbqSU%2BgOf57k01jO5I%3D&reserved=0>.

RFC 9232 gives a good overview about Network Telemetry. In section 3.1.3 (https://datatracker.ietf.org/doc/html/rfc9232#section-3.1.3<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Frfc9232%23section-3.1.3&data=05%7C01%7CThomas.Graf%40swisscom.com%7C502beb5de8ee458ffbcb08daf214f30f%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C638088469032680658%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HGGNAm4pzdE2cV0BNuy2Kzjq1G8xysgqizEQ7E1LSNI%3D&reserved=0>) is a summary of the protocols for forwarding plane data collection. As you pointed out, it makes sense to use one common Network Telemetry protocol. However today we have 3, IPFIX for data plane, BMP for BGP routing control-plane and YANG push for management-plane. These 3 protocols have different unique mandatory characteristics. IPFIX allows data reduction with sampling (RFC5476) an aggregation (RFC7015). While BMP allows to mirror BGP PDU's (lightweight) and add device dimensions such as peering, RIB and route-policy (context). And YANG push allows through its sophisticated data modelling to have configurational and operational metrics within a single data model.

In OPSAWG presentation of IETF 115 (https://datatracker.ietf.org/meeting/115/materials/slides-115-opsawg-export-of-on-path-delay-in-ipfix-00.pdf<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fmeeting%2F115%2Fmaterials%2Fslides-115-opsawg-export-of-on-path-delay-in-ipfix-00.pdf&data=05%7C01%7CThomas.Graf%40swisscom.com%7C502beb5de8ee458ffbcb08daf214f30f%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C638088469032680658%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YZ99wLc%2BMdCmXbYk1LLgyPWYFy5dVuLig8TdR1FIZzs%3D&reserved=0>) I described in slide 2 and 3 the benefit of adding the delay measurements to IPFIX. Having one single protocol for data-plane data collection. The ability to perform data correlation and aggregation with an existing proven IPFIX protocol. Does that resonate with you?

The use cases are described in section 5 of the draft (https://datatracker.ietf.org/doc/html/draft-tgraf-opsawg-ipfix-on-path-telemetry-01#section-5<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fhtml%2Fdraft-tgraf-opsawg-ipfix-on-path-telemetry-01%23section-5&data=05%7C01%7CThomas.Graf%40swisscom.com%7C502beb5de8ee458ffbcb08daf214f30f%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C638088469032680658%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=g4f1cnIkv6OWD%2BSFlxq%2F43xhgyBCiP2%2BcgXQtr5YFpI%3D&reserved=0>). Able to see for a given egress interface, BGP next-hop or SRv6 traffic engineered path or active segment how much delay at which node we accumulate. With YANG push this would be rather difficult and expensive to achieve. This was also confirmed by large network operators in their first on-path delay measurement deployments.

Best wishes
Thomas

From: ippm <[e-mail]<mailto:[e-mail]>> On Behalf Of [e-mail]<mailto:[e-mail]>
Sent: Tuesday, January 3, 2023 4:51 PM
To: Tianran Zhou <zhoutianran=[e-mail]<mailto:zhoutianran=[e-mail]>>; ippm <[e-mail]<mailto:[e-mail]>>
Cc: [e-mail]<mailto:[e-mail]>
Subject: [ippm] 回复: FW: WG Adoption Call for draft-tgraf-opsawg-ipfix-on-path-telemetry-01

Hi Tianran and all,

Why not use one protocol, such as grpc, to export all the iOAM metrics? It is ok to export one way delay in IPFIX. If other metrics, such as queue depth, buffer occupancy, etc,  have to be exported in grpc, it is not necessory to export one way delay in IPFIX.

________________________________
Best Regards,
Zhenqiang Li

[e-mail]<mailto:[e-mail]>

发件人： Tianran Zhou<mailto:zhoutianran=[e-mail]>
发送时间： 2022-12-22 10:40
收件人： 'IETF IPPM WG'<mailto:[e-mail]>
抄送： [e-mail]<mailto:[e-mail]>
主题： [ippm] FW: WG Adoption Call for draft-tgraf-opsawg-ipfix-on-path-telemetry-01
Hi IPPM,

The OPSAWG just started a WG adoption call for draft-tgraf-opsawg-ipfix-on-path-telemetry-01.
https://datatracker.ietf.org/doc/draft-tgraf-opsawg-ipfix-on-path-telemetry/<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-tgraf-opsawg-ipfix-on-path-telemetry%2F&data=05%7C01%7CThomas.Graf%40swisscom.com%7C502beb5de8ee458ffbcb08daf214f30f%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C638088469032680658%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=e8tWpkK4w3QDdinb4j5Bg2QaXomgUAfzWAkLbXteqzg%3D&reserved=0>

There are performance metrics registrations within this draft.
We would really appreciate your comments from IPPM.

Thanks,
Tianran

发件人: OPSAWG [mailto:[e-mail]] 代表 Tianran Zhou
发送时间: 2022年12月22日 10:26
收件人: [e-mail]<mailto:[e-mail]>
抄送: [e-mail]<mailto:[e-mail]>
主题: [OPSAWG] WG Adoption Call for draft-tgraf-opsawg-ipfix-on-path-telemetry-01

Hi WG,

This mail starts a WG Adoption Call for draft-tgraf-opsawg-ipfix-on-path-telemetry-01.
https://datatracker.ietf.org/doc/draft-tgraf-opsawg-ipfix-on-path-telemetry/<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-tgraf-opsawg-ipfix-on-path-telemetry%2F&data=05%7C01%7CThomas.Graf%40swisscom.com%7C502beb5de8ee458ffbcb08daf214f30f%7C364e5b87c1c7420d9beec35d19b557a1%7C0%7C0%7C638088469032680658%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=e8tWpkK4w3QDdinb4j5Bg2QaXomgUAfzWAkLbXteqzg%3D&reserved=0>


Please reply your supports or objections.

We would really appreciate your comments.


Since there are holidays, this call will last for 3 weeks, and end on Thursday, Jan 12, 2023.

Cheers,
Tianran (as co-chairs)



_______________________________________________

ippm mailing list

[e-mail]<mailto:[e-mail]>

https://www.ietf.org/mailman/listinfo/ippm