[Detnet] 答复: H3C comments for (draft-liu-detnet-large-scale-requirements)

[Zhushiyin <[e-mail]>]

Hi Fan and Peng,
    Thank for your comments.  Please see inline starts with Shiyin for my clarification, thanks.

发件人: Yangfan(Fan,IP Standards) [mailto:[e-mail]]
发送时间: 2022年8月4日 19:19
收件人: Peng Liu <[e-mail]>; zhoulei (CTS) <[e-mail]>
抄送: detnet <[e-mail]>; zhushiyin (CTS) <[e-mail]>; chengzuopin (CTS) <[e-mail]>
主题: RE: [Detnet] H3C comments for (draft-liu-detnet-large-scale-requirements)

Hi Lei, Shiyin and Peng,

Thank you for the contributions. I have a couple of questions for clarification, please see inline starts with Fan. Thank you.

Best,
Fan


From: detnet [mailto:[e-mail]] On Behalf Of Peng Liu
Sent: Tuesday, August 2, 2022 10:55 AM
To: zhou.leih <[e-mail]<mailto:[e-mail]>>
Cc: detnet <[e-mail]<mailto:[e-mail]>>; Zhushiyin <[e-mail]<mailto:[e-mail]>>; Chengzuopin <[e-mail]<mailto:[e-mail]>>
Subject: Re: [Detnet] H3C comments for (draft-liu-detnet-large-scale-requirements)

Hi Lei, Shiyin,

Thanks for your comments. I looked through the suggested text and the here are the preliminary replies:

For section 4.2, I 'd like to keep it.
For section 4.4, I suggest to keep the first paragraph and rewording it as " Moreover, the edge shaping based solution to reduce the micro-burst may also work to some extent." I think the second paragraph is better to be added in section 4.6.
For section 4.5, I'd like to keep it while rewording as "Moreover，a system models and design methodology are expected to support flexible planning of multiple paths and provide a solid foundation for the implementation of PREOF."
For section 4.7 and suggested section 4.8, I think they are similar to the cross domain and edge device enhancement, so the text is better to be merged into section 4.7 in current version.

The co authors will discuss more about the suggested text and then confirm with you.

Regards,
Peng

________________________________
[e-mail]<mailto:[e-mail]>

From: Zhoulei<mailto:[e-mail]>
Date: 2022-08-01 11:33
To: [e-mail]<mailto:[e-mail]>
CC: detnet<mailto:[e-mail]>; Zhushiyin<mailto:[e-mail]>; Chengzuopin<mailto:[e-mail]>
Subject: [Detnet] H3C comments for (draft-liu-detnet-large-scale-requirements)
Peng,

Thanks so much for your good summary and discussion.

As I mentioned before in our previous email , we are very interested in large-scale deterministic network technology so we would like to contribute on the requirement draft on large-scale deterministic network technology.

All in all, on behalf of Shiyin due to his email subscribe issue, I share our summarized comments on  technical requirements for large-scale deterministic network so far and further considerations with you.

If necessary, we would like to send the revised document on this draft with revision mark to you.

Any comments are appreciated.

Looking forward to your response.

===============================================================
4.2.  Support Large Single-hop Propagation Latency
  …
  For a cyclic based method, suppose a large-scale network wants to
   keep using the simple cycle mapping relationship, however the link
   distance between two nodes is longer.  Moreover, a downstream node
   may have many upstream nodes each with different link propagation
   delays (e.g., 9 us, 10 us, 11 us, 15 us and 20 us).  In order to
   absorb the longest link propagation delay, the length of cycle must
   be set to at least 20 us.  However, since packet's arrival time
   varies within the receiving cycle, larger cycle length means larger
   delay variance.
Suggest adding:
In addition, in order to support large single-hop propagation latency, the queuing mechanism for LAN networks needs to be extended, such as increasing the number of queues in 802.1QCH and specifying the corresponding forwarding period in the stream. Before that, it is required to establish a cycle mapping relationship between the nodes in the path and supporting system measurement mechanism for periodic forwarding to meet the established cycle mapping relationship

Fan: regarding “specifying the corresponding forwarding period”, I think period is not so accurate that may confuse readers 802.1Qch is EDF/LIS based.
Another clarification question regarding system measurement mechanism, what is the metric of measurement and whether the existing PM mechanisms can works,  or what’s the gap?

Shiyin：we mean the queues and  forwarding period has the cycle mapping relationship. Any mechanisms that can measure the node and link latency is OK.

 4.4.  Be Scalable to Numerous Network Devices and Massive Traffic Flows

   …
   The micro-burst will happen more often due to the massive traffic
   flows, so some methods to decrease it are
   needed.[I-D.du-detnet-layer3-low-latency]introduces a reference
   method requiring a scalable buffer to adjust the speed of sending the
   packets, so as to keep a uniform transmission rate, and it also
   support the flow aggregation.
Suggest adding:
Moreover, it is required to provide unified mechanisms to resolve the micro-burst problem within the deterministic domain, especially incast  problem within the large-scale nonlinear topology domain. The micro-burst is limited to edge access and jitter is reduced by the edges shaping so as to achieve bounded latency.
 Fan: mechanism to reduce micro burst is necessary and a local behavior, I don’t know how do you mean “unified”. Can you explain more?

Shiyin: “unified” means we need to look at the micro burst from a global perspective and try to avoid the occurrence of micro bursts in the forwarding node, because the collision of resources will bring delay jitter.


In addition, for refined management of forward resources and providing resource assurance for deterministic forwarding when establishing/ deleting sessions, it is required to establish unified mechanisms on quantification and measurement of resources associated with interface and queue
 Fan: resource collection and allocation should be covered in controller plane. Same doubt here about the unified mechanism.

Shiyin: correct, resource collection and allocation will be done in controller plane, like SDN controller.


4.5 Tolerate Failures of Links or Nodes and Topology Changes

   Network link failures are more common in large-scale networks.  Path
   switching or re-convergence of routing will cause high latency of
   packet loss and retransmission, which is usually in seconds before
   the network becomes stable again.  It is necessary to support certain
   mechanisms to adapt to failures of links or nodes and topology
   changes.

   The change of path or topology poses a higher challenge to packet
   replication and elimination.  The full disjoint paths when
   implementing the Packet Replication, Elimination, and Ordering
   Functions (PREOF) gives a better chance of survival when one of the
   nodes or links in the path fails.  At the same time, it brings the
   challenges of finding paths with similar distance and/or number of
   hops so that there is enough buffer space to absorb the latency
   difference caused by different paths when the scale is large.
Suggest adding:
Therefore, it is required to provide system models and design methodology so that the transmission of deterministic traffic flows through different paths has similar transmission latency, so as to support flexible planning of multiple paths and provide a solid foundation for the implementation of PREOF.
 Fan: how do you mean system models and design methodology? Are they algorithms of controller and implementation specific?
Since the section title relates to topology change, I suggest to add: it is also required to use mechanisms to verify the status of the PREOF multiple paths to provide path adjustment feedback.

Shiyin: we mean a mechanism is needed to select the proper multiple paths and  absorb the latency difference caused by different paths.


4.7.  Support Queueing Mechanisms Switchover Crossing Multi-domains

   In large-scale deterministic networks, it may across multiple network
   domains and adopt a variety of different queueing mechanisms within
   each domain.  It is required to support the inter-domain
   deterministic mechanism at the inter-domain boundary nodes such as
   the priority redefinition and rescheduling of queues to achieve the
   end-to-end latency, bounded jitter and packet loss ratio.

   Moreover, changing from one queueing mechanism to another may
   generate additional end-to-end latency and/or jitter which should be
   taken into consideration.  For example, when a flow is forwarded
   across multiple network domains based on different queueing
   mechanisms, such as a time synchronous Qbv mechanism[IEEE802.1Qbv]
   and an asynchronous Qcr mechanism [IEEE802.1Qcr], a collaboration
   mechanism crossing multi-domains MUST be considered, such as
   increasing the buffer of inter-domain devices to provide enough
   adjustment space for the flow to cross different queueing mechanisms,
   so as to provide end-to-end deterministic services across multiple
   network domains.

Suggest adding:
At the same time, due to different scheduling granularities or phase differences between the two domains, it is required that queuing Mechanisms support flow aggregation function and queue stitching function. Furthermore, it is necessary to reduce the coupling between two domains as much as possible, as long as the network meets a certain range of requirements and end-to-end devices implement the jitter compression, end-to-end deterministic latency services can be supported.

In addition, it is required to provide the feasibility of end-to-end DetNet service flow scheduling in a large-scale network and solve the bandwidth, cycle, and queue mismatch problems in the cycle-based cross-domain scheduling. Queueing mechanisms vary in different network standards or drafts (e.g., TSN, DetNet). In order to guarantee the bounded delay of these queue mechanisms in different domains, additional traffic shaping solutions need to be introduced at the edge of the network and between domains. These solutions are expected to eliminate the non-deterministic factors (additional delay and jitter) caused by mismatches between different queue mechanisms such as bandwidth, queue number, cycle size, etc.

Fan: a clarification question on what is the definition of “domain” in this draft. Is it administrative or technical (queuing) specific defined in ITU-T documents?
The functions mentioned above are very similar to the functions of service proxy on DetNet Edge Node (T-PE), which sits on the border of TSN and DetNet. But since DetNet transit node doesn’t include this function and detnet node has no idea about the mechanism used in other nodes, what would be required if different queuing mechanisms are used within one domain?

Shiyin:  “domain” means the region that has the different schedule mechanism like TSN and DetNet. The node that connect the TSN and DetNet need a queue mechanisms to pass through different domains.


Suggest adding new chapter “4.8 Supports hierarchical deterministic peers”.
Suggest adding:
4.8 Supports hierarchical deterministic peers
Large-scale deterministic network can often cross multiple domains, and one of DetNet network's goals is to provide end-to-end deterministic latency services. The technology and capabilities of each domain are different and the corresponding topology models are either segmented or nested. In this way, it is necessary to reduce the coupling between two domains as much as possible, as long as the network meets a certain range of requirements and end-to-end devices implement the jitter compression, end-to-end deterministic latency services can be supported. For example, the two-end user networks are carried by the operator's intermediate network. As long as the operator's intermediate network ensures its end-to-end latency, bounded jitter and reliability capabilities for deterministic flow,  the operator network can be made as a pipeline for the two-end user network. So gateway equipment deployed on both ends utilizes these mechanisms including flow access control, traffic shaping, traffic mark, deterministic scheduling, and jitter compression to meet the end-to-end deterministic latency service.

Fan: much related to the deployment and not sure it’s appropriate to put it in a requirement document.

Shiyin: As Pend’s advice , this section can be merged into section 4.7.


===================================================================

BR
Shiyin, Lei & Zuopin
New H3C
From: zhushiyin (CTS)
Sent: Monday, August 1, 2022 10:49 AM
To: zhoulei (CTS) <[e-mail]<mailto:[e-mail]>>
Subject: H3C comments for detnet

Hi lei
  Because my email subscribe issue, please represent me to send H3C comments to the detnet group. thanks


4.2.  Support Large Single-hop Propagation Latency
  …
  For a cyclic based method, suppose a large-scale network wants to
   keep using the simple cycle mapping relationship, however the link
   distance between two nodes is longer.  Moreover, a downstream node
   may have many upstream nodes each with different link propagation
   delays (e.g., 9 us, 10 us, 11 us, 15 us and 20 us).  In order to
   absorb the longest link propagation delay, the length of cycle must
   be set to at least 20 us.  However, since packet's arrival time
   varies within the receiving cycle, larger cycle length means larger
   delay variance.
Suggest adding:
In addition, in order to support large single-hop propagation latency, the queuing mechanism for LAN networks needs to be extended, such as increasing the number of queues in 802.1QCH and specifying the corresponding forwarding period in the stream. Before that, it is required to establish a cycle mapping relationship between the nodes in the path and supporting system measurement mechanism for periodic forwarding to meet the established cycle mapping relationship


4.4.  Be Scalable to Numerous Network Devices and Massive Traffic Flows

   …
   The micro-burst will happen more often due to the massive traffic
   flows, so some methods to decrease it are
   needed.[I-D.du-detnet-layer3-low-latency]introduces a reference
   method requiring a scalable buffer to adjust the speed of sending the
   packets, so as to keep a uniform transmission rate, and it also
   support the flow aggregation.
Suggest adding:
Moreover, it is required to provide unified mechanisms to resolve the micro-burst problem within the deterministic domain, especially incast  problem within the large-scale nonlinear topology domain. The micro-burst is limited to edge access and jitter is reduced by the edges shaping so as to achieve bounded latency.

In addition, for refined management of forward resources and providing resource assurance for deterministic forwarding when establishing/ deleting sessions, it is required to establish unified mechanisms on quantification and measurement of resources associated with interface and queue




4.5 Tolerate Failures of Links or Nodes and Topology Changes

   Network link failures are more common in large-scale networks.  Path
   switching or re-convergence of routing will cause high latency of
   packet loss and retransmission, which is usually in seconds before
   the network becomes stable again.  It is necessary to support certain
   mechanisms to adapt to failures of links or nodes and topology
   changes.

   The change of path or topology poses a higher challenge to packet
   replication and elimination.  The full disjoint paths when
   implementing the Packet Replication, Elimination, and Ordering
   Functions (PREOF) gives a better chance of survival when one of the
   nodes or links in the path fails.  At the same time, it brings the
   challenges of finding paths with similar distance and/or number of
   hops so that there is enough buffer space to absorb the latency
   difference caused by different paths when the scale is large.
Suggest adding:
Therefore, it is required to provide system models and design methodology so that the transmission of deterministic traffic flows through different paths has similar transmission latency, so as to support flexible planning of multiple paths and provide a solid foundation for the implementation of PREOF.


4.7.  Support Queueing Mechanisms Switchover Crossing Multi-domains

   In large-scale deterministic networks, it may across multiple network
   domains and adopt a variety of different queueing mechanisms within
   each domain.  It is required to support the inter-domain
   deterministic mechanism at the inter-domain boundary nodes such as
   the priority redefinition and rescheduling of queues to achieve the
   end-to-end latency, bounded jitter and packet loss ratio.

   Moreover, changing from one queueing mechanism to another may
   generate additional end-to-end latency and/or jitter which should be
   taken into consideration.  For example, when a flow is forwarded
   across multiple network domains based on different queueing
   mechanisms, such as a time synchronous Qbv mechanism[IEEE802.1Qbv]
   and an asynchronous Qcr mechanism [IEEE802.1Qcr], a collaboration
   mechanism crossing multi-domains MUST be considered, such as
   increasing the buffer of inter-domain devices to provide enough
   adjustment space for the flow to cross different queueing mechanisms,
   so as to provide end-to-end deterministic services across multiple
   network domains.

Suggest adding:
At the same time, due to different scheduling granularities or phase differences between the two domains, it is required that queuing Mechanisms support flow aggregation function and queue stitching function. Furthermore, it is necessary to reduce the coupling between two domains as much as possible, as long as the network meets a certain range of requirements and end-to-end devices implement the jitter compression, end-to-end deterministic latency services can be supported.

In addition, it is required to provide the feasibility of end-to-end DetNet service flow scheduling in a large-scale network and solve the bandwidth, cycle, and queue mismatch problems in the cycle-based cross-domain scheduling. Queueing mechanisms vary in different network standards or drafts (e.g., TSN, DetNet). In order to guarantee the bounded delay of these queue mechanisms in different domains, additional traffic shaping solutions need to be introduced at the edge of the network and between domains. These solutions are expected to eliminate the non-deterministic factors (additional delay and jitter) caused by mismatches between different queue mechanisms such as bandwidth, queue number, cycle size, etc.

Suggest adding new chapter “4.8 Supports hierarchical deterministic peers”.
4.8 Supports hierarchical deterministic peers
Large-scale deterministic network can often cross multiple domains, and one of DetNet network's goals is to provide end-to-end deterministic latency services. The technology and capabilities of each domain are different and the corresponding topology models are either segmented or nested. In this way, it is necessary to reduce the coupling between two domains as much as possible, as long as the network meets a certain range of requirements and end-to-end devices implement the jitter compression, end-to-end deterministic latency services can be supported. For example, the two-end user networks are carried by the operator's intermediate network. As long as the operator's intermediate network ensures its end-to-end latency, bounded jitter and reliability capabilities for deterministic flow,  the operator network can be made as a pipeline for the two-end user network. So gateway equipment deployed on both ends utilizes these mechanisms including flow access control, traffic shaping, traffic mark, deterministic scheduling, and jitter compression to meet the end-to-end deterministic latency service.
-------------------------------------------------------------------------------------------------------------------------------------
本邮件及其附件含有新华三集团的保密信息，仅限于发送给上面地址中列出
的个人或群组。禁止任何其他人以任何形式使用（包括但不限于全部或部分地泄露、复制、
或散发）本邮件中的信息。如果您错收了本邮件，请您立即电话或邮件通知发件人并删除本
邮件！
This e-mail and its attachments contain confidential information from New H3C, which is
intended only for the person or entity whose address is listed above. Any use of the
information contained herein in any way (including, but not limited to, total or partial
disclosure, reproduction, or dissemination) by persons other than the intended
recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender
by phone or email immediately and delete it!