Re: [Detnet] Comments on draft-ietf-detnet-scaling-requirements-03.txt

[Peng Liu <[e-mail]>]

Hi Kiran and Quan,

Thanks for your comments and discussions, I think most of the issues have been answered and we will revise the document accordingly. Here are some points I want to clarify more: 

<Section 3.1.1.: ‘This can be done, for example, by putting extra buffer space at the ingress of a new domain, increasing the dead time as a guard <band, or using some timing compensation mechanism.’ >> Can you provide references to dead time and timing compensation details for offline <reading? I am not familiar with technical details here but identifying 'how much' 2 domains are out of sync is also a requirement. Similarly, in <section 3.1.1 should we not say "identify (or estimate), “recover and absorb such time variance..." 
Quan-> The TSN connections scenario is a traditional use case for DetNet. I am not sure that it is a necessary requirement especially when DetNet domain provides mechanisms not requiring synchronization.The references of dead time and timing compensation details may refer to IEEE TSN such as ECQF[1]. [1]https://1.ieee802.org/tsn/802-1qdv/ 
[km] I believe that the references are necessary. When I read " increasing the dead time as a guard “ it was not obvious what the authors referring to. Either you would remove the example or provide the background context to it. Is there an expectation to be familiar with the TSN to understand the document?
[Quan]Yes, your comment will be revised in next version.

[Peng] We list some directions prevent that the requirements can't be met, however, we also need to prevent indicating any detailed solutions of the existing individual drafts. For this point, I will add the reference of Qdv, at the same time, some reference to the individual drafts were deleted as the update of the document. 

<Section 3.6 ,It is required to support bursty traffic and some methods to decrease the micro-burst. So the pre-planned , ingress traffic <conditioning, scalable queuing, and enhanced capacity of buffer are required to accommodate the flow fluctuation, >> It is slightly difficult to infer <whether traffic above means mixed traffic or is it always deterministic traffic? Is it expected to support both deterministic and best-effort traffic <over the same network. I'd suggest a discussion to clarify this aspect should be done (one or 2 sentences in introduction).  
Quan->I agree to make clarification for the bursty traffic. It includes the deterministic traffic and non-deterministic traffic and the ingress traffic control policy may be different. For non-deterministic traffic, the ingress may just drop it based on the policy. 
[km] ack

[Peng] The current version text is mainly for the DetNet flows themselves. According to the discussion, I'd like to add some clarfication text such as "the non-DetNet flows are also massive and may have potential impact on the scalability of the DetNet flows, for instance, cause the high utilization of the bandwidth and suppress the possibility of more resource reservation and the traffic steering of DetNet flows. However, it is assumed that there will be the strategy in the ingress to deal with the non-DetNet flows and prevent the real-time influence on the DetNet flows."

<Section 4. Data plane enhancements >>If the data plane is enhanced, should we say something about the existing DetNet data plane and <backward compatibility with it? 
Quan->Yes, I agree with you. The DetNet data plane should be enhanced and the new functions and related metadata should be supported as per [1]. Backward compatibility should be taken into account. [1]https://datatracker.ietf.org/doc/draft-xiong-detnet-large-scale-enhancements/ 
[km] I have not gone through this document. Is this a solution or requirements? If it is not a solution, it should be merged here otherwise we should  add requirements text.[Quan] I think it is a framework for enhanced DetNet data plane based on RFC8938. And I presented it in IETF meetings before. Maybe it can be merged if we get concensus after discussion.  
[Peng] The requirements document has merged some points of draft-xiong-detnet-large-scale-enhancements before adoption, some of the points haven't be merged because there is not sufficient concensus. If you think there are also the points need to be considered, we can discuss more.

<Lastly, I found control plane considerations section missing. Since large scale networks are formed from stitching of <multiple domains, at least <one activity standards worthy is how the domains will exchange information together? This is somewhat covered in Section 3.8.2. But my overall <concern with this requirements document is that it has very few constraints. There are no limits on time-sync requirements, mechanisms, <topologies, link speeds, buffers etc. Then how do we verify/claim that the DetNet flows will meet the requested QoS for all DetNet flows in that <large-scale network. Maybe we also need a requirement for control planes to signal feasibility, violations, and monitoring performance of the <deterministic flows. 
Quan->This draft mainly focus on the scaling-requirements in enhanced DetNet data plane. The control plane considerations has been discussed several times in IETF meetings before. Chairs suggested to discuss and update that in existing control plane draft[1]. And the control plane considerations and extensions has been proposed in [2][3]. Discussions and comments are very welcome. 
[1] https://datatracker.ietf.org/doc/draft-ietf-detnet-controller-plane-framework/ 
[2] https://datatracker.ietf.org/doc/draft-xiong-detnet-teas-te-extensions/ 
[3] https://www.ietf.org/archive/id/draft-xiong-detnet-large-scale-enhancements-02.html#name-controller-plane-management 
[km] I suggest to add a reference to these documents for the sake of completeness.
[km] Authors should also add a requirement that states verification mechanisms for successful delivery of flows or signaling for failures. And look forward to revision with comments addressed.
[Quan] Thanks. I will discuss with other authors to see if we should add requirements in control plane and manegement plane.

[Peng] This is a new point we have not considered before. As I understand, there is no 'verification mechanisms for successful delivery of flows or signaling for failures' in the current detnet function. Would you prefer to add it as new requirement section or be mentioned in the text?

Regards,
Peng


[e-mail]
 
From: [e-mail]
Date: 2023-08-24 19:05
To: [e-mail]
CC: [e-mail]
Subject: Re: [Detnet]Comments on draft-ietf-detnet-scaling-requirements-03.txt

Hi Kiran,

Thanks for your quick reply!
Please see inline with [Quan].

Regards,
Quan


Thanks for your prompt reply Quan! I will look forward to next revision. Added some comments inline with [km].


On August 23, 2023 at 2:47:25 AM, [e-mail] ([e-mail]) wrote:
Hi Kiran, 

Thank you for the detailed review and comments! Much appreciated. 
As a co-author, I try to reply you but I am not sure my understanding is correct. Happy to discuss with you! 
Please see inline with Quan->. 
Thanks! 


<Hello Authors, 
<My apologies for the late review. 
<The document is in a very good shape and covers several large-scale aspects. My only concern is in the very last comment at the bottom of this <email. Would like to hear your thoughts about that. Hope you find these comments useful. 
<Section 1. Introduction * There are a large number of flows which may be difficult to identify per-flow state and cause the high utilization.(Section <3.4) >>High utilization of what? - could be bandwidth, buffers, energy, cpu, etc. 
Quan->Thanks for pointing out. I think it mainly focus on high utilization of bandwidth. It is better to fix with “...and cause the high utilization of bandwidth ". 
[KM] ack. If I am right, this should apply to bursty traffic also. Right?
[Quan]Yes, I think so.

< * The mechanisms used to ensure bounded latency (e.g. queuing mechanism) may be multiple or have different configuration/parameters in <multi-domains. >> The first part of the sentence is not clear whether the "mechanisms may be multiple" in a single or multi-domain. I suggest use <'different' instead of 'multiple'. Did authors mean one domain will implement one type of mechanism? Or do you consider the situation where each <hop may process packet for bounded/definite latency differently? 
Quan->It may support more than one type of queuing mechanisms in a DetNet domain based on the different bounded latency requirements. But for now, one domain will select one type of mechanism for a flow and the queuing mechanism and the packet processing of the nodes along the path will be the same. 

<Section 3.1.1.: ‘This can be done, for example, by putting extra buffer space at the ingress of a new domain, increasing the dead time as a guard <band, or using some timing compensation mechanism.’ >> Can you provide references to dead time and timing compensation details for offline <reading? I am not familiar with technical details here but identifying 'how much' 2 domains are out of sync is also a requirement. Similarly, in <section 3.1.1 should we not say "identify (or estimate), “recover and absorb such time variance..." 
Quan-> The TSN connections scenario is a traditional use case for DetNet. I am not sure that it is a necessary requirement especially when DetNet domain provides mechanisms not requiring synchronization.The references of dead time and timing compensation details may refer to IEEE TSN such as ECQF[1]. 
[1]https://1.ieee802.org/tsn/802-1qdv/ 
[km] I believe that the references are necessary. When I read " increasing the dead time as a guard “ it was not obvious what the authors referring to. Either you would remove the example or provide the background context to it. Is there an expectation to be familiar with the TSN to understand the document?
[Quan]Yes, your comment will be revised in next version.


<Section 3.1.3 >>wondering what is a better term - Full time synchronization or end-to-end time synchronization? I am also concerned that partial <synchronization could mean something else – like the traffic was only synchronized for a certain part. What if we used alternate terms like inter-<domain time synchronization and intra-domain time synchronization to mean full and partial? 
Quan-> I think section 3.1.3 mainly proposed an example such as frequency synchronization not focus on the inter-domain, intra-domain or end-to-end time synchronization. I am not sure about the definition of full time synchronization and partial synchronization. If there is no consensus, maybe“Provide Mechanisms not Requiring Strict Time Synchronization" is better? 
[km] Yes, that works. What’s confusing in this paragraph is use of full synchronization, strict synchronization and partial synchronization. These terms can be confusing based on the context. I will also suggest to replace all occurrences of 'full synchronization' with 'strict synchronization'.
[Quan]Yes, this may be revised in next version.


<Section 3.2 ‘It is much greater than that of a LAN, and introduces impacts on queuing mechanisms, such as cyclic or time aware scheduling <method. So the queuing mechanism for LAN networks needs to be extended,’ >> I think we can rephrase the second path of the sentence. The <requirement is to consider propagation delays in end-to-end computations. Why only restrict to LAN queueing. "queuing for LAN needs to be <extended" does not appeal as correct to me.  
Quan->Good point! The mechanisms can not be restricted to TSN mechanisms extensions. 


<Section 3.3, It is practical to have a few flow- based or aggregated-flow based status in the local network.But in higher speed and larger scale <networks, it is hardly feasible. If [IEEE802.1Qcr] is in use, it requires more buffers comparing to the other full/partial time synchronized mechanisms. <Therefore, it requires optimizations to support higher link speeds. >> The text started to discuss flow-state in small-scale vs large-scale networks. <But then there is a discussion about buffers which is related to packet-queueing. I think these are 2 separate things. Do you want to say explicitly <that managing status for large number of flows will need more resources to maintain the state on per flow basis.  
Quan->When supporting higher link speeds, more data can be sent if the network transmission time remains the same. Then more flows and aggregated flows may be transmitted. And the ATS [IEEE802.1Qcr] is a per-flow queuing mechanism which means more states should be maintained and more buffers should be reserved. 

<Section 3.4, maybe related to 3.6 Curious question: DetNet are described to have always sufficient resources so that they will be congestion-free. <Does high-utilization mean that there is a possibility of congestion? 
Quan->In my understanding, one of the three techniques of DetNet architecture [RFC8655] is to allocate sufficient resources to avoid the congestion but that can not guarantee the bounded latency. For a node at a particular time, the large number of flows will be transmitted at the same time and they may be queued and it will significantly fill up the capacity of the link. The instantaneous bandwidth utilization of this link may be high but can not over 100%. The time-based queuing mechanisms can schedule the flows and tolerate high utilization. 

<Section 3.6 ,It is required to support bursty traffic and some methods to decrease the micro-burst. So the pre-planned , ingress traffic <conditioning, scalable queuing, and enhanced capacity of buffer are required to accommodate the flow fluctuation, >> It is slightly difficult to infer <whether traffic above means mixed traffic or is it always deterministic traffic? Is it expected to support both deterministic and best-effort traffic <over the same network. I'd suggest a discussion to clarify this aspect should be done (one or 2 sentences in introduction).  
Quan->I agree to make clarification for the bursty traffic. It includes the deterministic traffic and non-deterministic traffic and the ingress traffic control policy may be different. For non-deterministic traffic, the ingress may just drop it based on the policy. 
[km] ack


<Section 3.8.1 Support Configuration of Multiple Mechanisms >> have authors considered provisioning as a better term? Since the scale of <distributing queueing mechanisms and corresponding metadata can be a daunting task in large-scale networks, more dynamic or in-band means <may be needed. Moreover, it will be good to discuss that having a common set of set of parameters for different mechanisms will help with <interoperability. Section 3.8.2 s/Both of the two cases may may generate/Both of the cases may generate· 
Quan-> Thanks for your advice. I agree with you. “Support provisioning of Multiple Mechanisms” is better. And the common data field for different mechanisms has been discussed in draft [1] and [2]. 
[1]https://datatracker.ietf.org/doc/draft-xiong-detnet-data-fields-edp/ 
[2]https://datatracker.ietf.org/doc/draft-geng-detnet-dataplane-enchancment-encap/ 

<Section 4. Data plane enhancements >>If the data plane is enhanced, should we say something about the existing DetNet data plane and <backward compatibility with it? 
Quan->Yes, I agree with you. The DetNet data plane should be enhanced and the new functions and related metadata should be supported as per [1]. Backward compatibility should be taken into account. 
[1]https://datatracker.ietf.org/doc/draft-xiong-detnet-large-scale-enhancements/ 
[km] I have not gone through this document. Is this a solution or requirements? If it is not a solution, it should be merged here otherwise we should  add requirements text.
[Quan] I think it is a framework for enhanced DetNet data plane based on RFC8938. And I presented it in IETF meetings before. Maybe it can be merged if we get concensus after discussion.  

<Lastly, I found control plane considerations section missing. Since large scale networks are formed from stitching of <multiple domains, at least <one activity standards worthy is how the domains will exchange information together? This is somewhat covered in Section 3.8.2. But my overall <concern with this requirements document is that it has very few constraints. There are no limits on time-sync requirements, mechanisms, <topologies, link speeds, buffers etc. Then how do we verify/claim that the DetNet flows will meet the requested QoS for all DetNet flows in that <large-scale network. Maybe we also need a requirement for control planes to signal feasibility, violations, and monitoring performance of the <deterministic flows. 
Quan->This draft mainly focus on the scaling-requirements in enhanced DetNet data plane. The control plane considerations has been discussed several times in IETF meetings before. Chairs suggested to discuss and update that in existing control plane draft[1]. And the control plane considerations and extensions has been proposed in [2][3]. Discussions and comments are very welcome. 
[1] https://datatracker.ietf.org/doc/draft-ietf-detnet-controller-plane-framework/ 
[2] https://datatracker.ietf.org/doc/draft-xiong-detnet-teas-te-extensions/ 
[3] https://www.ietf.org/archive/id/draft-xiong-detnet-large-scale-enhancements-02.html#name-controller-plane-management 
[km] I suggest to add a reference to these documents for the sake of completeness.
[km] Authors should also add a requirement that states verification mechanisms for successful delivery of flows or signaling for failures. And look forward to revision with comments addressed.
[Quan] Thanks. I will discuss with other authors to see if we should add requirements in control plane and manegement plane.

Cheers,
Kiran


<Thanks Kiran