Re: [Detnet] [DetNet] TQF vs CSQF

[[e-mail]]

Hi Toerless,






Thank you for your detailed views and suggestions.


Please see inline #PSF.






Regards,


PSF











Original



From: ToerlessEckert <[e-mail]>
To: 彭少富10053815;
Cc: [e-mail] <[e-mail]>;
Date: 2023年08月21日 01:00
Subject: Re: [Detnet] [DetNet] TQF vs CSQF



Thanks Peng, inline.
 
On Tue, Aug 15, 2023 at 06:07:55PM +0800, [e-mail] wrote:
> Hi WG,  
>  
>   
>  
> Due to concerns within the working group regarding the differences between TQF
>  
> (https://www.ietf.org/archive/id/draft-peng-detnet-packet-timeslot-mechanism-03.txt) and  
>  
> CSQF(https://www.ietf.org/archive/id/draft-chen-detnet-sr-based-bounded-latency-03.txt),   
>  
>  
> clarification was attempted in this thread. It should be noted that there may still be some  
>  
>  
> reference papers that have not been draftting in the current CSQF document. For these  
>  
> potential contents, we can further compare them based on the updated version of CSQF  
>  
> later.  This comparison is temporarily based on the latest version of CSQF (i.e., 03-version).  
>  
> The purpose of our clarification is to contribute valid solutions to meet large-scaling  
>  
> deterministic requirements.
>  
>  
>  
>  
> Overall, if understand correctly, CSQF proposes a method of carrying a cycle-id stack in  
>  
> the data packets, which is not fundamentally different from TCQF  
>  
> (https://www.ietf.org/archive/id/draft-eckert-detnet-tcqf-04.txt) that carry a single cycle-id  
>  
> in the data pakckets and maintain cycle-id mapping relationships on nodes. For both of them,  
>  
> the forwarding plane is based on the 3-buffer (or 3-bin) round robin queue mechanism  
>  
> described by TSN ECQF (https://1.ieee802.org/tsn/802-1qdv/) or  
>  
> https://www.ietf.org/archive/id/draft-dang-queuing-with-multiple-cyclic-buffers-00.txt,  
 
The number of cycle buffers in TCQF and CSQF is not fixed to 3. I guess there may be cases
where 2 works (i have not tried to spend time looking or describing those as i think they're
a bit of a corner case), and there are of course important cases for the number of buffers
to be larger than 3. We do have some description of example if i remember correctly in TCQF.
 




#PSF: Here 3-buffer (or 3-bin) is used as a likely term compared to the 2-queue mode 


of CQF, but not to limit that it can only be 3. How many buffers are required, accordong 


to the design principle (see draft CSQF section 3.3.  CSQF Timing Model) , depends on 


the ratio of intra-node forwarding delay variation to the cycle duration (allocate tolerated 


buffers or increase cycle duration). There may be an argument that why CSQF must obey 


the above design principle, it is easy to extend to use arbitrary number of buffers. This is 


certainly true, and that means any Round robin/TDMA related queueing mechanisms may 


be covered by CSQF. This may be a little crazy. The key point is in fact not related to the 


number of buffers, but to the original design principle. As mentioned below, the design 


principle of TQF is timeslot round robin queue (or PIFO queue) within an instantiated 


orchestration period, but the similar principle is not seen in CSQF.




Note that in TCQF, the number of cycles needs to be the same along a path, whereas in CSQF

it can be different across different hops. I don't think to remember if we have given a simple
example for that additional flexibility of CSQF though, but for example instead of 3 cycles
you could use 6 cycles on maybe only some hops, each with half the cycle time of the "normal" 3 cycles,
and then by assigning the right cycle sequence have 50% of traffic with one cycle less
latency across such a hop.
 


#PSF: If I understand correctly, TCQF is actually similar to CSQF that different node may 

have different intra-node fowarding delay variation, so that different nodes may need 

different cycles. For example, TCQF may maintain the cycle-id mapping relationship: 

x -> x+2 on node A, but x -> x+3 on node B just because node B has a larger forwarding 

delay variation. The example you provided is not seen in draft CSQF, that may be challenge 

for TCQF/CSQF to process the inter-operation of inconsistent cycle time (i.e., from a specific 

cycle time to half the cycle time), e.g, cycle mapping may be not constant, and looking forward 


to more descriptions of CSQF.  Note that TSN ECQF described a count-based bin method to 


process this case, but depends on state machine per stream; TQF naturely support this case 


with the same orchestration period instance (this is also why we repeatedly emphasize that 


the orchestration period is an important design factor of TQF). 







> that is, CSQF does not describe a new queueing mechanism, but rather a method of carrying  
> cycle-id(s).
 
Thats all a matter where you are looking from.
 
Do you think that SRv6 is not a new routing mechanismm compared to IPv6 routing, but rather
a method of carrying the address(es) by which the packet is routed ?
 
That may be one way to look at it, but i wonder who is being helped by this point of view
given the difference in functionality and mechanisms you need to use with/without SRv6. Aka:
given the additional benefits of SRv6.
 


#PSF: My means is to get the common queueing mechanism of all *CQF* variants and then 

compare this common mechanism with TQF's queueing mechanism.




Of course, implementation wise, TCQF and CSQF do share all the core functionalities to implement  
(cycle buffers), and i am quite open to merge TCQF with CSQF into one draft if thats what the
WG would prefer to move the work forward.
 


#PSF: We have the same understanding here, that the core functionalities to implement 

cycle buffers can be seen in draft-dang or TSN ECQF.







> 1) Therefore, we believe it is necessary to first differentiate between TQF and CSQF in the  
>  
> design of the underlying queueing mechanism.
>  
> TQF attempts to construct specific orchestration period instance in IP/MPLS networks,
>  
> as well as timeslot resources within that orchestration period instance, and a timeslot
>  
> based queueing mechanism. Where, the type of queue based on timeslots can be either
>  
> round robin or PIFO. No matter which option, queue resources are specific representations
>  
> of timeslot resources and are related to orchestration period instance (derived from gating
>  
> cycle of TSN TAS). Taking Round Robin as an example, each timeslot in the orchestration
>  
> period corresponds to a queue (or buffer), which is its simple design concept (however,
>  
> we also considered optimization in this case, see the role of scheduling period).
 
I think i expressed previously on the mailing list, and i definitely did so during my
IETF117 presentations:
 
I think we should not conflate the functionality of per-hop controlled latency forwarding such
as proposed via TCQF, CSQF or gLBF with the longer-period orchestration of interleaving of
flows.  TQF does conflate these two functions, and i think this would have us (DetNet)
end up with a single, unnecessarily monolythic and functionality limited solution.
 


#PSF: This statement for TQF may be not true. In fact, the orchestration period of TQF is 

not a new thing, which is derived from the gating cycle with specific length (containing 

multiple timeslots) of TSN-TAS that has already been designed to instantiate on each 

device to help queue scheduling and not  acted as a conflated object. We believe that if 

trying to introduce timeslot resources to IP/MPLS, the first thing is how to define a 

timeslot, including its length, its identification, etc, these parameters are all based on a 

known orchestration period, and these parameters themselves provide a foundation for 

timeslot based path calculation and queue resource establishment. 




For example, if i simple have a large set of low-bitrate flows from an ingres node PE1 to
an egres node PE2, i can interlave those flows with timed gated on PE1 completely independent
of the per-hop forwarding mechanism. I could use RFC2212, TSN-CQF, TSN-ATS, TCQF, CSQF or gLBF
as controlled latency hop-by-hop forwarding - with the same PE1 ingress timed gates to do the interleaving.
 


#PSF: Perhaps we can not decouple flow interleaving with TCQF/CSQF, if here we don't talk about 

overprovision method as described in TSN ECQF. That is, without flow interleaving, TCQF/CSQF 

can not guarantee per-hop latency, and can not act as controlled latency mechanism alone as others.

That means, without flow interleaving, RFC2212, ATS, ATS+damper (your gLBF) can still give 

controlled latency due to regulation,but TCQF/CSQF can not.







Likewise, if i want to increase utilization with DetNet traffic for traffic from/two different
ingress/egress PE (flows which happen to just be on se same output interface on some P node),
then i can equally use such timed gates on the ingress PE and combine with TSN-CQF, TCQF or CSQF.
 


#PSF: Agree.




This modularity is why i wrote and presented draft-eckert-detnet-flow-interleaving,
It exactly proposes that we should specificy such timed gates for use with DetNet.
This should be as much as possible based on TSNs functionality, but as much as desired so that
DetNet implementers do not have to understand/extract/profile/implement any TSN
specifics of such gates that are not needed/desired for DetNet.
 


#PSF: TQF can be taken regarded as a solution of flow interleaving.  The timeslot is 

naturely isolated and interleaved. However, we think this modularity is integrated, and you 

think it is decoupled. : )




Note that draft-eckert-detnet-flow-interleaving is not the specification i think we should
have for gates, or "long-term orchestration" of flows as you call it, so if we agree on the
principle, there is certainly a good amount of spec work still to be done. But please lets write
it independent of the per-hop mechanisms.
 


#PSF: Great, for this point our understanding is basically consistent. In current version 

of TQF, according to integrated modularity as mentioned above, it described timeslot 

resource definition, signaling, path calculation, queueing allocation rule, to provide 

deterministic by a systematic approach. You may find that, for TQF, discussing per-hop 

mechanisms alone (as they are nothing more than round robins, PIFO, etc) does 

not make sense.  However, we may continue to consider this suggestion.







Of course, given how i hope we do want to support per-hop mechanisms with explicit tagging
of packets, our gates also need to be able to generate not only the scheduling time of packets
of flows (such as the TSN gates), but also the tags required by the per-hop mechanisms we
want to support (which of course is optional, in the above PE1/PE2 flow aggregation case together
with eg.: rfc2212 or tsn-ats you wouldn't need tags).
 


#PSF: Agree.


> CSQF also adopts the round robin queue, adding the possible additional tolerance queue
>  
> based on CQF's 2-buffer. The design concept of the tolerance queue is to absorb the
>  
> forwarding delay jitter within the node. Therefore, CSQF generally uses 3 queues as
>  
> sufficient. Refer to the following description of CSQF section 3.3.  CSQF Timing Model:
>  
> "As described in Section 2.2, both the RQ and TQ can be used for absorbing processing
>  
> jitter, and the upper bound of the absorbing capacity is 2T.  So, if the processing jitter
>  
> is less than 2T, the three-queue model can work.  Otherwise, more buffer is needed to
>  
> absorb the jitter, through increasing the duration of the cycle or by adding more queues.  
>  
> Increasing the duration of the cycles is equivalent to increasing the depth of the queues
>  
> (adding more buffer for each queue)." 
 
I have not tried to edit-read all of CSQF, especially not the pieces which are the same with
TCQF, but CQSF, like TCQF would rougly want to use 3+J buffers when
 
   J*CT <= (O(MTIE) + O(LDV)) < (J+1)*CT.
 
Aka: if the sum of clock jitter between nodes (MTIE) plus the Link Delay Variation (LDV)
is larger than 1 cycle time, then you want J=1+3 = 4 cycles. If it is larger than 2 cycles,
you want 2+3 = 5 cycles - and so on.
 


#PSF: There may be different descriptions of the individual components of delay 

variation  to calculate required cycles I found, which will not be detailed here.




> From the above, it can be seen that only observing the round robin used by TQF and the round  
>  
> robin used by CSQF seems similar, but there is a significant difference in the definition of queues  
>  
> ("timeslot queue" vs "SQ/RQ/TQ") and the selection rule for the number of queues ("orchestration  
>  
> period related" vs "intra-node forwarding delay jitter related"). Note that the basic round robin  
>  
> queue (and corresponding TDMA ideas) is not dedicated or exclusive to CSQF or TQF.
 
The cycle assignments in CSQF can of course serve different purposes, and theoretically
i could do the flow interleaving across (in my draft-eckert-detnet-flow-interleaving example)
1000 cycles all explicitly with CSQF cycles. But obviously, i would not want to put any
unnecessary complexity and scale challenge into my hop-by-hop forwarding, when i can
equally well do it with just timed gates on the ingress PE. Hence to me the rule is that
the numbrer of cycles we should require routers to support hop-by-hop in CSQF (and TCQF) is
solely based on uses that can not be done equally well with ingress timed gates. To me this
is primarily  jitter (MTIE, LDV) and the latency priorization i mentioned before.
 


#PSF: So you provide another design principle to calculate the required cycles for CSQF 

that depends on the best-effort result of interleaving only on ingress PE. This may be hard 

to implement, or there may be a circular dependency, because the flow interleaving is related 

with calculated paths of streams, while the calculated paths depend on the instantiated cycles. 

IMO a clear design principle for the required cycles unrelated with flows adding/deleting 

may be better.




technically speaking, i think we would want to have <= 15 cycles in CSQF so that we could come
up with an encoding that does not use more than 4 bits per-hop to carry the cycle-ID. Of course,
SRH would give us a lot more bits, but SRH is quite inefficient to encode a strict path with
per-hop SIDs, so i very much assume that we would want to deploy CSQF together with some
form of compressed source routing header such as draft-ietf-spring-srv6-srh-compression, and
try to minimize the number of per-hop SIDs we need to support CSQF.
 


#PSF: Agree with your considerations about CSQF. Just for comparison, in TQF, it is the 

timeslot id within the orchestration period (but not the scheduling period) carried in packets, 

so 15 may be not sufficient. For example, to support some possible SBI (service burst interval) 

with length 1ms, an orchestration period may contain 100 timeslots.




> Correspondingly, the semantics of cycle-id and timeslot-id are not the same. In particular, the  
>  
> timeslot-id of TQF does not require any relationship between the timeslot lengths of all nodes due
>  
>  to its simple identification of timeslots within the orchestration period instance. Our limitation is  
>  
> just that all nodes inter-operate based on the same orchestration period instance. While the range  
>  
> of values for cycle-id may be the number of SQ/RQ/TQ queues (or buffers, or bins) implemented  
>  
> on the forwarding plane.
 
Yes. timeslot-id would rather be a term to use i think in the context of draft-eckert-detnet-flow-interleaving
type orchestration and the timed gates it requires.
 


#PSF: ACK.




> In addition, with the help of PIFO, TQF attempts to discuss the forwarding behavior of in-time,  
>  
> which is also different from CSQF.
>  
>  
>  
>  
> 2) Then let's compare the packet encapsulation on the data plane.
>  
> The information carried in the data packets is related to the parameters used by the queueing  
>  
> mechanism.  
>  
> A CSQF data packets may carry a cycle-id stack, which is used to indicate the ID of RQ/TQ
>  
> accessed by each node along the path.
>  
> While a TQF data packet may carry the instance number of the orchestration period and
>  
> the timeslot id (e.g, a single global timeslot id, or local timeslot id stack, or a single global
>  
> timeslot offset, or local timeslot offset stack) within that orchestration period, used to
>  
> indicate that each node along the path accesses the timeslot queue resources of that
>  
> orchestration period instance.
 
Would be good to come up with a simple to understsnd example of the added user value of carring this additional
metadata in packets as compared to TCQF/CSQF + ingres-edge-gates. So far, i do not see it.
 


#PSF: In current version of TQF it just mentioned that orchestration period instance and 

timeslot information should be carried in packets. In later versions, we will provide detailed

encapsulation design and related examples.




>  
> 3) Then the comparison of the resource reservation schemes on the control plane
>  
> TQF attempts to model the timeslot resources of a specific orchestration period instance
>  
> on a port that enables it. Each orchestration period instance is allocated an initial bandwidth,
>  
> containing each timeslot with an initial maximum reservable burst amount and real-time
>  
> free amount. The selection of orchestration period length is related to the length of the
>  
> service burst interval. It also provide the signaling method and a budget based timeslot
>  
> resource reservation algorithm.
 
I think this is covered by hopefully well understandable examples in draft-eckert-detnet-flow-interleaving.
 


#PSF: We may firstly get agreement on the above necessary schemes, that is proposed 

as a potential solution for large scaling requirements. Examples are not enough.




>  
> CSQF mentioned that resources is interface occupation time, and the allocation unit is cycle.
>  
> However, the specific resource reservation algorithm is the local behavior of the controller.
>  
> There is no contents about modeling and signaling of timeslot resource on the device.  
 
Thats why i wrote draft-eckert-detnet-flow-interleaving.
 


#PSF: Understand, we are trying to solve the same problem. Open to cooperate on it.




> Hope the above clarification be helpful.  
 
Thanks
>  
> If you have any comments/suggestions, please send to us.
 
+1 for my response ;-)
 
> Regards,
>  
> PSF (on behalf of co--authors of TQF)
 
Cheers
    Toerless (on behalf of the co-authors of TCQF/CSQF ;-)
 
_______________________________________________
detnet mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/detnet