Re: [Detnet] [DetNet] TQF vs CSQF

[Toerless Eckert <[e-mail]>]

Thanks Peng, inline.

On Tue, Aug 15, 2023 at 06:07:55PM +0800, [e-mail] wrote:
> Hi WG, 
> 
>  
> 
> Due to concerns within the working group regarding the differences between TQF
> 
> (https://www.ietf.org/archive/id/draft-peng-detnet-packet-timeslot-mechanism-03.txt) and 
> 
> CSQF(https://www.ietf.org/archive/id/draft-chen-detnet-sr-based-bounded-latency-03.txt),  
> 
> 
> clarification was attempted in this thread. It should be noted that there may still be some 
> 
> 
> reference papers that have not been draftting in the current CSQF document. For these 
> 
> potential contents, we can further compare them based on the updated version of CSQF 
> 
> later.  This comparison is temporarily based on the latest version of CSQF (i.e., 03-version). 
> 
> The purpose of our clarification is to contribute valid solutions to meet large-scaling 
> 
> deterministic requirements.
> 
> 
> 
> 
> Overall, if understand correctly, CSQF proposes a method of carrying a cycle-id stack in 
> 
> the data packets, which is not fundamentally different from TCQF 
> 
> (https://www.ietf.org/archive/id/draft-eckert-detnet-tcqf-04.txt) that carry a single cycle-id 
> 
> in the data pakckets and maintain cycle-id mapping relationships on nodes. For both of them, 
> 
> the forwarding plane is based on the 3-buffer (or 3-bin) round robin queue mechanism 
> 
> described by TSN ECQF (https://1.ieee802.org/tsn/802-1qdv/) or 
> 
> https://www.ietf.org/archive/id/draft-dang-queuing-with-multiple-cyclic-buffers-00.txt, 

The number of cycle buffers in TCQF and CSQF is not fixed to 3. I guess there may be cases
where 2 works (i have not tried to spend time looking or describing those as i think they're
a bit of a corner case), and there are of course important cases for the number of buffers
to be larger than 3. We do have some description of example if i remember correctly in TCQF.

Note that in TCQF, the number of cycles needs to be the same along a path, whereas in CSQF
it can be different across different hops. I don't think to remember if we have given a simple
example for that additional flexibility of CSQF though, but for example instead of 3 cycles
you could use 6 cycles on maybe only some hops, each with half the cycle time of the "normal" 3 cycles,
and then by assigning the right cycle sequence have 50% of traffic with one cycle less
latency across such a hop.

> that is, CSQF does not describe a new queueing mechanism, but rather a method of carrying 
> cycle-id(s).

Thats all a matter where you are looking from.

Do you think that SRv6 is not a new routing mechanismm compared to IPv6 routing, but rather
a method of carrying the address(es) by which the packet is routed ?

That may be one way to look at it, but i wonder who is being helped by this point of view
given the difference in functionality and mechanisms you need to use with/without SRv6. Aka:
given the additional benefits of SRv6.

Of course, implementation wise, TCQF and CSQF do share all the core functionalities to implement 
(cycle buffers), and i am quite open to merge TCQF with CSQF into one draft if thats what the
WG would prefer to move the work forward.

> 1) Therefore, we believe it is necessary to first differentiate between TQF and CSQF in the 
> 
> design of the underlying queueing mechanism.
> 
> TQF attempts to construct specific orchestration period instance in IP/MPLS networks,
> 
> as well as timeslot resources within that orchestration period instance, and a timeslot
> 
> based queueing mechanism. Where, the type of queue based on timeslots can be either
> 
> round robin or PIFO. No matter which option, queue resources are specific representations
> 
> of timeslot resources and are related to orchestration period instance (derived from gating
> 
> cycle of TSN TAS). Taking Round Robin as an example, each timeslot in the orchestration
> 
> period corresponds to a queue (or buffer), which is its simple design concept (however,
> 
> we also considered optimization in this case, see the role of scheduling period).

I think i expressed previously on the mailing list, and i definitely did so during my
IETF117 presentations:

I think we should not conflate the functionality of per-hop controlled latency forwarding such
as proposed via TCQF, CSQF or gLBF with the longer-period orchestration of interleaving of
flows.  TQF does conflate these two functions, and i think this would have us (DetNet)
end up with a single, unnecessarily monolythic and functionality limited solution.

For example, if i simple have a large set of low-bitrate flows from an ingres node PE1 to
an egres node PE2, i can interlave those flows with timed gated on PE1 completely independent
of the per-hop forwarding mechanism. I could use RFC2212, TSN-CQF, TSN-ATS, TCQF, CSQF or gLBF
as controlled latency hop-by-hop forwarding - with the same PE1 ingress timed gates to do the interleaving.

Likewise, if i want to increase utilization with DetNet traffic for traffic from/two different
ingress/egress PE (flows which happen to just be on se same output interface on some P node),
then i can equally use such timed gates on the ingress PE and combine with TSN-CQF, TCQF or CSQF.

This modularity is why i wrote and presented draft-eckert-detnet-flow-interleaving,
It exactly proposes that we should specificy such timed gates for use with DetNet.
This should be as much as possible based on TSNs functionality, but as much as desired so that
DetNet implementers do not have to understand/extract/profile/implement any TSN
specifics of such gates that are not needed/desired for DetNet.

Note that draft-eckert-detnet-flow-interleaving is not the specification i think we should
have for gates, or "long-term orchestration" of flows as you call it, so if we agree on the
principle, there is certainly a good amount of spec work still to be done. But please lets write
it independent of the per-hop mechanisms.

Of course, given how i hope we do want to support per-hop mechanisms with explicit tagging
of packets, our gates also need to be able to generate not only the scheduling time of packets
of flows (such as the TSN gates), but also the tags required by the per-hop mechanisms we
want to support (which of course is optional, in the above PE1/PE2 flow aggregation case together
with eg.: rfc2212 or tsn-ats you wouldn't need tags).


> CSQF also adopts the round robin queue, adding the possible additional tolerance queue
> 
> based on CQF's 2-buffer. The design concept of the tolerance queue is to absorb the
> 
> forwarding delay jitter within the node. Therefore, CSQF generally uses 3 queues as
> 
> sufficient. Refer to the following description of CSQF section 3.3.  CSQF Timing Model:
> 
> "As described in Section 2.2, both the RQ and TQ can be used for absorbing processing
> 
> jitter, and the upper bound of the absorbing capacity is 2T.  So, if the processing jitter
> 
> is less than 2T, the three-queue model can work.  Otherwise, more buffer is needed to
> 
> absorb the jitter, through increasing the duration of the cycle or by adding more queues. 
> 
> Increasing the duration of the cycles is equivalent to increasing the depth of the queues
> 
> (adding more buffer for each queue)."

I have not tried to edit-read all of CSQF, especially not the pieces which are the same with
TCQF, but CQSF, like TCQF would rougly want to use 3+J buffers when

   J*CT <= (O(MTIE) + O(LDV)) < (J+1)*CT.

Aka: if the sum of clock jitter between nodes (MTIE) plus the Link Delay Variation (LDV)
is larger than 1 cycle time, then you want J=1+3 = 4 cycles. If it is larger than 2 cycles,
you want 2+3 = 5 cycles - and so on.

> From the above, it can be seen that only observing the round robin used by TQF and the round 
> 
> robin used by CSQF seems similar, but there is a significant difference in the definition of queues 
> 
> ("timeslot queue" vs "SQ/RQ/TQ") and the selection rule for the number of queues ("orchestration 
> 
> period related" vs "intra-node forwarding delay jitter related"). Note that the basic round robin 
> 
> queue (and corresponding TDMA ideas) is not dedicated or exclusive to CSQF or TQF.

The cycle assignments in CSQF can of course serve different purposes, and theoretically
i could do the flow interleaving across (in my draft-eckert-detnet-flow-interleaving example)
1000 cycles all explicitly with CSQF cycles. But obviously, i would not want to put any
unnecessary complexity and scale challenge into my hop-by-hop forwarding, when i can
equally well do it with just timed gates on the ingress PE. Hence to me the rule is that
the numbrer of cycles we should require routers to support hop-by-hop in CSQF (and TCQF) is
solely based on uses that can not be done equally well with ingress timed gates. To me this
is primarily  jitter (MTIE, LDV) and the latency priorization i mentioned before.

technically speaking, i think we would want to have <= 15 cycles in CSQF so that we could come
up with an encoding that does not use more than 4 bits per-hop to carry the cycle-ID. Of course,
SRH would give us a lot more bits, but SRH is quite inefficient to encode a strict path with
per-hop SIDs, so i very much assume that we would want to deploy CSQF together with some
form of compressed source routing header such as draft-ietf-spring-srv6-srh-compression, and
try to minimize the number of per-hop SIDs we need to support CSQF.

> Correspondingly, the semantics of cycle-id and timeslot-id are not the same. In particular, the 
> 
> timeslot-id of TQF does not require any relationship between the timeslot lengths of all nodes due
> 
>  to its simple identification of timeslots within the orchestration period instance. Our limitation is 
> 
> just that all nodes inter-operate based on the same orchestration period instance. While the range 
> 
> of values for cycle-id may be the number of SQ/RQ/TQ queues (or buffers, or bins) implemented 
> 
> on the forwarding plane.

Yes. timeslot-id would rather be a term to use i think in the context of draft-eckert-detnet-flow-interleaving
type orchestration and the timed gates it requires.

> In addition, with the help of PIFO, TQF attempts to discuss the forwarding behavior of in-time, 
> 
> which is also different from CSQF.
> 
> 
> 
> 
> 2) Then let's compare the packet encapsulation on the data plane.
> 
> The information carried in the data packets is related to the parameters used by the queueing 
> 
> mechanism. 
> 
> A CSQF data packets may carry a cycle-id stack, which is used to indicate the ID of RQ/TQ
> 
> accessed by each node along the path.
> 
> While a TQF data packet may carry the instance number of the orchestration period and
> 
> the timeslot id (e.g, a single global timeslot id, or local timeslot id stack, or a single global
> 
> timeslot offset, or local timeslot offset stack) within that orchestration period, used to
> 
> indicate that each node along the path accesses the timeslot queue resources of that
> 
> orchestration period instance.

Would be good to come up with a simple to understsnd example of the added user value of carring this additional
metadata in packets as compared to TCQF/CSQF + ingres-edge-gates. So far, i do not see it.

> 
> 3) Then the comparison of the resource reservation schemes on the control plane
> 
> TQF attempts to model the timeslot resources of a specific orchestration period instance
> 
> on a port that enables it. Each orchestration period instance is allocated an initial bandwidth,
> 
> containing each timeslot with an initial maximum reservable burst amount and real-time
> 
> free amount. The selection of orchestration period length is related to the length of the
> 
> service burst interval. It also provide the signaling method and a budget based timeslot
> 
> resource reservation algorithm.

I think this is covered by hopefully well understandable examples in draft-eckert-detnet-flow-interleaving.

> 
> CSQF mentioned that resources is interface occupation time, and the allocation unit is cycle.
> 
> However, the specific resource reservation algorithm is the local behavior of the controller.
> 
> There is no contents about modeling and signaling of timeslot resource on the device. 

Thats why i wrote draft-eckert-detnet-flow-interleaving.

> Hope the above clarification be helpful. 

Thanks
> 
> If you have any comments/suggestions, please send to us.

+1 for my response ;-)

> Regards,
> 
> PSF (on behalf of co--authors of TQF)

Cheers
    Toerless (on behalf of the co-authors of TCQF/CSQF ;-)