Re: [Detnet] questions about draft-eckert-detnet-mpls-tc-tcqf

[Toerless Eckert <[e-mail]>]

Hi Shaofu,

Thanks for asking, and sorry for the answer to being so long, but i did
ponder about those questions quite a bit already.

On Thu, Nov 11, 2021 at 03:00:10PM +0800, [e-mail] wrote:
> Hi authors,
> 
> Thanks for the effort to introduce TSN CQF to IP network, especially, a large scale WAN.

Thanks. Hope the answers  below will be satisfactory to support the draft.

> I have seen examples of single T-CQF flow competed with best-effor backgroup flows, that is OK. However, the competetion between multiple T-CQF flows may be more interesting.

> One may say that there is a central controller to plan and arrange T-CQF flows, and never permit more T-CQF flows match the same local cycle-id at the intermediate node then to avoid congestion. I think that may be possible for a small network, but may be challange for large scale WAN.

For use with DetNet, and as the really as the initial production deployment
goal, this is exactly how it would work. And this is exactly how CQF in TSN
is also used, so there is nothing new in principle.

Wrt to scalability, i think there are well known options from TSN and new
ones from wide-area/high-speed networking:

- you need to admit for every flow a maximum number of bits into a cycle.
  Important business cases like telemtry (traffic management in cities)
  or financial stock ticker (market data telemetry ;-) have small packets.
  This allows more flows.
- Typical TSN networks use 1 Gbps links. Typical Service Provider Networks
  will use 1 Tbps in a few years. That is a factor 1000 number of bits/cycle
  and therefore number of flows.
- The cycle time can be configured larger or smaller. The larger, the more bits
  will fit. The larger the physical propagation latency of paths is, the larger
  you may be willing to make cycle time.
- The admission controller may not admit every flow into every cycle, but
  manage bandwidth in terms of a sequence of cycles. This is also done
  by TSN controllers. This can increase the number of bursty flows that can
  be admitted sinificantly.

> Can you help me to understand how does T-CQF solve the congestion ?  For example, an intermediate node with multiple incoming ports may receive T-CQF packets from many sources, and  these packets are put into the same cycle-id related queue. Can they be guaranteed to be sent together in the next cycle?  Appreciate for additional description of that in the document.

Explanations like those bove do IMHO not belong into a normative forwarding
spec, which draft-eckert-detnet-mpls-tc-tcqf intends to be. Instead
they belong into an informatinal "architecture" document. 

As i think i said during our IETF112 WG meeting: If the working group
adopts draft-eckert-detnet-mpls-tc-tcqf, and is interested to see
all the collateral controller-plane aspects be document (such as
the explanations above), then i would be happy to see that we
adopt such an architecture document. For example, our two pre-existing
draft could be good starting points:

draft-dang-queuing-with-multiple-cyclic-buffers [1]
draft-qiang-detnet-large-scale-detnet [2]

When we wrote [1], we ONLY targeted it for DetNet, and did not ever consider
a case where we would have congestion, because we thought that would not
be in scope of DetNet.

When we where then sent around various WGs,  the colleagues who wrote [2]
actually did consider use within and outside of DetNet, aka: congestion
could be an option. The text does not say much about it though.

After my talk with Balazs yesterday, i conclde that even when we have
congestion, T-CQF would still be a DetNet solution, because it will
still guarantee a bounded latency. So this is i think an exciting
opportunity to add to the TCQF architectural document with text like the
following:

| When you use TCQF without an admission controller, congestion may happen
| and cycles may be overrun with traffic, and packets that will not fit a 
| cycle have to be dropped. The packets that do not get dropped will get
| forwarded with a guaraneed bounded latency though, so this
| type of operation is still a DetNet application. For example,
| such a deployment could exclusively use TCP traffic and achieve
| bounded latency for those TCP flows. Because theree is no further
| distinction of packets, a deployment needs to have all packets
| be either admission controlled or all be congestion controlled.

I consider this congestion controlled deployment with bounded latency 
a great area of research for congestion control researcher, but i
don't think that we have a well understood operational model yet.
So i might actually want to present about that option to e.g.: ICCRG,
we might even get some of those researcher then become interested in
DetNet!

Cheers
    Toerless