Re: [Detnet] questions about draft-eckert-detnet-mpls-tc-tcqf Fri, 12 November 2021 09:04 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 7D43B3A0637; Fri, 12 Nov 2021 01:04:10 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.863
X-Spam-Status: No, score=-1.863 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, CTE_8BIT_MISMATCH=0.036, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id gFBUWl5hQ03X; Fri, 12 Nov 2021 01:04:06 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id ACC813A05DE; Fri, 12 Nov 2021 01:04:05 -0800 (PST)
Received: from (unknown []) by Forcepoint Email with ESMTPS id DA78962C49C47A411A14; Fri, 12 Nov 2021 17:04:02 +0800 (CST)
Received: from ([]) by with SMTP id 1AC94026047934; Fri, 12 Nov 2021 17:04:01 +0800 (GMT-8) (envelope-from
Received: from mapi (njxapp05[null]) by mapi (Zmail) with MAPI id mid201; Fri, 12 Nov 2021 17:04:00 +0800 (CST)
Date: Fri, 12 Nov 2021 17:04:00 +0800
X-Zmail-TransId: 2afd618e2e00d4c03d94
X-Mailer: Zmail v1.0
Message-ID: <>
In-Reply-To: <>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
X-MAIL: 1AC94026047934
Archived-At: <>
Subject: Re: [Detnet] questions about  draft-eckert-detnet-mpls-tc-tcqf
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions on Deterministic Networking BoF and Proposed WG <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Fri, 12 Nov 2021 09:04:11 -0000

Hi Toerless,

Thanks for clarifications. Please see inline.
(Sorry that I can only send messages in plain text)


日 期 :2021年11月12日 00:24
主 题 :Re: questions about  draft-eckert-detnet-mpls-tc-tcqf
Hi Shaofu,
Thanks for asking, and sorry for the answer to being so long, but i did
ponder about those questions quite a bit already.

[PSF] Thanks, it is helps me.
On Thu, Nov 11, 2021 at 03:00:10PM +0800, wrote:
> Hi authors,
> Thanks for the effort to introduce TSN CQF to IP network, especially, a large scale WAN.
Thanks. Hope the answers  below will be satisfactory to support the draft.

> I have seen examples of single T-CQF flow competed with best-effor backgroup flows, that is OK. However, the competetion between multiple T-CQF flows may be more interesting.
> One may say that there is a central controller to plan and arrange T-CQF flows, and never permit more T-CQF flows match the same local cycle-id at the intermediate node then to avoid congestion. I think that may be possible for a small network, but may be challange for large scale WAN.

For use with DetNet, and as the really as the initial production deployment
goal, this is exactly how it would work. And this is exactly how CQF in TSN
is also used, so there is nothing new in principle.
Wrt to scalability, i think there are well known options from TSN and new
ones from wide-area/high-speed networking:
- you need to admit for every flow a maximum number of bits into a cycle.
Important business cases like telemtry (traffic management in cities)
or financial stock ticker (market data telemetry ;-) have small packets.
This allows more flows.
- Typical TSN networks use 1 Gbps links. Typical Service Provider Networks
will use 1 Tbps in a few years. That is a factor 1000 number of bits/cycle
and therefore number of flows.
- The cycle time can be configured larger or smaller. The larger, the more bits
will fit. The larger the physical propagation latency of paths is, the larger
you may be willing to make cycle time.
- The admission controller may not admit every flow into every cycle, but
manage bandwidth in terms of a sequence of cycles. This is also done
by TSN controllers. This can increase the number of bursty flows that can
be admitted sinificantly.

[PSF] Agree with all the above points. However, there is a assumption here, that the cumulative sum of the bits inserted by multiple source nodes in a single cycle will not exceed the maximum number of bits that can be inserted in a single cycle on the intermediate node. With the increasing number of source nodes connecting deterministic end-systems, or the number of bits that some source nodes need to insert in a single cycle bursts (because the end-system generates burst traffic), this assumption will be frequently violated. At this time, if the admission control limits that some source nodes delay sending traffic, I am not sure whether the delay jitter caused by such delay will affect the operation of the service.

> Can you help me to understand how does T-CQF solve the congestion ?  For example, an intermediate node with multiple incoming ports may receive T-CQF packets from many sources, and  these packets are put into the same cycle-id related queue. Can they be guaranteed to be sent together in the next cycle?  Appreciate for additional description of that in the document.
Explanations like those bove do IMHO not belong into a normative forwarding
spec, which draft-eckert-detnet-mpls-tc-tcqf intends to be. Instead
they belong into an informatinal "architecture" document.
As i think i said during our IETF112 WG meeting: If the working group
adopts draft-eckert-detnet-mpls-tc-tcqf, and is interested to see
all the collateral controller-plane aspects be document (such as
the explanations above), then i would be happy to see that we
adopt such an architecture document. For example, our two pre-existing
draft could be good starting points:
draft-dang-queuing-with-multiple-cyclic-buffers [1]
draft-qiang-detnet-large-scale-detnet [2]
When we wrote [1], we ONLY targeted it for DetNet, and did not ever consider
a case where we would have congestion, because we thought that would not
be in scope of DetNet.
When we where then sent around various WGs,  the colleagues who wrote [2]
actually did consider use within and outside of DetNet, aka: congestion
could be an option. The text does not say much about it though.

[PSF] The order should be a mature architecture firstly, then the related control and data plane documents. Looking forward to more details for architecture. 

After my talk with Balazs yesterday, i conclde that even when we have
congestion, T-CQF would still be a DetNet solution, because it will
still guarantee a bounded latency. So this is i think an exciting
opportunity to add to the TCQF architectural document with text like the
| When you use TCQF without an admission controller, congestion may happen
| and cycles may be overrun with traffic, and packets that will not fit a
| cycle have to be dropped. The packets that do not get dropped will get
| forwarded with a guaraneed bounded latency though, so this
| type of operation is still a DetNet application. For example,
| such a deployment could exclusively use TCP traffic and achieve
| bounded latency for those TCP flows. Because theree is no further
| distinction of packets, a deployment needs to have all packets
| be either admission controlled or all be congestion controlled.

[PSF] Yes, I guess there are no TCQF states per flow maintained at the transit node, thus nobody can ensure the dropped packets are always from the same unluck flow. 

I consider this congestion controlled deployment with bounded latency
a great area of research for congestion control researcher, but i
don't think that we have a well understood operational model yet.
So i might actually want to present about that option to e.g.: ICCRG,
we might even get some of those researcher then become interested in

[PSF] Same opinion. : )