Re: [mpls] [Detnet] DetNet: draft-eckert-detnet-mpls-tc-tcqf-03 request for feedback

["Black, David" <[e-mail]>]

Hi Zongpeng,

I agree that detnet traffic shaping at or near traffic source (e.g., client or edge router) is important to ensure that the amount of traffic sent is within the applicable DetNet SLA.

My understanding of TCQF is that it addresses detnet in-network effects (e.g., jitter accumulation) that exceed what CQF is able to cope with, and that those effects (e.g., in-network jitter) are not readily addressed by shaping at or near the source.

More detailed explanations of in-network effects can be found in Yizhou's Philadelphia slides:
https://datatracker.ietf.org/meeting/114/materials/slides-114-detnet-6-ipv6-options-for-cyclic-queuing-and-forwarding-variants-00

Thanks, --David

From: [e-mail] <[e-mail]>
Sent: Wednesday, August 3, 2022 11:28 PM
To: Black, David; Toerless Eckert
Cc: DetNet WG; mpls; draft-eckert-detnet-mpls-tc-tcqf; Black, David
Subject: Re: RE: [Detnet] DetNet: draft-eckert-detnet-mpls-tc-tcqf-03 request for feedback


[EXTERNAL EMAIL]
Hi, David

    Some personal understanding here.


    There may be many detnet tunnels in the network. It will not lose packet, however it requries a limited sending speed.


    The detnet tunnel perhaps will provide services to many individual flows, and each flow should be shaped somewhere before entering into the tunnel.

    If the shaping is done only at the edge router, I think the result is similar to the service of a non-detnet tunnel.
    If the shaping is also done on the client, I think what we need is that the application is DetNet aware, and do not put too much data into the transprort layer. Hence, there would not have too many packets in the L3.



    IMHO, if the traffic is well shaped and the DetNet application can control its sending speed, the current L4 can work well with the underlayer DetNet tunnel.

Best Regards
Zongpeng Du

________________________________
[e-mail]<mailto:[e-mail]> & [e-mail]<mailto:[e-mail]>

From: Black, David<mailto:[e-mail]>
Date: 2022-08-03 21:27
To: Toerless Eckert<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>
CC: DetNet WG<mailto:[e-mail]>; [e-mail]<mailto:[e-mail]>; draft-eckert-detnet-mpls-tc-tcqf<mailto:[e-mail]>; Black, David<mailto:[e-mail]>
Subject: RE: [Detnet] DetNet: draft-eckert-detnet-mpls-tc-tcqf-03 request for feedback
Hi Toerless,

I'll observe that the rather small MPLS TC field is being used to serve a lot of purposes.  A question to consider (and one that likely reflects my limited MPLS expertise) is whether cycle tag information is always needed in the forwarding layer F-label vs. an alternative where the cycle tag is in the service layer S-label, resulting in an approach where the service layer handles tagged cycles, relying on the unmodified forwarding layer of the MPLS data plane between service nodes.  That could result in a hierarchy where service nodes are deployed to bound the level of cycle skew exposed to the unmodified forwarding layer that does not see the cycle tags.  Does that make sense?

On ECN ...

> Wrt. ECN: For TCQF, we do not need ECN: There CANNOT be any TCQF/DetNet traffic
> with bounded latency that would need ECN markings. That would simply be a failure
> of the bounded latency service.

There may be a "never say never" reality check lurking in there ;-).

While use of ECN to signal actual congestion is almost certainly indicative of a DetNet failure, I'll repeat my comments to Pascal on this topic:

> > I think we have two related goals in this discussion that ought to be distinguished:
> > - What's the minimal/necessary/ideal L4 transport functionality for DetNet?
> > - What would it take to run an existing transport stack (L4 and above) over DetNet?
> > This discussion started around the first goal, where I agree with you that the ECN-based dynamic windowing
> > functionality is not needed.  That functionality may be useful towards the second goal (e.g., run HTTP/TLS/QUIC
> > over a DetNet data plane for device management).

I suggest more consideration of that second goal.

Are you taking the position that it is *never* appropriate to run an existing web-based management stack (e.g., based on HTTP/TLS/TCP or HTTP/TLS/QUIC) over a DetNet MPLS data plane?

Thanks, --David

-----Original Message-----
From: detnet <[e-mail]<mailto:[e-mail]>> On Behalf Of Toerless Eckert
Sent: Tuesday, August 2, 2022 10:41 PM
To: [e-mail]<mailto:[e-mail]>
Cc: DetNet WG; [e-mail]<mailto:[e-mail]>; draft-eckert-detnet-mpls-tc-tcqf
Subject: Re: [Detnet] DetNet: draft-eckert-detnet-mpls-tc-tcqf-03 request for feedback


[EXTERNAL EMAIL]

Thanks for the interest Zongpeng,

Given how the TC-TCQF solution is now within scope of DetNet charter, i think this
is also a good time to also get more input/review from the MPLS WG, hence Cc'ing it
(the chair where already suggesting to engage with th MPLS WG on this in the past).

For background: The draft currently covers what i think is the most easy deployable option to
enable tagged-cyclic-queuing-and-forwarding (TCQF) with the MPLS forwarding plane -
without changes on the wire, by reusing the existing MPLS header field TC.

If we choose to adopt this work in DetNet it would be good work to document not
only this most simple TC encoding, but also any other encoding that do not require
on-the-wire-changes, but that may be necessary when this most simple method is
considered to be not sufficient for some MPLS networks.

TCQF encoding that would encode the cycle in a new heder field (change on the wire)
should IMHO be work reserved for another draft. Primarily because i think we can
much faster finish specifying and deploying the mechanism with the options that do
not require changes on the wire - and collect experience with it (this is true for
MPLS as well as IP/IPv6, where we can use DSCP).

The draft therefore proposed allocation of MPLS TC values for use with TC-TCQF is
described in section 3.3 of the draft TC-TCQF draft. We would need 3 or 4 TC values.

Here is a bit more detail:

RFC3270 defines how to use the TC field
(actually, it calls it EXP, the field was later renamed to TC in RFC5462)

What our TCQF draft proposes is to propose to use the so-called
"TC- Inferred-PSC LSP (E-LSP) behavior". This is described in section 1.2 of
RFC3270.

This is IMHO the most widely deployed method for using DiffServ with MPLS
and also the most simple to deploy, because it can get away without additional
control plane / labels. What this effectively means is that the network operator
configures in a DiffServ style

When a deployment has already used more than 4/5 TC and therefore
does not have 3 or 4 TC free, then (i think!) the operator could allocate additional
labels space where the TC values are free. Depending on how MPLS experts read
RFC3270, this might require resorting to calling those LSPs SIDs and call that
encoding then an SR-MPLS mechanism.

We could also look into allocating 3 or 4 additional L-LSP, but that looks like
the most complicated option to me.

IMHO, a first RFC does not have to be complete with all possible encoding
options. I'd rather see an RFC finished that can be deployed, and when that
raises interest in the industry, but the vendors recognize some customers who
already used up all TC, then this would be good justification to do a -bis of
the RFC with the more complicated encodings.

Wrt. ECN: For TCQF, we do not need ECN: There CANNOT be any TCQF/DetNet traffic
with bounded latency that would need ECN markings. That would simply be a failure
of the bounded latency service. The non-TCQF TC values may indicate ECN
for non-TCQF traffic, but to the best of my knowledge ECN with TC is practically
not deployed.

Hope this help, please comment!

Cheers
    Toerless

On Tue, Aug 02, 2022 at 09:14:24AM +0800, [e-mail]<mailto:[e-mail]> wrote:
> Hi, Toerless
>
>     I am Zongpeng Du from China Mobile.
>
>     We are interested in the draft. However, we have a question about the method to convey the tag in the TOS of an MPLS label.
>
>     MPLS label: 32bits
>
>     00-19     20-21                                                 23                                    24-31
>     label       TC: Traffic Class(QoS and ECN)        S: Bottom-of-Stack          TTL: Time-to-live
>
> As shown in the Figure, the three bits for TC have been defined as the format of "QoS and ECN".
>     IMO, unless the IETF agrees to disuse it, we can not use the bits that have been used.
>     Do I have any misunderstanding here?
>
>    Or, the label is special defined, such as an S-Label, so that we can define new use of the TC field ?
>
> Best Regards
> Zongpeng Du
>
>
>
> [e-mail]<mailto:[e-mail]> & [e-mail]<mailto:[e-mail]>
>
> From: Toerless Eckert
> Date: 2022-07-12 03:20
> To: detnet
> CC: draft-eckert-detnet-mpls-tc-tcqf
> Subject: [Detnet] DetNet: draft-eckert-detnet-mpls-tc-tcqf-03 request for feedback
> Dear DetNet WG
>
> I just posted rev 3 of https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-eckert-detnet-mpls-tc-tcqf/__;!!LpKI!mdHkRkynsjbKwb-lZDUF4l-5wA4KD2nDtRSdgiUzwt_OR1Ia9AOMNJyskNQCYsQzZKh4xiORb915kQ$<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-eckert-detnet-mpls-tc-tcqf/__;!!LpKI!mdHkRkynsjbKwb-lZDUF4l-5wA4KD2nDtRSdgiUzwt_OR1Ia9AOMNJyskNQCYsQzZKh4xiORb915kQ$> [datatracker[.]ietf[.]org]
>
> Diff over previous version 2:
>   https://urldefense.com/v3/__http://www.ietf.org/*rfcdiff?url1=https:**Awww.ietf.org*archive*id*draft-eckert-detnet-mpls-tc-tcqf-02.txt&url2=https:**Awww.ietf.org*archive*id*draft-eckert-detnet-mpls-tc-tcqf-03.txt__;Ly8vLy8vLy8vLy8!!LpKI!mdHkRkynsjbKwb-lZDUF4l-5wA4KD2nDtRSdgiUzwt_OR1Ia9AOMNJyskNQCYsQzZKh4xiNYE2hEcw$<https://urldefense.com/v3/__http:/www.ietf.org/*rfcdiff?url1=https:**Awww.ietf.org*archive*id*draft-eckert-detnet-mpls-tc-tcqf-02.txt&url2=https:**Awww.ietf.org*archive*id*draft-eckert-detnet-mpls-tc-tcqf-03.txt__;Ly8vLy8vLy8vLy8!!LpKI!mdHkRkynsjbKwb-lZDUF4l-5wA4KD2nDtRSdgiUzwt_OR1Ia9AOMNJyskNQCYsQzZKh4xiNYE2hEcw$> [ietf[.]org]
>
> This version adds new text about the per-flow ingres enqueueing into the
> cycles, informational text about high-speed HW implementation considerations
> and a pointer to a research paper also describing such HW validation.
>
> I think this makes the proposed work complete, and as authors, we would
> like to ask the WG for adoption - and the participants for feedback.
>
>   Btw: The authors are also happy to discuss any restructuring, for example
>   Eshan told me it might be better to separaate the genreic mechanism from
>   the MPLS encoding (separate drafts).
>
> Quick summary, if you have not kept track:
>
>   TCQF stands for Tagged Cyclic Queuing and Forwarding, and is a proposal
>   how to adopt the IEEE TSN "Cyclic Queuing and Forwarding" Queuing for
>   'native' use in DetNet, specifically with MPLS forwarding plane.
>
>   CQF is a great queuing mechanism for DetNet because it does not only offer
>   bounded latency, but also tight-bounded jitter, important for all
>   industrial "synchronous" applications. It also is most easily scalable
>   in HW and operations for metropolitan or larger size WAN DetNet deployments
>   because it does not have per-hop state on every router.
>
>   By tagging packets with the cycle number, TCQF overcomes also the issues
>   that keeps CQF limited to small-scale (few Km) Ethernet: it can support
>   large link delays and link-delay variation, and it reduces relatively
>   the required clock accuracy, therefore making it feasible to use this
>   on 100, 400 Gbs or faster networks.
>
>   There is an expired problem description draft i wrote some time ago,
>   if you need more background: https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-eckert-detnet-bounded-latency-problems/__;!!LpKI!mdHkRkynsjbKwb-lZDUF4l-5wA4KD2nDtRSdgiUzwt_OR1Ia9AOMNJyskNQCYsQzZKh4xiOUluv-oQ$<https://urldefense.com/v3/__https:/datatracker.ietf.org/doc/draft-eckert-detnet-bounded-latency-problems/__;!!LpKI!mdHkRkynsjbKwb-lZDUF4l-5wA4KD2nDtRSdgiUzwt_OR1Ia9AOMNJyskNQCYsQzZKh4xiOUluv-oQ$> [datatracker[.]ietf[.]org]
>
> Cheers
>     Toerless
>
> _______________________________________________
> detnet mailing list
> [e-mail]<mailto:[e-mail]>
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/detnet__;!!LpKI!mdHkRkynsjbKwb-lZDUF4l-5wA4KD2nDtRSdgiUzwt_OR1Ia9AOMNJyskNQCYsQzZKh4xiMZSWyvmw$<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/detnet__;!!LpKI!mdHkRkynsjbKwb-lZDUF4l-5wA4KD2nDtRSdgiUzwt_OR1Ia9AOMNJyskNQCYsQzZKh4xiMZSWyvmw$> [ietf[.]org]

--
---
[e-mail]<mailto:[e-mail]>

_______________________________________________
detnet mailing list
[e-mail]<mailto:[e-mail]>
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/detnet__;!!LpKI!mdHkRkynsjbKwb-lZDUF4l-5wA4KD2nDtRSdgiUzwt_OR1Ia9AOMNJyskNQCYsQzZKh4xiMZSWyvmw$<https://urldefense.com/v3/__https:/www.ietf.org/mailman/listinfo/detnet__;!!LpKI!mdHkRkynsjbKwb-lZDUF4l-5wA4KD2nDtRSdgiUzwt_OR1Ia9AOMNJyskNQCYsQzZKh4xiMZSWyvmw$> [ietf[.]org]