Re: [mpls] [Detnet] DetNet: draft-eckert-detnet-mpls-tc-tcqf-03 request for feedback

[Toerless Eckert <[e-mail]>]

On Wed, Aug 03, 2022 at 01:27:33PM +0000, Black, David wrote:
> Hi Toerless,
> 
> I'll observe that the rather small MPLS TC field is being used to serve a lot of purposes.  A question to consider (and one that likely reflects my limited MPLS expertise) is whether cycle tag information is always needed in the forwarding layer F-label vs. an alternative where the cycle tag is in the service layer S-label, resulting in an approach where the service layer handles tagged cycles, relying on the unmodified forwarding layer of the MPLS data plane between service nodes.  That could result in a hierarchy where service nodes are deployed to bound the level of cycle skew exposed to the unmodified forwarding layer that does not see the cycle tags.  Does that make sense?

The goal definitely is to come up with the most easily first-round deployable
solution, so i am happy to put the bounded latency function into whatever layer
(Service or Forwarding) where it is most easily implemented. But: a) not sure
what the other detnet folks think about that, or b) we probably would need
to ask the MPLS WG what they think aout looking up QoS from the second label
in the stack - complexity wise.

> On ECN ...
> 
> > Wrt. ECN: For TCQF, we do not need ECN: There CANNOT be any TCQF/DetNet traffic
> > with bounded latency that would need ECN markings. That would simply be a failure
> > of the bounded latency service.
> 
> There may be a "never say never" reality check lurking in there ;-).
> 
> While use of ECN to signal actual congestion is almost certainly indicative of a DetNet failure, I'll repeat my comments to Pascal on this topic:
> 
> > > I think we have two related goals in this discussion that ought to be distinguished:
> > > 	- What's the minimal/necessary/ideal L4 transport functionality for DetNet?
> > >	- What would it take to run an existing transport stack (L4 and above) over DetNet?
> > > This discussion started around the first goal, where I agree with you that the ECN-based dynamic windowing
> > > functionality is not needed.  That functionality may be useful towards the second goal (e.g., run HTTP/TLS/QUIC
> > > over a DetNet data plane for device management).
> 
> I suggest more consideration of that second goal.
> 
> Are you taking the position that it is *never* appropriate to run an existing web-based management stack (e.g., based on HTTP/TLS/TCP or HTTP/TLS/QUIC) over a DetNet MPLS data plane?

I think the initial customers for DetNet are those who have the no-loss, no-jitter, no-congestion
requirements. Hence i want the first round that we can do without new packet headers to be
focussing on that, so we (DetNet) gets a good foot in the market. And especially that we
can establish simpler DetNet-Only solutions, instead of creating an unfortunately even
more complex solution we have now: layering DetNet on top of TSN to get bounded latency.

So, i primarily would not want to spare bit for more loftiger goals in this first-round
encoding options (reuse TC, DSCP), unless we know we have them available without  cutting
into that first goal.

Once we know we have the packet header bits to do more:

First of all, i would like us to think about OAM bits to recognize when there are errors,
e.g.: when there is packet drop even though there should be non, because something
malfunctioned. How would we do this ? Aka: Whats the best OAM for failure of
bounded latency that we can give those first-round applications.

If that OAM turns out to also be usable for classical congestion controlled applications
(which i don't think), then we're done. Otherwise we'd have to look for encoding
of bits for both those OAM goals and the ECN goals.

If we go with SR-MPLS, our LSE would turn into SIDs, so we could have SIDs for just
DetNet, not shared by other traffic. Meaning wew have the full 3 bit free. And we
could then also easier (IMHO) hack around even more, redesignating e.g.: some more
bits from TTL (if Cisco gives license to IETF on its patent for that...).

Aka: Lots of options, which is why i was trying to figure out whats hopefully most important
first so we can get more deployment experience from a first round RFC.

Cheers
    Toerless

> Thanks, --David
> 
> -----Original Message-----
> From: detnet <[e-mail]> On Behalf Of Toerless Eckert
> Sent: Tuesday, August 2, 2022 10:41 PM
> To: [e-mail]
> Cc: DetNet WG; [e-mail]; draft-eckert-detnet-mpls-tc-tcqf
> Subject: Re: [Detnet] DetNet: draft-eckert-detnet-mpls-tc-tcqf-03 request for feedback
> 
> 
> [EXTERNAL EMAIL] 
> 
> Thanks for the interest Zongpeng,
> 
> Given how the TC-TCQF solution is now within scope of DetNet charter, i think this
> is also a good time to also get more input/review from the MPLS WG, hence Cc'ing it
> (the chair where already suggesting to engage with th MPLS WG on this in the past).
> 
> For background: The draft currently covers what i think is the most easy deployable option to
> enable tagged-cyclic-queuing-and-forwarding (TCQF) with the MPLS forwarding plane -
> without changes on the wire, by reusing the existing MPLS header field TC.
> 
> If we choose to adopt this work in DetNet it would be good work to document not
> only this most simple TC encoding, but also any other encoding that do not require
> on-the-wire-changes, but that may be necessary when this most simple method is
> considered to be not sufficient for some MPLS networks.
> 
> TCQF encoding that would encode the cycle in a new heder field (change on the wire)
> should IMHO be work reserved for another draft. Primarily because i think we can
> much faster finish specifying and deploying the mechanism with the options that do
> not require changes on the wire - and collect experience with it (this is true for
> MPLS as well as IP/IPv6, where we can use DSCP).
> 
> The draft therefore proposed allocation of MPLS TC values for use with TC-TCQF is
> described in section 3.3 of the draft TC-TCQF draft. We would need 3 or 4 TC values.
> 
> Here is a bit more detail:
> 
> RFC3270 defines how to use the TC field
> (actually, it calls it EXP, the field was later renamed to TC in RFC5462)
> 
> What our TCQF draft proposes is to propose to use the so-called 
> "TC- Inferred-PSC LSP (E-LSP) behavior". This is described in section 1.2 of
> RFC3270.
> 
> This is IMHO the most widely deployed method for using DiffServ with MPLS
> and also the most simple to deploy, because it can get away without additional
> control plane / labels. What this effectively means is that the network operator
> configures in a DiffServ style 
> 
> When a deployment has already used more than 4/5 TC and therefore
> does not have 3 or 4 TC free, then (i think!) the operator could allocate additional
> labels space where the TC values are free. Depending on how MPLS experts read
> RFC3270, this might require resorting to calling those LSPs SIDs and call that
> encoding then an SR-MPLS mechanism.
> 
> We could also look into allocating 3 or 4 additional L-LSP, but that looks like
> the most complicated option to me.
> 
> IMHO, a first RFC does not have to be complete with all possible encoding
> options. I'd rather see an RFC finished that can be deployed, and when that
> raises interest in the industry, but the vendors recognize some customers who
> already used up all TC, then this would be good justification to do a -bis of
> the RFC with the more complicated encodings.
> 
> Wrt. ECN: For TCQF, we do not need ECN: There CANNOT be any TCQF/DetNet traffic
> with bounded latency that would need ECN markings. That would simply be a failure
> of the bounded latency service. The non-TCQF TC values may indicate ECN
> for non-TCQF traffic, but to the best of my knowledge ECN with TC is practically
> not deployed.
> 
> Hope this help, please comment!
> 
> Cheers
>     Toerless
> 
> On Tue, Aug 02, 2022 at 09:14:24AM +0800, [e-mail] wrote:
> > Hi, Toerless   
> > 
> >     I am Zongpeng Du from China Mobile.
> > 
> >     We are interested in the draft. However, we have a question about the method to convey the tag in the TOS of an MPLS label.
> > 
> >     MPLS label: 32bits
> >     
> >     00-19     20-21                                                 23                                    24-31
> >     label       TC: Traffic Class(QoS and ECN)        S: Bottom-of-Stack          TTL: Time-to-live
> > 
> > As shown in the Figure, the three bits for TC have been defined as the format of "QoS and ECN". 
> >     IMO, unless the IETF agrees to disuse it, we can not use the bits that have been used.
> >     Do I have any misunderstanding here?
> > 
> >    Or, the label is special defined, such as an S-Label, so that we can define new use of the TC field ? 
> > 
> > Best Regards
> > Zongpeng Du
> > 
> > 
> > 
> > [e-mail] & [e-mail]
> >  
> > From: Toerless Eckert
> > Date: 2022-07-12 03:20
> > To: detnet
> > CC: draft-eckert-detnet-mpls-tc-tcqf
> > Subject: [Detnet] DetNet: draft-eckert-detnet-mpls-tc-tcqf-03 request for feedback
> > Dear DetNet WG
> >  
> > I just posted rev 3 of https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-eckert-detnet-mpls-tc-tcqf/__;!!LpKI!mdHkRkynsjbKwb-lZDUF4l-5wA4KD2nDtRSdgiUzwt_OR1Ia9AOMNJyskNQCYsQzZKh4xiORb915kQ$ [datatracker[.]ietf[.]org]
> >  
> > Diff over previous version 2:
> >   https://urldefense.com/v3/__http://www.ietf.org/*rfcdiff?url1=https:**Awww.ietf.org*archive*id*draft-eckert-detnet-mpls-tc-tcqf-02.txt&url2=https:**Awww.ietf.org*archive*id*draft-eckert-detnet-mpls-tc-tcqf-03.txt__;Ly8vLy8vLy8vLy8!!LpKI!mdHkRkynsjbKwb-lZDUF4l-5wA4KD2nDtRSdgiUzwt_OR1Ia9AOMNJyskNQCYsQzZKh4xiNYE2hEcw$ [ietf[.]org]
> >  
> > This version adds new text about the per-flow ingres enqueueing into the
> > cycles, informational text about high-speed HW implementation considerations
> > and a pointer to a research paper also describing such HW validation.
> >  
> > I think this makes the proposed work complete, and as authors, we would
> > like to ask the WG for adoption - and the participants for feedback.
> >  
> >   Btw: The authors are also happy to discuss any restructuring, for example
> >   Eshan told me it might be better to separaate the genreic mechanism from
> >   the MPLS encoding (separate drafts). 
> >  
> > Quick summary, if you have not kept track:
> >  
> >   TCQF stands for Tagged Cyclic Queuing and Forwarding, and is a proposal
> >   how to adopt the IEEE TSN "Cyclic Queuing and Forwarding" Queuing for
> >   'native' use in DetNet, specifically with MPLS forwarding plane.
> >  
> >   CQF is a great queuing mechanism for DetNet because it does not only offer
> >   bounded latency, but also tight-bounded jitter, important for all
> >   industrial "synchronous" applications. It also is most easily scalable
> >   in HW and operations for metropolitan or larger size WAN DetNet deployments
> >   because it does not have per-hop state on every router.
> >  
> >   By tagging packets with the cycle number, TCQF overcomes also the issues
> >   that keeps CQF limited to small-scale (few Km) Ethernet: it can support
> >   large link delays and link-delay variation, and it reduces relatively
> >   the required clock accuracy, therefore making it feasible to use this
> >   on 100, 400 Gbs or faster networks.
> >  
> >   There is an expired problem description draft i wrote some time ago,
> >   if you need more background: https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-eckert-detnet-bounded-latency-problems/__;!!LpKI!mdHkRkynsjbKwb-lZDUF4l-5wA4KD2nDtRSdgiUzwt_OR1Ia9AOMNJyskNQCYsQzZKh4xiOUluv-oQ$ [datatracker[.]ietf[.]org]
> >  
> > Cheers
> >     Toerless
> >  
> > _______________________________________________
> > detnet mailing list
> > [e-mail]
> > https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/detnet__;!!LpKI!mdHkRkynsjbKwb-lZDUF4l-5wA4KD2nDtRSdgiUzwt_OR1Ia9AOMNJyskNQCYsQzZKh4xiMZSWyvmw$ [ietf[.]org]
> 
> -- 
> ---
> [e-mail]
> 
> _______________________________________________
> detnet mailing list
> [e-mail]
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/detnet__;!!LpKI!mdHkRkynsjbKwb-lZDUF4l-5wA4KD2nDtRSdgiUzwt_OR1Ia9AOMNJyskNQCYsQzZKh4xiMZSWyvmw$ [ietf[.]org]
> 

-- 
---
[e-mail]