Re: [Detnet] Role of the transport layer in the DetNet Architecture

Hi Pascal,

Lou's setting a fine example as a WG chair - he nudged me to provide a more productive response to your note, and after further discussion between us, some interesting "where to dig" ideas emerged.

So, starting from your transport layer goals:

> Bottom line, we'd want the application to have to support the minimum and let the transport layer pack the data
> to fit the DetNet SLA and send it at the appropriate time, which could effectively be signaled over the UNI.
> The stream behavior of TCP is appropriate, but the dynamic windowing and the retries and undesirable.
> TCP has reordering but does not do PREOF along the path.

I see four pieces here that ought to be teased apart:
   1) Transmission timing ("pack the data to fit the DetNet SLA")
   2) TCP (and the like) dynamic windowing
   3) Loss and retries
   4) PREOF interaction

Quick summary of what could be done:
1) Transmission timing: Don't ask the transport protocol to do this.
        - Disable sender pacing so the transport protocol (e.g., TCP) sends immediately (subject to its window, see item 2)
        - At the sender, use a DetNet shaper below the transport protocol to align sent traffic to the DetNet SLA.
2) Dynamic Windowing: Use a scalable congestion control with ECN congestion marking algorithms customized to DetNet (more below).
3) Loss and retry behavior. Defer this, and target a lossless DetNet data plane for now to make initial progress on item 2.
        - In initial work, treat any loss as a "cry for help" from the DetNet data plane, e.g., indicating a DetNet SLA violation.
        - Deal with DetNet data planes that exhibit non-congestive losses later (e.g., starting from a transport protocol that doesn't do retries, such as DCCP [RFC 4340] or Datagram QUIC [RFC 9221]).
4) PREOF in the network should "just work".

So, focusing on 2) Dynamic Windowing, there's an opportunity presented by changes in congestion control technology.

Transport protocols (e.g., TCP) that use classic congestion controls (e.g., Reno, CUBIC) tend to fill buffers at bottleneck nodes when there's more traffic to send than the network can accommodate.  That's not desirable behavior for DetNet, particularly as there will be a sender buffer between the transport protocol and the DetNet shaper (item 1) that ought not to be overstuffed.  Starting with Datacenter TCP (DCTCP) a new class of scalable congestion controls has emerged that tends to empty buffers at bottleneck nodes under the same conditions.  These scalable congestion controls use ECN feedback to manage the sender's dynamic window, and do so in a very different fashion than classic drop-equivalent ECN.  The most visible current activities in this area are L4S, TCP Prague, and Prague for QUIC (all discussed briefly in yesterday's ICCRG meeting).  Details of what a "scalable congestion control" means are to be found in Section 4 and Appendix A of draft-ietf-tsvwg-ecn-l4s-id (https://datatracker.ietf.org/doc/draft-ietf-tsvwg-ecn-l4s-id/), but be aware that this is not a light read.

So, an opportunity to make a TCP-like transport protocol useful for DetNet would be to figure out what sort of active queue management (AQM) ought to be used to mark ECN congestion indications for the various possible DetNet queuing mechanisms (e.g., CQF, ATS) where the goal of the ECN congestion indications is to cause the "right thing" (whatever that is) to happen with the sender's transmission window so that the sender's DetNet shaper has "enough" traffic to do its "right thing."  Hacking together a running prototype is strongly suggested as a next step, because the interactions between transport protocols, DetNet's queuing mechanisms, and congestion marking algorithms are subtle, to put it mildly ... and there's a bunch of effort to be invested in figuring out what the two "right thing"s and one "enough" in this paragraph might mean in practice ...

Thanks, --David

-----Original Message-----
From: Balázs Varga A <balazs.a.varga@ericsson.com> 
Sent: Tuesday, July 26, 2022 8:45 AM
To: Black, David; Pascal Thubert (pthubert); DetNet WG; raw@ietf.org
Cc: Black, David
Subject: RE: Role of the transport layer in the DetNet Architecture

[EXTERNAL EMAIL] 

Hi,

Agree with David. Adding just one more aspect: DetNet MPLS never 
touch L4 during the MPLS based forwarding.

Cheers
Bala'zs

-----Original Message-----
From: detnet <detnet-bounces@ietf.org> On Behalf Of Black, David
Sent: Tuesday, July 19, 2022 3:20 PM
To: Pascal Thubert (pthubert) <pthubert=40cisco.com@dmarc.ietf.org>; DetNet WG <detnet@ietf.org>; raw@ietf.org
Cc: Black, David <David.Black@dell.com>
Subject: Re: [Detnet] Role of the transport layer in the DetNet Architecture

Hi Pascal,

> So my question is: should we spin off work in TSVWG?
How long an explanation of the word "no" would you like ... how many years ... :-)?

Seriously, a new L4 transport protocol for DetNet seems rather unlikely to be adopted, and the mention of TCP in the email below looks like an exercise in "strawman demolition," e.g., as RFC 8655 does not mention TCP.

I would suggest looking at RTP and perhaps the multi-stream support in QUIC (uses UDP) or SCTP (via SCTP/UDP)  to understand what sort of adaptations to DetNet would be both plausible and useful.

Thanks, --David

-----Original Message-----
From: detnet <detnet-bounces@ietf.org> On Behalf Of Pascal Thubert (pthubert)
Sent: Tuesday, July 19, 2022 3:18 AM
To: DetNet WG; raw@ietf.org
Subject: [Detnet] Role of the transport layer in the DetNet Architecture

[EXTERNAL EMAIL] 

Dear all, 

At the RAW interim we discussed again the role of UNI and Transport layer, see in the architecture https://urldefense.com/v3/__https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-2b0c3973f1cab171&q=1&e=14c922c0-2015-4d1d-b68f-cb029ff5417a&u=https*3A*2F*2Fwww.rfc-editor.org*2Frfc*2Frfc8655.html*2Afig_DetNetservice__;JSUlJSUl!!LpKI!mLX9vm8aqPY8KibxgO1XK4yIyPC-FgoEzzgUzLqKOw5oFZGMHkk5WEV3h_PArs-rK5G5ZqsQwObAEeQFVKsOz7VR2S8$ [protect2[.]fireeye[.]com].	
RAW typically operates within the DetNet Service subLayer. 

The question is whether the DetNet Service sublayer is fully a L3 operation or if some of it is really L4. E.g., L3 unicast processing in a router is typically one packet in, same packet out. PREOF is certainly beyond that.

Also, a DetNet End System may not have TSN and/or Time Sync handy all the way to the app layer for the app to send the right volume of data at the right time. A transport that is not adapted may delay the transmission and interfere with the application process desires. Bottom line, we'd want the application to have to support the minimum and let the transport layer pack the data to fit the DetNet SLA and send it at the appropriate time, which could effectively be signaled over the UNI. The stream behavior of TCP is appropriate, but the dynamic windowing and the retries and undesirable. TCP has reordering but does not do PREOF along the path. 

So we see the ideal transport is not available off the shelf. We could make it look like UDP to the network, but there's still a need for a particular functionality to suit DetNet. And if we agree to place PREOF at L4, then the packet should emerge to L4 at the relay nodes (see https://urldefense.com/v3/__https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-78e6c37e3d3533b3&q=1&e=14c922c0-2015-4d1d-b68f-cb029ff5417a&u=https*3A*2F*2Fwww.rfc-editor.org*2Frfc*2Frfc8655.html*2Afig_detnet__;JSUlJSUl!!LpKI!mLX9vm8aqPY8KibxgO1XK4yIyPC-FgoEzzgUzLqKOw5oFZGMHkk5WEV3h_PArs-rK5G5ZqsQwObAEeQFVKsO8e_qJV8$ [protect2[.]fireeye[.]com]) and plunge again in L3. Which is fine, that's what L4 proxies do, remember DLSW anyone?

I wrote some basic of this in https://urldefense.com/v3/__https://datatracker.ietf.org/doc/html/draft-thubert-tsvwg-detnet-transport-01__;!!LpKI!mLX9vm8aqPY8KibxgO1XK4yIyPC-FgoEzzgUzLqKOw5oFZGMHkk5WEV3h_PArs-rK5G5ZqsQwObAEeQFVKsOOh98TzQ$ [datatracker[.]ietf[.]org]. The document was intended to introduce the need for next steps as DetNet becomes real, which is now on us. There was great feedback (e.g., Xuesong and Mach) but the focus was elsewhere.

So my question is: should we spin off work in TSVWG? If so, could we prepare a problem statement, e.g., with the doc above as starting point?

I will not be in person at IETF 114, sorry for that. But it would be nice that the topic shows on the agendas for open discussions.

Keep safe;

Pascal 

_______________________________________________
detnet mailing list
detnet@ietf.org
https://urldefense.com/v3/__https://protect2.fireeye.com/v1/url?k=31323334-501d5122-313273af-454445555731-6a55b9dd3f9aedbf&q=1&e=14c922c0-2015-4d1d-b68f-cb029ff5417a&u=https*3A*2F*2Furldefense.com*2Fv3*2F__https*3A*2F*2Fwww.ietf.org*2Fmailman*2Flistinfo*2Fdetnet__*3B*21*21LpKI*21hG_2Ga7PoRVYUsVHjqhyX3pHQWG3nvwEu2xPrtdF95Yv_aq_c7VE43Y747tcgpwb9kjLhxTeRRwP1GJnog_3mYZqXxC9GR2D*24__;JSUlJSUlJSUlJSUlJSUlJQ!!LpKI!mLX9vm8aqPY8KibxgO1XK4yIyPC-FgoEzzgUzLqKOw5oFZGMHkk5WEV3h_PArs-rK5G5ZqsQwObAEeQFVKsOiKC_Z8M$ [protect2[.]fireeye[.]com] [ietf[.]org]

_______________________________________________
detnet mailing list
detnet@ietf.org
https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/detnet__;!!LpKI!mLX9vm8aqPY8KibxgO1XK4yIyPC-FgoEzzgUzLqKOw5oFZGMHkk5WEV3h_PArs-rK5G5ZqsQwObAEeQFVKsO-PzkXeY$ [ietf[.]org]