[Detnet] DetNet and latency

Toerless Eckert <tte@cs.fau.de> Wed, 10 November 2021 19:04 UTC

Return-Path: <eckert@i4.informatik.uni-erlangen.de>
X-Original-To: detnet@ietfa.amsl.com
Delivered-To: detnet@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 025953A103D for <detnet@ietfa.amsl.com>; Wed, 10 Nov 2021 11:04:37 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.649
X-Spam-Level:
X-Spam-Status: No, score=-1.649 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=no autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZLvV1u3RWtE3 for <detnet@ietfa.amsl.com>; Wed, 10 Nov 2021 11:04:32 -0800 (PST)
Received: from faui40.informatik.uni-erlangen.de (faui40.informatik.uni-erlangen.de [131.188.34.40]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E41973A0F4A for <detnet@ietf.org>; Wed, 10 Nov 2021 11:04:31 -0800 (PST)
Received: from faui48e.informatik.uni-erlangen.de (faui48e.informatik.uni-erlangen.de [IPv6:2001:638:a000:4134::ffff:51]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by faui40.informatik.uni-erlangen.de (Postfix) with ESMTPS id E08AF548064; Wed, 10 Nov 2021 20:04:24 +0100 (CET)
Received: by faui48e.informatik.uni-erlangen.de (Postfix, from userid 10463) id CE60C4E9D54; Wed, 10 Nov 2021 20:04:24 +0100 (CET)
Date: Wed, 10 Nov 2021 20:04:24 +0100
From: Toerless Eckert <tte@cs.fau.de>
To: "detnet@ietf.org" <detnet@ietf.org>
Cc: balazs.a.varga@ericsson.com
Message-ID: <YYwXuIiKSPwtbDyb@faui48e.informatik.uni-erlangen.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/detnet/eETLj5w78moNS8x02OXi6Thruxc>
Subject: [Detnet] DetNet and latency
X-BeenThere: detnet@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Discussions on Deterministic Networking BoF and Proposed WG <detnet.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/detnet>, <mailto:detnet-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/detnet/>
List-Post: <mailto:detnet@ietf.org>
List-Help: <mailto:detnet-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/detnet>, <mailto:detnet-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 10 Nov 2021 19:04:37 -0000

Balazs and i had a good chat after the meeting today.

One interesting takeaway:

 DetNet/TSN != bounded latency is mandatory

Balazs stated that he thinks the DetNet architectur should be read such that
ANY DetNet service layer function is optional, but for something to be
called DetNet it needs to use at least ONE service function.o

Example: I have some financial trading with IP Multicast, which builds
its own low, but not guaranteed bounded latency for QoS and really only
wants low packet loss, e.g.: PREF from DetNet. That type of deployment
would be perfectly called a DetNet deployment, even though no
bounded latency solution is used, but only PREF.

I very much like that, because that example is actually why i was originally
interested in DetNet (and then i got sucked into bounded latency ;-).
But i got really differnt opinions about this from different people.

So, i don't think this is really very clear from what people usually
think about DetNet. Because it does not make logically very much sense
with the name of the WG: What is Deterministic ? The only Deterministic
thing we have is the latency calculus. Loss is always stochastic.

The same problem is of course true for Time Sensitive Networking.
So a solution where we only use loss-related
function (PREF) from TSN doesn't sound very TSN'ish.

Cheers
    toerless

On Wed, Nov 03, 2021 at 11:10:29AM -0700, Norman Finn wrote:
> Toerless,
> 
> This situation is known to 802.1.  See https://www.ieee802.org/1/files/public/docs2019/df-finn-CQF-latency-matching-0919-v01.pdf for one possible solution.  When combining flows with reservations, you have to introduce buffers = delays at one or more points along the fast path so that you don't exceed the contract on the stream following the merge.
> 
> -- Norm
> 
> > On Sep 14, 2021, at 15:22, Toerless Eckert <tte@cs.fau.de> wrote:
> > 
> > If i do PEF from e.g. 3 copies of a flow, i can easily create a burst
> > size that is 3 times of the permitted burst size of the flow on the
> > PEF router. This is higher than you would ever get without PEF. None
> > of the calculus in our bounded latency draft covers this.
> > 
> > If you where to simply put PEF in front of an unchanged GuaranteedService
> > (GS) shaper stage or TSN-ATS/UBS regulator+priority-queuing stage, then
> > you would require a lot more buffer and create a lot more latency
> > than what the standard calculus for these mechanisms calculates.
> > 
> > So. how do you think DetNet model covers this burstyness and latency of PEF ?
> > 
> > My gut feeling is that there is no lower-bounded-latency solution
> > than to insert a PEF specific regulator after PEF that re-shapes
> > the flow to its reserved envelope. But i think the actual calculus
> > for PEF latency is TBD.
> > 
> > Cheers
> >    Toerless
> > 
> > On Fri, Sep 10, 2021 at 02:08:31PM +0000, Balázs Varga A wrote:
> >> Hi Toerless,
> >> 
> >> Yes, POF/PEF may cause burst at the DetNet service sub-layer, 
> >> but DetNet forwarding sub-layer provides the functions to cope 
> >> with those bursts. 
> >> 
> >> Adding additional buffering to PEF/POF would be a duplication 
> >> of functions and increase latency.
> >> 
> >> Please note that TSN/DetNet consider always the worst case 
> >> scenario. That means practically, resource allocation based on
> >> PCR. 
> >> 
> >> Cheers
> >> Bala'zs
> >> 
> >> -----Original Message-----
> >> From: Toerless Eckert <tte@cs.fau.de> 
> >> Sent: Wednesday, September 8, 2021 9:36 PM
> >> To: Balázs Varga A <balazs.a.varga@ericsson.com>
> >> Cc: detnet@ietf.org
> >> Subject: Re: [Detnet] draft-varga-detnet-pof (sorry, lot of input)
> >> 
> >> Being a bit delayed by the draft i was promising, i just wanted to bring up in the context of your work one point i stumbled across:
> >> 
> >> Lets say we have:
> >> 
> >>      T-PE1 -> S-PE1 -> LSR-P -> S-PE2 -> T-PE2
> >> 
> >> Lets assume the detnet domain runs from T-PE1 to T-PE2 But now (and i think/hop this is covered by the DetNet architecture), i want to do Packet Duplication Function on S-PE1 and Packet Elimination Function of S-PE2.
> >> 
> >> When i do introduce Packet Ordering Function in conjunction with Packet Elimination on S-PE2, i think that this could create bursts of packets that can exceed the traffic contract known to the owner of the flow (e.g.: CIR/PIR etc..), because this burstyness depends for example on the differential latency of the for example two paths/copies sent between S-PE1 and S-PE2.
> >> 
> >> In effect, i think that one can not simply rely on the bounded latency iand buffering model of any pre-existing boundd latency algorithm as long as that only takes the flow CIR/PIR etc into account. 
> >> 
> >> Aka: In your POF model, where PEF is first performed, those bursts go into the POF, and if we want to hide all this POF/PEF impact from some bounded-latency operation happening after the POF, i think we need to include into the POF some more buffering functionality that reduces burstyness to what the egres bounded latency function can hanle without becoming yet another more complex ingres-edge "gate"
> >> function.
> >> 
> >> Cheers
> >>    Toerless
> >> 
> >> On Wed, Aug 11, 2021 at 11:49:24PM +0200, Toerless Eckert wrote:
> >>> High level answer:
> >>> 
> >>> If we would start to describe the management interface, e.g.:
> >>> what we want be able to configure and observe about all of PREOF 
> >>> (maybe starting with POF), then i think we will immediately get to the 
> >>> point that we expect a much more explicit description of what 
> >>> PREOF/POF has to do so that it can comply with that management 
> >>> inerface.
> >>> 
> >>> I am saying this because when i did design features in the past it did 
> >>> work out best by first starting to think about how the operator should 
> >>> see the feaure. Otherwise a lot was forgotten. In the IETF i did 
> >>> admittedly wiggle myself around doing this 100% correctly, by just 
> >>> informally referring to the CLI/config aspects (such as RFC8994), but 
> >>> that was primarily because for that work the manaement work is really 
> >>> big, so it didn't first. Writing a yang module into a POF draft itself 
> >>> should be rather painless in comparison.
> >>> 
> >>> From there on out, we can start to see how well we could describe the 
> >>> per-hop-behavior, if that is how we are allowed to call it.
> >>> 
> >>> Details hat may be missing are about stuff like: what if a packet 
> >>> arrives whose sequence number does not fall into the window of 
> >>> sequence numbers that the POF currently accepts ? Ok, we discard it. 
> >>> We update a counter tracking these events. Do we update our POF state 
> >>> machinery ? Maybe that needs to be configurable.
> >>> Should the window-size of sequence number be configurable, etc. pp
> >>> 
> >>> Good info about TSN. Thanks
> >>> 
> >>> Toerless
> >>> 
> >>> 
> >>> On Wed, Aug 04, 2021 at 10:15:15AM +0000, Balázs Varga A wrote:
> >>>> Hi Toerless,
> >>>> 
> >>>> Many thanks for the comments. Some clarifications/reactions:
> >>>> 
> >>>> a) "per hop behavior"
> >>>> In general I agree, "unexpected behavior of DetNet products" must be avoided.
> >>>> We have already POF specifics in DetNet RFCs:
> >>>> - RFC8655: introduces the term POF
> >>>> - RFC8938: gives some characteristics of POF (uses sequence numbers, 
> >>>> range of packet order protection, buffer resources)
> >>>> - RFC8964: defines POF packet processing (section 4.2.2.3) What is 
> >>>> missing in your view?
> >>>> 
> >>>> Regarding terminology I also dislike "input vs. outbut timing for 
> >>>> traffic " as POF is not about explicit timing but order of packets.
> >>>> 
> >>>> b) Management interface
> >>>> Yes, management is an important topic. If I understand your comment 
> >>>> correctly You are more concerned about the operation/maintenance 
> >>>> related counters/parameters. I think it is a valid comment, but in 
> >>>> my view it might be better to discuss it in the OAM related drafts.
> >>>> In this draft all the POF algorithm's specific parameters are 
> >>>> defined. E.g., required buffer space can be derived from these parameters and the network/flow specifics.
> >>>> 
> >>>> c) TSN standards about ordering function There were discussions 
> >>>> about ordering in TSN, but nothing standardized.
> >>>> 
> >>>> d) "Another algorithm option: configure for every packet copy path a fixed time delay"
> >>>> I think this topic needs more clarification. Some comments:
> >>>> - I do not see differences in topologies used by TSN and by DetNet. 
> >>>> Rings are also typical in TSN networks.
> >>>> - We have a path specific delay parameter defined in section "4.4. 
> >>>> The Advanced POF Algorithm", namely "POFMaxDelay_i".
> >>>> - I am not sure your proposal is a POF function or a PDF (Packet Delay Function).
> >>>> 
> >>>> Again, thanks for the review and the comments.
> >>>> 
> >>>> Cheers
> >>>> Bala'zs
> >>>> 
> >>>> -----Original Message-----
> >>>> From: detnet <detnet-bounces@ietf.org> On Behalf Of Toerless Eckert
> >>>> Sent: Saturday, July 31, 2021 12:41 AM
> >>>> To: detnet@ietf.org
> >>>> Subject: [Detnet] draft-varga-detnet-pof (sorry, lot of input)
> >>>> 
> >>>> Balazs, *:
> >>>> 
> >>>> I very much support work on POF, however:
> >>>> 
> >>>> I think we should aim higher than informational guidelines
> >>>> 
> >>>> I think we can build a normative document on two interfaces:
> >>>> 
> >>>> a) "per hop behavior"
> >>>> 
> >>>>   We can stanardize the externally observable behavior
> >>>>   between arriving and sent packets on the node.
> >>>> 
> >>>>   This was regularily done for DiffServ Traffic Classes in
> >>>>   TSVWG, where it is calle Per Hop Behavior. When i tried
> >>>>   to apply this term to the input vs. outbut timing for
> >>>>   traffic from what today is an IntServ flow (DetNet),
> >>>>   David Black was somewhat concerned about re-use of the
> >>>>   term, which is why i wrote "per hop behavior". IMHO,
> >>>>   it is perfectly valid to also call this PHB, but i will
> >>>>   leave it up to David to recommend appropriate terminology.
> >>>> 
> >>>>   I am only interested to be able to get as little as possible
> >>>>   unexpecte behavior of DetNet products, and we can only 
> >>>>   arrive that if we standardize the behavior.
> >>>> 
> >>>> b) Management interface
> >>>> 
> >>>>   Yang module specifying what the operator 
> >>>>   can observe about the function about the POF function, e.g: 
> >>>> 
> >>>>   "static" parameters such as how many buffers there are available,
> >>>>     algorithm (you do have algorithm in the draft)
> >>>>    - maybe read/write if configurable.
> >>>> 
> >>>>   "dynamic parameters" - read-only for monitoring, troubleshooting:
> >>>>     - buffers dynamically used,
> >>>>     - latency between first and further copies
> >>>>     - missing packets
> >>>>     - ...
> >>>> 
> >>>> Now wrt to other content:
> >>>> 
> >>>> c) It would be lovely to have a section summarizing what if anythng
> >>>>   TSN did standardize about their ordering function for all those
> >>>>   IETF participants too lazy to read all that documentation
> >>>>   and/or rejecting the notion of having to pay money to read a document.
> >>>> 
> >>>>   (informational, appendix).
> >>>> 
> >>>> d) I think it would be great to have anoher algorithm option
> >>>>   by which one could configure for every packet copy path
> >>>>   (assuming we can identify that in the forwardin plane)
> >>>>   a fixed time delay.
> >>>> 
> >>>>   The reason for this, and why i think it was not an issue in
> >>>>   TSN (with its typical topologies) is that in DetNets
> >>>>   we much more likely will have rings with significant path 
> >>>>   latency differences between A and B path,
> >>>>   and whenever the A (shorter) path looses a packet, we would wait
> >>>>   until the B path packet arrives, resulting in a lot of
> >>>>   jitter. If the DetNet controller could configure 
> >>>>   the B-A pah lateny difference as a delay, we could get rid
> >>>>   of this problem.
> >>>> 
> >>>> Still have to dig deeper into the algo details you proposed for more comments.
> >>>> 
> >>>> Cheers
> >>>>    Toerless
> >>>> 
> >>>> _______________________________________________
> >>>> detnet mailing list
> >>>> detnet@ietf.org
> >>>> https://www.ietf.org/mailman/listinfo/detnet
> >>> 
> >>> --
> >>> ---
> >>> tte@cs.fau.de
> >> 
> >> --
> >> ---
> >> tte@cs.fau.de
> > 
> > -- 
> > ---
> > tte@cs.fau.de
> > 
> > _______________________________________________
> > detnet mailing list
> > detnet@ietf.org
> > https://www.ietf.org/mailman/listinfo/detnet

-- 
---
tte@cs.fau.de