Re: [Detnet] draft-varga-detnet-pof (sorry, lot of input)

[Toerless Eckert <[e-mail]>]

When Balazs and i talked today, he brought up the idea
that we should introduce this component as another
buildin block beside R)eplication, E)limination, O)rdering.
Separate, because in lightweight deployment, we do not
care about latency/jitter - see prior email. But whenever
we do, we would need the block. So if we call it B)uffering,
we end up maybe with PREBOF ? ;-))

I'll check out what IEEE does, thanks for the pointer.
Without looking, i am thinking that a Shaper is the most
simple option - not only to compensate for the peaks
created by E)limination function, but also those created
by the O)rdering function.

Cheers
    toerless

On Wed, Nov 03, 2021 at 11:10:29AM -0700, Norman Finn wrote:
> Toerless,
> 
> This situation is known to 802.1.  See https://www.ieee802.org/1/files/public/docs2019/df-finn-CQF-latency-matching-0919-v01.pdf for one possible solution.  When combining flows with reservations, you have to introduce buffers = delays at one or more points along the fast path so that you don't exceed the contract on the stream following the merge.
> 
> -- Norm
> 
> > On Sep 14, 2021, at 15:22, Toerless Eckert <[e-mail]> wrote:
> > 
> > If i do PEF from e.g. 3 copies of a flow, i can easily create a burst
> > size that is 3 times of the permitted burst size of the flow on the
> > PEF router. This is higher than you would ever get without PEF. None
> > of the calculus in our bounded latency draft covers this.
> > 
> > If you where to simply put PEF in front of an unchanged GuaranteedService
> > (GS) shaper stage or TSN-ATS/UBS regulator+priority-queuing stage, then
> > you would require a lot more buffer and create a lot more latency
> > than what the standard calculus for these mechanisms calculates.
> > 
> > So. how do you think DetNet model covers this burstyness and latency of PEF ?
> > 
> > My gut feeling is that there is no lower-bounded-latency solution
> > than to insert a PEF specific regulator after PEF that re-shapes
> > the flow to its reserved envelope. But i think the actual calculus
> > for PEF latency is TBD.
> > 
> > Cheers
> >    Toerless
> > 
> > On Fri, Sep 10, 2021 at 02:08:31PM +0000, Balázs Varga A wrote:
> >> Hi Toerless,
> >> 
> >> Yes, POF/PEF may cause burst at the DetNet service sub-layer, 
> >> but DetNet forwarding sub-layer provides the functions to cope 
> >> with those bursts. 
> >> 
> >> Adding additional buffering to PEF/POF would be a duplication 
> >> of functions and increase latency.
> >> 
> >> Please note that TSN/DetNet consider always the worst case 
> >> scenario. That means practically, resource allocation based on
> >> PCR. 
> >> 
> >> Cheers
> >> Bala'zs
> >> 
> >> -----Original Message-----
> >> From: Toerless Eckert <[e-mail]> 
> >> Sent: Wednesday, September 8, 2021 9:36 PM
> >> To: Balázs Varga A <[e-mail]>
> >> Cc: [e-mail]
> >> Subject: Re: [Detnet] draft-varga-detnet-pof (sorry, lot of input)
> >> 
> >> Being a bit delayed by the draft i was promising, i just wanted to bring up in the context of your work one point i stumbled across:
> >> 
> >> Lets say we have:
> >> 
> >>      T-PE1 -> S-PE1 -> LSR-P -> S-PE2 -> T-PE2
> >> 
> >> Lets assume the detnet domain runs from T-PE1 to T-PE2 But now (and i think/hop this is covered by the DetNet architecture), i want to do Packet Duplication Function on S-PE1 and Packet Elimination Function of S-PE2.
> >> 
> >> When i do introduce Packet Ordering Function in conjunction with Packet Elimination on S-PE2, i think that this could create bursts of packets that can exceed the traffic contract known to the owner of the flow (e.g.: CIR/PIR etc..), because this burstyness depends for example on the differential latency of the for example two paths/copies sent between S-PE1 and S-PE2.
> >> 
> >> In effect, i think that one can not simply rely on the bounded latency iand buffering model of any pre-existing boundd latency algorithm as long as that only takes the flow CIR/PIR etc into account. 
> >> 
> >> Aka: In your POF model, where PEF is first performed, those bursts go into the POF, and if we want to hide all this POF/PEF impact from some bounded-latency operation happening after the POF, i think we need to include into the POF some more buffering functionality that reduces burstyness to what the egres bounded latency function can hanle without becoming yet another more complex ingres-edge "gate"
> >> function.
> >> 
> >> Cheers
> >>    Toerless
> >> 
> >> On Wed, Aug 11, 2021 at 11:49:24PM +0200, Toerless Eckert wrote:
> >>> High level answer:
> >>> 
> >>> If we would start to describe the management interface, e.g.:
> >>> what we want be able to configure and observe about all of PREOF 
> >>> (maybe starting with POF), then i think we will immediately get to the 
> >>> point that we expect a much more explicit description of what 
> >>> PREOF/POF has to do so that it can comply with that management 
> >>> inerface.
> >>> 
> >>> I am saying this because when i did design features in the past it did 
> >>> work out best by first starting to think about how the operator should 
> >>> see the feaure. Otherwise a lot was forgotten. In the IETF i did 
> >>> admittedly wiggle myself around doing this 100% correctly, by just 
> >>> informally referring to the CLI/config aspects (such as RFC8994), but 
> >>> that was primarily because for that work the manaement work is really 
> >>> big, so it didn't first. Writing a yang module into a POF draft itself 
> >>> should be rather painless in comparison.
> >>> 
> >>> From there on out, we can start to see how well we could describe the 
> >>> per-hop-behavior, if that is how we are allowed to call it.
> >>> 
> >>> Details hat may be missing are about stuff like: what if a packet 
> >>> arrives whose sequence number does not fall into the window of 
> >>> sequence numbers that the POF currently accepts ? Ok, we discard it. 
> >>> We update a counter tracking these events. Do we update our POF state 
> >>> machinery ? Maybe that needs to be configurable.
> >>> Should the window-size of sequence number be configurable, etc. pp
> >>> 
> >>> Good info about TSN. Thanks
> >>> 
> >>> Toerless
> >>> 
> >>> 
> >>> On Wed, Aug 04, 2021 at 10:15:15AM +0000, Balázs Varga A wrote:
> >>>> Hi Toerless,
> >>>> 
> >>>> Many thanks for the comments. Some clarifications/reactions:
> >>>> 
> >>>> a) "per hop behavior"
> >>>> In general I agree, "unexpected behavior of DetNet products" must be avoided.
> >>>> We have already POF specifics in DetNet RFCs:
> >>>> - RFC8655: introduces the term POF
> >>>> - RFC8938: gives some characteristics of POF (uses sequence numbers, 
> >>>> range of packet order protection, buffer resources)
> >>>> - RFC8964: defines POF packet processing (section 4.2.2.3) What is 
> >>>> missing in your view?
> >>>> 
> >>>> Regarding terminology I also dislike "input vs. outbut timing for 
> >>>> traffic " as POF is not about explicit timing but order of packets.
> >>>> 
> >>>> b) Management interface
> >>>> Yes, management is an important topic. If I understand your comment 
> >>>> correctly You are more concerned about the operation/maintenance 
> >>>> related counters/parameters. I think it is a valid comment, but in 
> >>>> my view it might be better to discuss it in the OAM related drafts.
> >>>> In this draft all the POF algorithm's specific parameters are 
> >>>> defined. E.g., required buffer space can be derived from these parameters and the network/flow specifics.
> >>>> 
> >>>> c) TSN standards about ordering function There were discussions 
> >>>> about ordering in TSN, but nothing standardized.
> >>>> 
> >>>> d) "Another algorithm option: configure for every packet copy path a fixed time delay"
> >>>> I think this topic needs more clarification. Some comments:
> >>>> - I do not see differences in topologies used by TSN and by DetNet. 
> >>>> Rings are also typical in TSN networks.
> >>>> - We have a path specific delay parameter defined in section "4.4. 
> >>>> The Advanced POF Algorithm", namely "POFMaxDelay_i".
> >>>> - I am not sure your proposal is a POF function or a PDF (Packet Delay Function).
> >>>> 
> >>>> Again, thanks for the review and the comments.
> >>>> 
> >>>> Cheers
> >>>> Bala'zs
> >>>> 
> >>>> -----Original Message-----
> >>>> From: detnet <[e-mail]> On Behalf Of Toerless Eckert
> >>>> Sent: Saturday, July 31, 2021 12:41 AM
> >>>> To: [e-mail]
> >>>> Subject: [Detnet] draft-varga-detnet-pof (sorry, lot of input)
> >>>> 
> >>>> Balazs, *:
> >>>> 
> >>>> I very much support work on POF, however:
> >>>> 
> >>>> I think we should aim higher than informational guidelines
> >>>> 
> >>>> I think we can build a normative document on two interfaces:
> >>>> 
> >>>> a) "per hop behavior"
> >>>> 
> >>>>   We can stanardize the externally observable behavior
> >>>>   between arriving and sent packets on the node.
> >>>> 
> >>>>   This was regularily done for DiffServ Traffic Classes in
> >>>>   TSVWG, where it is calle Per Hop Behavior. When i tried
> >>>>   to apply this term to the input vs. outbut timing for
> >>>>   traffic from what today is an IntServ flow (DetNet),
> >>>>   David Black was somewhat concerned about re-use of the
> >>>>   term, which is why i wrote "per hop behavior". IMHO,
> >>>>   it is perfectly valid to also call this PHB, but i will
> >>>>   leave it up to David to recommend appropriate terminology.
> >>>> 
> >>>>   I am only interested to be able to get as little as possible
> >>>>   unexpecte behavior of DetNet products, and we can only 
> >>>>   arrive that if we standardize the behavior.
> >>>> 
> >>>> b) Management interface
> >>>> 
> >>>>   Yang module specifying what the operator 
> >>>>   can observe about the function about the POF function, e.g: 
> >>>> 
> >>>>   "static" parameters such as how many buffers there are available,
> >>>>     algorithm (you do have algorithm in the draft)
> >>>>    - maybe read/write if configurable.
> >>>> 
> >>>>   "dynamic parameters" - read-only for monitoring, troubleshooting:
> >>>>     - buffers dynamically used,
> >>>>     - latency between first and further copies
> >>>>     - missing packets
> >>>>     - ...
> >>>> 
> >>>> Now wrt to other content:
> >>>> 
> >>>> c) It would be lovely to have a section summarizing what if anythng
> >>>>   TSN did standardize about their ordering function for all those
> >>>>   IETF participants too lazy to read all that documentation
> >>>>   and/or rejecting the notion of having to pay money to read a document.
> >>>> 
> >>>>   (informational, appendix).
> >>>> 
> >>>> d) I think it would be great to have anoher algorithm option
> >>>>   by which one could configure for every packet copy path
> >>>>   (assuming we can identify that in the forwardin plane)
> >>>>   a fixed time delay.
> >>>> 
> >>>>   The reason for this, and why i think it was not an issue in
> >>>>   TSN (with its typical topologies) is that in DetNets
> >>>>   we much more likely will have rings with significant path 
> >>>>   latency differences between A and B path,
> >>>>   and whenever the A (shorter) path looses a packet, we would wait
> >>>>   until the B path packet arrives, resulting in a lot of
> >>>>   jitter. If the DetNet controller could configure 
> >>>>   the B-A pah lateny difference as a delay, we could get rid
> >>>>   of this problem.
> >>>> 
> >>>> Still have to dig deeper into the algo details you proposed for more comments.
> >>>> 
> >>>> Cheers
> >>>>    Toerless
> >>>> 
> >>>> _______________________________________________
> >>>> detnet mailing list
> >>>> [e-mail]
> >>>> https://www.ietf.org/mailman/listinfo/detnet
> >>> 
> >>> --
> >>> ---
> >>> [e-mail]
> >> 
> >> --
> >> ---
> >> [e-mail]
> > 
> > -- 
> > ---
> > [e-mail]
> > 
> > _______________________________________________
> > detnet mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/detnet

-- 
---
[e-mail]