Re: [Detnet] Martin Duke's Discuss on draft-ietf-detnet-pof-08: (with DISCUSS and COMMENT)

[John Scudder <[e-mail]>]

For what it’s worth, it also seems to me like a weird design choice to forward out-of-order packets, when the option exists to not do so (by dropping instead). But, I accept that the authors and working group are well aware of this and have made an informed choice to insist that the POF function as the document specifies. Also, I acknowledge that the considerations may not be as obvious as they appear at first blush — I can see that if the latency bound is respected, it shouldn’t be possible for an out-of-order packet to slip through… so a change to the algorithm as written would require a non-trivial analysis of the revised algorithm. 

In short, IMO it’s OK to let this characteristic stand, while also adding more descriptive text to make it clear what the design limitations are.

Thanks,

—John

> On Jan 9, 2024, at 10:55 AM, Balázs Varga A <balazs.a.varga=[e-mail]> wrote:
> 
> 
> Hi Martin,
>  
> I think we are somewhat misaligned regarding the “guarantee of in-order delivery”.
> POF described in the draft considers the latency bound (!), when the out-of-order
> packets are re-ordered. This latency bound is an essential requirement for DetNet
> flows. As per your concern I have added new text (in the introduction section) in
> the latest versions to clarify that:
> NEW TEXT
>   POF ensures in-order delivery for packets being within
>   the latency bound of the (DetNet) flow. POF does not correct
>   errors in the packet flow e.g., duplicate packets, too
>   late packets.
> END
>  
> Does this clarified the concern?
>  
> In my understanding all concerns are now fixed with the latest draft version.
> If anything missed please let it know.
>  
> Thanks
> Bala’zs
>  
>  
> From: Martin Duke <[e-mail]> 
> Sent: Thursday, January 4, 2024 7:53 PM
> To: Balázs Varga A <[e-mail]>
> Cc: The IESG <[e-mail]>; [e-mail]; [e-mail]; [e-mail]; [e-mail]
> Subject: Re: Martin Duke's Discuss on draft-ietf-detnet-pof-08: (with DISCUSS and COMMENT)
>  
> The changes mostly look good.
>  
> I gather the other ADs would like to think about this guarantee of in-order delivery bit. If we end up sticking with the current design, I would insist that the document be clearer that the POF is not actually ensuring in-order delivery, in order to reduce packet loss.
>  
> On Thu, Jan 4, 2024 at 5:25 AM Balázs Varga A <[e-mail]> wrote:
> Hi Martin, further replies inline [BV]. Thanks Bala’zs
>  
> From: Martin Duke <[e-mail]> 
> Sent: Wednesday, January 3, 2024 8:14 PM
> To: Balázs Varga A <[e-mail]>
> Cc: The IESG <[e-mail]>; [e-mail]; [e-mail]; [e-mail]; [e-mail]
> Subject: Re: Martin Duke's Discuss on draft-ietf-detnet-pof-08: (with DISCUSS and COMMENT)
>  
> HI Balazs,
>  
> Replies inline.
>  
> On Wed, Jan 3, 2024 at 6:57 AM Balázs Varga A <[e-mail]> wrote:
> Hi Martin,
> 
> Thanks for your review. Please find clarifications below:
> 
> 1. Sequencing of PRF, PEF, and POF functions.
> <Reply> Implementation of PRF and PEF are not defied in details in IETF. A possible implementation 
> of an Elimination (PEF) functionality is described in the IEEE 802.1CB-2017 standard. PEF has its own 
> internal states and various elimination algorithms (e.g., vector recovery, match recovery) can be 
> used. If PRF and PEF are not designed and configured correctly, then PEF may result in duplicate 
> packet delivery (e.g., match recovery is used for bulk flows). 
> In section 4.1 the intention was to highlight, that POF cannot repair such scenarios.
> 
> POF is usually the last thing before egress, but not always. It may be located within the DetNet 
> network if PREOF is designed so.
>  
> [MD] Fair enough. Perhaps you could make this change:
> OLD
> Error cases in which the POF is presented duplicate packets can lead to out of
> order delivery of duplicate packets as well as to increased delays.
>  
> NEW
> Error cases in which the PRF or PEF implementation errors present duplicate packets to the POF can lead to out of
> order delivery of duplicate packets as well as to increased delays.
>  
> [BV] Not only implementation errors can lead to duplicate packets,
> so a somewhat finetuned change as per your comment:
> OLD
> Error cases in which the POF is presented duplicate packets can lead to out of
> order delivery of duplicate packets as well as to increased delays.
>  
> NEW
> Error cases in which duplicate packets are presented to the POF can lead to out of
> order delivery of duplicate packets as well as to increased delays.
>  
>  
> 
> 2. Requirements for ordering vs loss
> <Reply> These POF algorithms are designed for DetNet networks as part of the DetNet service sub-layer 
> (PREOF) functionalities. POF never drops any packet. Only the PEF has the task to drop duplicates. 
> The only task for POF is to correct the order of packets (if it is possible with its configuration). 
> Just again, if PRF and PEF are not designed and configured correctly, then POF may not re-order 
> the packets in some scenarios. If POF parameters are not designed and configured correctly,
> it may not re-order the packets.
>  
> [MD] I'm not going to hold indefinitely on this point, because it's not my design or use case.  If you and the AD *really* think this is how it should work, I'll relent.
>  
> But ISTM that by forwarding out of order packets you are undermining the POF function in the service of avoiding packet loss, which is not the POF's job.
>  
> Forwarding a packet below pof_last_sent is by definition out of order. I would think a POF would focus on guaranteeing order delivery, and rely on the reliabilty
> functions to provide reliability.
>  
> [BV] Ack.
>  
> 
> 3. Strange assumptions in the advanced algorithm.
> 3a. Is there an assumption that there is no PRF beyond the POF?
> <Reply> There is no such an assumption. It is design specific and depends on the topology, flow 
> parameters, etc., how the PREOF functions are located in the network. A POF function (and its 
> parameters) has to be designed based on the delay budget upto the point of the POF function. 
> If there are further PRF/PEF stages behind the POF, then You may need a further POF function 
> after them.
>  
> [MD] If a POF is not accounting for downstream delay, it would appear that limiting the timeout isn't actually ensuring that traffic falls within the delay guarantees.
>  
> [BV] POF parameters are configured to fulfill the delay bound.
> This is shown e.g., in Figure3. + text above it:
> “ … the
>    remaining delay budget of a flow at the POF point is larger than
>   "POFMaxDelay" time.”
>  
> 
> 3b. The introduction suggests that out-of-order delivery is a result of the PRF 
> rather than some sort of link-layer pathology.
> <Reply> The out-of-order delivery is a result of PEF (and not the PRF). 
> 
> 3b. ... So it's sufficient for the timeout to be the limited by the
> remaining time budget for the longest path.
> <Reply> No. The "remaining time budget for the longest path" shows the latest delivery time of 
> a packet that can fulfil the bounded latency requirement of the flow. The necessary buffering 
> time is determined by the latency difference of the paths. You have to be prepared for the worst
> case scenario: packet "n" received over the shortest path and packet "n-1" received over the 
> longest path. Packet "n" has to be buffered until packet "n-1" arrives (i.e., for the latency 
> difference of the paths). The advanced POF algorithm is used if the remaining delay budget of a 
> flow at the POF point is smaller than the latency difference of the paths. (Therefore, buffering for 
> "remaining time budget for the longest path" can not re-order packet "n" and "n-1".)
>  
> [MD] Yes, thank you for pointing out my analytical error. I withdraw point (3b).
>  
> [BV] Ack.
>  
> 4. Is there some reason that "Consensus Boilerplate" is set to NO on the
> datatracker page? Informational RFCs have IETF consensus!
> <Reply> I think this may be an error on the page.
>  
> [MD] Ok, this is easy to fix.
>  
> [BV] Thanks
>  
>  
> 
> I hope these have clarified your concerns.
> 
> Thanks & Cheers
> Bala'zs
> 
> -----Original Message-----
> From: Martin Duke via Datatracker <[e-mail]> 
> Sent: Tuesday, January 2, 2024 10:49 PM
> To: The IESG <[e-mail]>
> Cc: [e-mail]; [e-mail]; [e-mail]; [e-mail]; [e-mail]
> Subject: Martin Duke's Discuss on draft-ietf-detnet-pof-08: (with DISCUSS and COMMENT)
> 
> Martin Duke has entered the following ballot position for
> draft-ietf-detnet-pof-08: Discuss
> 
> When responding, please keep the subject line intact and reply to all email addresses included in the To and CC lines. (Feel free to cut this introductory paragraph, however.)
> 
> 
> Please refer to https://www.ietf.org/about/groups/iesg/statements/handling-ballot-positions/
> for more information about how to handle DISCUSS and COMMENT positions.
> 
> 
> The document, along with other ballot positions, can be found here:
> https://datatracker.ietf.org/doc/draft-ietf-detnet-pof/
> 
> 
> 
> ----------------------------------------------------------------------
> DISCUSS:
> ----------------------------------------------------------------------
> 
> There are several elements here that are either poorly phrased or misconceived
> in some way.
> 
> 1. Sequencing of PRF, PEF, and POF functions.
> 
> Section 4.1 says "However, the PREOF functions run independently without any
> state exchange required between the PEF and the POF or the PRF and the POF.
> Error cases in which the POF is presented duplicate packets can lead to out of
> order delivery of duplicate packets as well as to increased delays."
> 
> but then Section 4.6 says "In DetNet scenarios there is always an Elimination
> function before the POF (therefore duplicates are not considered by the POF)."
> 
> I can't reconcile these statements. How can there be a duplicate packet error
> case if the PEF is always before the POF?
> 
> A statement that the POF is always the last thing before egress would clean up
> some logical holes, like in (3a) below.
> 
> 2. Requirements for ordering vs loss
> 
> What is the purpose of Sec 4.3 directly forwarding packets with (sequence
> number < POF Last Sent + 1)? There's an implicit requirement that delivering
> the packet in order is less important that not dropping it, but is a strange
> requirement for a *Packet Ordering Function*. If Detnet is to be decomposed
> into three functions, it is very difficult reason about if the POF guarantees
> ordering, but sometimes it ignores that if it's trying to avoid losses. Just do
> PRF/PEF if you want to avoid losses!
> 
> So I would suggest that the POF forward (sequence number == POF Last Sent + 1)
> and drop anything earlier.
> 
> 3. Strange assumptions in the advanced algorithm.
> 
> 3a. Is there an assumption that there is no PRF beyond the POF? If there is,
> it's possible that there are different path delays beyond the POF, and that has
> to be accounted for in the algorithm. My guess is that you are assuming that,
> given Figure 1. In that case it should be stated explicitly (See point #1).
> 
> 3b. The introduction suggests that out-of-order delivery is a result of the PRF
> rather than some sort of link-layer pathology. If so, I don't see why it's
> necessary for the POF timeout to be path-dependent. That is, the shorter-path
> packets will by definition be in-order[1], and the longer path will be
> out-of-order. So it's sufficient for the timeout to be the limited by the
> remaining time budget for the longest path.
> 
> [1] Unless there's a loss on that path. But in that case, there is no advantage
> to a longer delay, so my proposed algorithm holds.
> 
> Maybe I'm getting the underlying assumptions wrong?
> 
> 4. Is there some reason that "Consensus Boilerplate" is set to NO on the
> datatracker page? Informational RFCs have IETF consensus!
> 
> 
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
> 
> Thanks to Kyle Rose for the TSVART review.
>