Re: [Detnet] FYI: draft-eckert-detnet-glbf-01.txt

[Jinoo Joung <[e-mail]>]

Toerless, please see in-line.

On Tue, Oct 3, 2023 at 8:19 AM Toerless Eckert <[e-mail]> wrote:

> Thanks, Jinoo, inline
>
> On Sat, Sep 23, 2023 at 08:19:12AM +0900, Jinoo Joung wrote:
> > Toerless,
> >
> > Your argument to me and Shaofu is summarized in the following statement.
> >
> > "The latency calculusfor C-SCORE is based on the assumption of a TSpec
> limit
> > for each flow. To me it seems clear that this calcullus does not hold
> > true when the TSpec increases across hops."
> >
> > Another way to put it would be:
> > The increased burst size of a flow will harm the latency bound calculated
> > based on the initial burst size of the flow.
>
> Yes.
>
> > My answer is:
> > The increased burst size of a flow does NOT harm the latency bound
> > calculated based on the initial burst size of the flow.
> >
> > Let's say a flow has TSpec rt+b.
> > The worst case happens is that the actual arrival follows rt+b;
> > that is at t=0 b arrives, and then packets arrive one after another, with
> > rate r.
> >
> > CLAIM: The largest E2E latency is that of the LAST packet of the initial
> > burst b. Let's call this packet p(b).
> > Proof:
> > 1) For any packet that arrives before p(b), it departs the network before
> > p(b), so CLAIM holds.
>
> Are you assuming that all packets of the burst are inserted into the
> first hop of the network with infinite speed, so that their arrival times
> into the network are about the same ? Otherwise there is no proof that
> an increase in departure time from the network is equivalent to an
> increase of latency through the network for these packets culminating in
> the
> lasst packet having the highestend-to-end latency.
>

JJ: Infinite link capacity is assumed here.


>
> > 2) Consider a packet that arrives later than p(b). Let's call it p(b)+n.
>
> So its from a second or further burst of the flow ?
>

JJ: Packet p(b)+n is a subsequent packet from the flow.
It can be either from another burst, or a stand-alone packet.


> > For simplicity of discussion, let's assume that every packet's length is
> 1.
> > Assume that p(b)+n joins the burst b at a point of the network.
> > Then by definition of the burst, at that point the arrival time
> difference
> > between p(b) and p(b)+n is larger than or equal to n/r.
>
> Not if there was rate distortion introduced.
>

JJ: The rate does not get distorted. Only the burst size increases.
The service is guaranteed to flow at the initial rate.


>
> > Assume that the burst to which these two packets belong is resolved at
> > another point of the network.
> > Between the join point and the resolve point, they go together.
> > Then the departure time difference is less than or equal to n/r, Since
> the
> > flow is served at a rate larger than or equal to r.
> > Therefore the latency difference is the departure time difference minus
> the
> > arrival time difference,
> > which is less than or equal to zero.
>
> I can't paint a picture of this part..
>
> Cheers
>     Toerless
>
> > CLAIM holds
> >
> > So, for both cases CLAIM holds.
> >
> > I hope this helps.
> > The burst accumulation effect is well known from decades ago.
> > Do you really believe that the network calculus theory has not been aware
> > of that?
> >
> > Best,
> > Jinoo
> >
> >
> > On Sat, Sep 23, 2023 at 12:39 AM Toerless Eckert <[e-mail]> wrote:
> >
> > > (eliminating many lines of empty space - your mailer is curious that
> > > way...)
> > >
> > > On Fri, Sep 22, 2023 at 04:36:16PM +0800, [e-mail]
> wrote:
> > > > Hi Toerless,
> > > >
> > > >
> > > > You provide a good example in previous mail to show how does bursts
> > > accumulation
> > > > of a single flow generate due to bursts aggregation from other flows.
> > > Here, I apologize
> > > > to invent a new word "bursts aggregation", to distinguish it with
> > > "bursts accumulation".
> > > > The former is something about multiple flows, while the later is
> > > something about multiple
> > > > service-burst-intervals of a single flow.
> > > >
> > > > Yes, bursts accumulation may cause TSpec distortion of the observed
> > > flow. I believe we
> > > > have agreement on this.
> > > >
> > > > However, the difference in our views is, the solution effect of
> > > different schedulings on
> > > > bursts accumulation.
> > > >
> > > > If I get your point, you think that re-shaping (including IR), or
> > > Damper, must be introduced
> > > > to eliminate bursts accumulation.
> > > >
> > > >
> > > > IMO, IR/Damper is just one option (note that I take Damper as similar
> > > design of IR, e.g, per
> > > > "traffic class + incoming port", except with different calculation
> > > method of eligibility time for
> > > > packets), other options may be time-rank, such as latency
> compensation
> > > in Deadline,
> > > > visual finish time in C-SCORE.
> > >
> > > Let me repeat what i just replied to Jinou:
> > >
> > > None of the scheduling mechanisms considered for rfc2212 treats a
> flow's
> > > adjacent packets
> > > differently just because they are clustered closer togeter than they
> > > should (because of
> > > prior hops burst aggregation/accumulation).
> > >
> > > Shaper/IR and Damper rectify this problem directly, albeit differently
> -
> > > resulting
> > > also to an in-time/work-preserving vs on-time/zero-jitter service
> > > experience. And
> > > a difference in whether we have per-hop state/processing scaling
> issues.
> > >
> > > > We can simply understand the difference between IR/Damper option and
> > > time-rank option
> > > >
> > > >
> > > > as below:
> > > >
> > > >
> > > > In IR/Damper option, early arrived packet (note: compared to the
> > > previous arrived
> > > > packet) is delayed in the shaping buffer for a period of time untill
> its
> > > eligibility time, before
> > >
> > > compared to the previousarrived packet ... from same flow .. only in
> > > shaper/IR, not damper
> > > damper delays all packets up to quasi synchronous latency.
> > >
> > > > that, it is REFUSED entry into the queueing sub-system.
> > > >
> > > >
> > > > In time-rank option, early arrived packet is TOLERATED entry into the
> > > queueing
> > > > sub-system but just with a large rank (or low priority) to not affect
> > > the urgency of scheduling
> > > > eligibility arrivals (thus get bounded latency for them).
> > >
> > > I remember discussions with Jakoov Stein re. draft-stein-srtsn, which
> is
> > > AFAIK also an
> > > instance of such a time-rank forwarding with deadlines, and he pointed
> out
> > > in later
> > > parts of the discussion that his (students) research also showed that
> the
> > > mechanism
> > > is a heuristic bounded latency solution, but not a deterministic.
> > >
> > > > That is, even in the case of bursts accumulation, a well-designed
> > > scheduling can still get a
> > > > bounded latency.
> > >
> > > And i fundamentally disagree, because i can not find a principle in
> these
> > > mechanisms
> > > that restores per-flow TSpec increase correctly. To elaborate iin more
> > > detail on my
> > > 4 * 100 Mbps flows via Gbps link: WFQ would not restore a 1 Mbps time
> > > difference between
> > > distorted packets p-1 and p of my example flow, it would only limit the
> > > burst to be
> > > equal or less to that of a 2.5 Mbps flow in the best case, because
> this is
> > > effectively what packet
> > > bursts of 400 * 1 Mbps across a 1 Gbps link do get with WFQ. But if the
> > > distorted p-1 and
> > > p packet happen to get into a burst of less than 400 flows packets,
> then
> > > each of p-1 and
> > > p would get an even higher share of the available 1 Gbps bandwidth and
> > > thus their distortion
> > > would stay higher. When there is no contention on some following hop,
> > > their distortion would
> > > not be eliminated at all.
> > >
> > > > However, I agree with you that there is still difference result of
> the
> > > amount of bursts accumulation
> > > > for in-time and on-time mode, with different buffer cost.
> > >
> > > My unfortunate feeling is that there is no way to build an
> > > in-time/work-preserving
> > > bounded latency solution that is per-hop stateless, because i need
> > > shaper/IR for
> > > in-time/work-preserving, and i can not build IR/shaper per-hop
> stateless.
> > >
> > > Note that our LBF work  (https://ieeexplore.ieee.org/document/9110431)
> > > equally attempted to
> > > provide such a deadline, and departure priority calculation based
> latency
> > > management - in-time.
> > > But we also failed to figure out how to provide a bounded latency
> calculus
> > > because of this
> > > TSpec distortion issue. Which is why we then went to the gLBF approach.
> > >
> > > And of course it is extremely annoying because as soon as you do such
> > > an advanced scheduler (PIFO, per-packet calculation of departure
> > > priority/time), you do
> > > minimize the effect of course (like WQF and C-SCORE and LBF do), but
> you
> > > don't eliminate it.
> > >
> > > Aka: close, but i fear no deterministic cigar.
> > >
> > > And btw: I'd be happy to be proven wrong, but the per-hop math/calculus
> > > that is used in rfc2212
> > > and elsewhere is all based on the premise of per-hop known and same
> TSpec
> > > for the flows.
> > >
> > > Cheers
> > >     Toerless
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Regards,
> > > >
> > > >
> > > > PSF
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Original
> > > >
> > > >
> > > >
> > > > From: ToerlessEckert <[e-mail]>
> > > > To: 彭少富10053815;
> > > > Cc: [e-mail] <[e-mail]>;[e-mail] <
> [e-mail]
> > > >;[e-mail] <[e-mail]
> >;
> > > > Date: 2023年09月22日 01:15
> > > > Subject: Re: [Detnet] FYI: draft-eckert-detnet-glbf-01.txt
> > > >
> > > >
> > > > Peng,
> > > >
> > > > I am always trying to escape having to get into the depts of the
> math,
> > > and instead
> > > > trying to find as simple as possible example. And the text from
> rfc2212
> > > may be somewhat
> > > > misleading when it talks about classes.
> > > >
> > > > The simple logic, why i think that WFQ does not help the math over
> FIFO
> > > are this:
> > > >
> > > > - I can have all flows with the same r_i and b_i, and then all flows
> are
> > > treated equal.
> > > > - I can all bursts be single packets , in which case the queue i
> build
> > > up is just
> > > >   one packet/flow.
> > > >
> > > > In these cases, i think WFQ will not treat packets differently from
> > > FIFO. Correct
> > > > me if i am wrong. And the problem of TSpec distortion still exists.
> > > >
> > > > Aka:  WFC (and in extrapolation C-SCORE), woul likely have benefits
> > > reducing the
> > > > TSpec distortion to some degree if b_i is multiple packets, because
> then
> > > a back-to-back
> > > > burst of packets from one flow would be broken up, but if the
> interface
> > > serving
> > > > rate is still significantly higher than the sum(r_i), then i will
> still
> > > continue
> > > > to pass bursts hop-by-hop that lead to TSpec distortion (IMHO).
> > > >
> > > > Cheers
> > > >     Toerless
> > > >
> > > >
> > > > On Thu, Sep 21, 2023 at 12:02:01PM +0800, [e-mail]
> wrote:
> > > > > Hi Jinoo,
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Thanks for your explanation.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > I agree with you that for a single observed flow, especialy with
> ideal
> > > fluid model,
> > > > >
> > > > >
> > > > > each scheduler in the network can provide a guaranteed service for
> it,
> > > even in the case of
> > > > >
> > > > >
> > > > > flow aggregation.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > For example, multiple flows, each with arrival curve A_i(t) = b_i +
> > > r_i * t, may belongs to the
> > > > >
> > > > >
> > > > > same traffic class and consume the resources (burst and bandwidth)
> of
> > > the same
> > > > >
> > > > >
> > > > > out-interface on the intermediate node. Suppose that the scheduler
> > > provide a rate-latency
> > > > >
> > > > >
> > > > > service curve R*(t - T) for that traffic class, where R >=
> sum(r_i).
> > > Then, if each flow arrived
> > > > >
> > > > >
> > > > > idealy, i.e., complied its own arrival curve, for the oberserved
> flow,
> > > it will be ensured to get a
> > > > >
> > > > >
> > > > > guaranteed service rate R' from the total service rate R.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Suppose that on each node the guaranteed service curve for the
> > > observed flow (e.g, flow 1)
> > > > >
> > > > >
> > > > > is R'*(t - T') , then, according to "Pay Bursts Only Once" rule,
> the
> > > E2E latency may be:
> > > > >
> > > > >
> > > > > T' * hops + b_1/R'. It seems that E2E latency just consider the
> burst
> > > of flow 1 (i.e., b_1) and
> > > > >
> > > > >
> > > > > only once, and never consider other flows..
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > However,  the truth is hidden in T'.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > According to section 6 "Aggregate Scheduling" discussion in netcal
> > > book,  the guaranteed
> > > > >
> > > > >
> > > > > service curve for flow 1, i.e., R'*(t - T'), may be deduced by:
> > > > >
> > > > >
> > > > >     R*(t - T) - sum(b_2 + r_2*t, b_3 + r_3*t, ..., b_n + r_n*t)
> > > > >
> > > > >
> > > > > = (R - r_2 - r_3 -...- r_n) * (t - ((b_2 + b_3 + ... + b_n + R*T) /
> > > (R - r_2 - r_3 -...- r_n)))
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Thus, R' equals to (R - r_2 - r_3 -...- r_n),
> > > > >
> > > > >
> > > > > and T' equals to ((b_2 + b_3 + ... + b_n + R*T) /  (R - r_2 - r_3
> > > -...- r_n)).
> > > > >
> > > > >
> > > > > It can be seen that all bursts of other flows contribute the
> result of
> > > T', on each node,
> > > > >
> > > > >
> > > > > again and again.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > If we take each b_i as 1/n of B, r_i as 1/n of R, T as 0, to simply
> > > compare the above PBOO
> > > > >
> > > > >
> > > > > based latency estiamation and traffic class based latency
> estimation,
> > > we may find that
> > > > >
> > > > >
> > > > > the former is b_1/r_1 + hops*(n-1)*b_1/r_1, while the later is
> > > b_1/r_1. It is amazing that
> > > > >
> > > > >
> > > > > the former is about n times the latter .
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Please correct me if I misunderstand.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Regards,
> > > > >
> > > > >
> > > > > PSF
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Original
> > > > >
> > > > >
> > > > >
> > > > > From: JinooJoung <[e-mail]>
> > > > > To: 彭少富10053815;
> > > > > Cc: Toerless Eckert <[e-mail]>;DetNet WG <[e-mail]>;
> > > [e-mail] <[e-mail]>;
> > > > > Date: 2023年09月19日 16:13
> > > > > Subject: Re: [Detnet] FYI: draft-eckert-detnet-glbf-01.txt
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hello Shaofu.
> > > > >
> > > > >
> > > > > Thanks for the most relevant question, which, as you have said, is
> > > related to Toerless' last question regarding "reshaping on merge
> points".
> > > > >
> > > > >
> > > > > If I may repeat Toerless' question below:
> > > > >
> > > > > "If you are relying on the same math as what rfc2212 claims, just
> > > replacing
> > > > > stateful WFQ with stateless, then it seems to me that you would
> > > equally need
> > > > > the shapers demanded by rfc2212 on merge-points. I do not see them
> in
> > > C-SCORE."
> > > > >
> > > > > And your question is:
> > > > >
> > > > > " "Pay Bursts Only Once"  may be only applied in the case that the
> > > network provides
> > > > > a dedicated service rate to a flow. Our network naturally
> aggregates
> > > flows at every node,
> > > > > therefore does not dedicate a service rate to a flow, and PBOO does
> > > not apply."
> > > > > My answer is:
> > > > >
> > > > > "Pay Bursts Only Once" is applied in the case that the network
> provides
> > > > > a GUARANTEED service rate to a flow.
> > > > >
> > > > > Fair queuing, C-SCORE, or even FIFO scheduler can guarantee a
> service
> > > rate to a flow.
> > > > >
> > > > > As long as a flow, as a single entity, is guaranteed a service
> rate,
> > > it is not considered aggregated or merged.
> > > > > Therefore reshaping is not necessary, and PBOO holds.
> > > > >
> > > > > Below is my long answer. Take a look if you'd like.
> > > > >
> > > > > Fair queuing, Deficit round robin, or even FIFO scheduler
> guarantees a
> > > service rate to a flow,
> > > > > if the total arrival rate is less than the link capacity.
> > > > > The only caveat is that FIFO can only guarantee the service rate
> that
> > > is equal to the arrival rate,
> > > > > while FQ and DRR can adjust the service rate to be larger than the
> > > arrival rate.
> > > > > If such rate-guaranteeing schedulers are placed in a network,
> > > > > then a flow is guaranteed to be served with a certain service rate,
> > > > > and not considered to be "aggregated", in the intermediate nodes.
> > > > >
> > > > > RFC2212, page 9~10, in subsection "Policing", states the
> followings:
> > > > >
> > > > > "Reshaping is done at all heterogeneous source branch points and at
> > > all source merge points."
> > > > >
> > > > >
> > > > > "Reshaping need only be done if ..."
> > > > >
> > > > >
> > > > >
> > > > > "A heterogeneous source branch point is a spot where the
> > > > > multicast distribution tree from a source branches to multiple
> > > > > distinct paths, and the TSpec’s of the reservations on the various
> > > > > outgoing links are not all the same."
> > > > >
> > > > > "A source merge point is where the distribution paths or trees from
> > > two different sources
> > > > > (sharing the same reservation) merge."
> > > > >
> > > > > In short, RFC2212 states that reshaping CAN be necessary at the
> flow
> > > aggregation and deaggregation points.
> > > > >
> > > > > Flow aggregation and deaggregation are something happening usually
> at
> > > the network boundary, between networks, etc,
> > > > > with careful planning. Flow multiplexing into a FIFO is not
> considered
> > > an aggregation.
> > > > >
> > > > > Best,
> > > > >
> > > > > Jinoo
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Sep 19, 2023 at 10:57 AM <[e-mail]> wrote:
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Hi Jinoo, Toerless,
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Sorry to interrupt your discussion.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > According to NetCal book (
> > > https://leboudec.github.io/netcal/latex/netCalBook.pdf),
> > > > >
> > > > >
> > > > > "Pay Bursts Only Once"  may be only applied in the case that the
> > > network provide
> > > > >
> > > > >
> > > > > a dedicate service rate (may be protected by a dedicate queue, or
> even
> > > a dedicate
> > > > >
> > > > >
> > > > > sub-link) for the observed flow, such as the guarantee service
> defined
> > > in RFC2212.
> > > > >
> > > > >
> > > > > In brief, there is no other flows sharing service rate with the
> > > observed flow. That is,
> > > > >
> > > > >
> > > > > there is no fan-in, no so-called " competition flows belongs to the
> > > same traffic class
> > > > >
> > > > >
> > > > > at the intermediate node".
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Traffic class is often used to identify flow aggregation, which is
> not
> > > the case that
> > > > >
> > > > >
> > > > > "Pay Bursts Only Once" may be applied. It seems that in IP/MPLS
> > > network, flow
> > > > >
> > > > >
> > > > > aggregation is naturely. The picture Toerless showed in the
> previous
> > > mail is exactly
> > > > >
> > > > >
> > > > > related with flow aggregation, i.e, the observed flow may be
> > > interfered by some
> > > > >
> > > > >
> > > > > competition flows belongs to the same traffic class at node A and
> > > again interfered by
> > > > >
> > > > >
> > > > > other competition flows belongs to the same traffic class at node B
> > > separately.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Please correct me if I misunderstood.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Regards,
> > > > >
> > > > >
> > > > > PSF
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > _______________________________________________
> > > > > detnet mailing list
> > > > > [e-mail]
> > > > > https://www.ietf.org/mailman/listinfo/detnet
> > > > >
> > > > >
> > > > > Toerless,
> > > > > It seems that you argue two things in the last email.
> > > > >
> > > > > 1) Your first argument: The E2E latency is the sum of per-hop
> > > latencies..
> > > > >
> > > > > You are half-right.
> > > > > According to RFC2212, page 3, the generic expression for the E2E
> > > latency bound of a flow is:
> > > > >
> > > > > [(b-M)/R*(p-R)/(p-r)] + (M+Ctot)/R + Dtot.    (1)
> > > > >
> > > > >
> > > > > Let's call this expression (1).
> > > > >
> > > > > Here, b is the max burst, M the max packet length, R the service
> rate,
> > > p the peak rate, r the arrival rate;
> > > > > and Ctot and Dtot are the sums of C and D, which are so called
> "error
> > > terms",  over the hops.
> > > > > Thus, the E2E latency bound can be a linear function of the hop
> > > count,
> > > > > since Ctot and Dtot are functions of hop count.
> > > > > However, the first term [(b-M)/R*(p-R)/(p-r)], which includes b, is
> > > not..
> > > > > So you can see the E2E latency is NOT just the sum of per-hop
> > > latencies..
> > > > >
> > > > > 2) Your second argument: C-SCORE cannot be free from burst
> > > accumulation and other flows' burst.
> > > > > My short answer: It is free from burst accumulation or other flows'
> > > burst.
> > > > >
> > > > > Imagine an ideal system, in which your flow is completely isolated.
> > > > > It is ALONE in the network, whose link has the rate R in every hop.
> > > > > No other flows at all.
> > > > >
> > > > > Assume the flow's arrival process is indeed rt+b. At time 0, b
> arrives
> > > instantly.
> > > > > (This is the worst arrival, such that it makes the worst latency.)
> > > > > Then your flow experiences the worst latency b/R at the first node,
> > > > > and M/R (transmission delay of a packet) at the subsequent nodes.
> > > > >
> > > > >
> > > > > I.e.  (b-M)/R + H*M/R, where H is the hop count.
> > > > >
> > > > > This is a special case of (1), where R is the same with r, C is M,
> and
> > > D is zero.
> > > > >
> > > > >
> > > > > Fair queuing and C-SCORE are the best approximations of such an
> ideal
> > > system, with D equals Lmax/LC,
> > > > >
> > > > > where Lmax and LC are the max packet length in the link and the
> > > capacity of the link, respectively.
> > > > > Therefore the E2E latency bound of C-SCORE is
> > > > >
> > > > > (b-M)/R + H*(M/R + Lmax/LC),
> > > > >
> > > > >
> > > > > which is again another special case of  (1).
> > > > > Note that b is divided by R, not LC.
> > > > >
> > > > > It is well known that a FIFO scheduler's D (error term) is a
> function
> > > of the sum of other flows' bursts.
> > > > > Their E2E latency bounds are approximately
> > > > > (b-M)/R + H*[M/R + (Sum of Bursts)/LC].
> > > > >
> > > > >
> > > > > ATS or UBS also relies on FIFO, but the bursts are suppressed as
> the
> > > initial values,
> > > > >
> > > > > therefore enjoys a much better E2E latency expression.
> > > > >
> > > > > Hope this helps.
> > > > >
> > > > > Best,
> > > > > Jinoo
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Sun, Sep 17, 2023 at 1:42 AM Toerless Eckert <[e-mail]>
> wrote:
> > > > >
> > > > > On Thu, Sep 14, 2023 at 08:08:24PM +0900, Jinoo Joung wrote:
> > > > >  > Toerless, thanks for the reply.
> > > > >  > In short:
> > > > >  >
> > > > >  > C-SCORE's E2E latency bound is NOT affected by the sum of
> bursts,
> > > but only
> > > > >  > by the flow's own burst.
> > > > >
> > > > >                      +----+
> > > > >            If1   --->|    |
> > > > >            ...       |  R |x--->
> > > > >            if100 --->|    |
> > > > >                      +----+
> > > > >
> > > > >  If you have a router with for example 100 input interfaces, all
> > > sending packets
> > > > >  to the same ooutput interfaces, they all will have uncorrelated
> flows
> > > arriving
> > > > >  from the different interfaces, and at least each interface can
> have a
> > > packet
> > > > >  arriving t the same time in R.
> > > > >
> > > > >  The basic caluculus of UBS is simply the most simple and hence
> > > conservative,
> > > > >  assuming all flows packet can arrive from different interface
> without
> > > > >  rate limits. But of course you can do the latency calculations in
> a
> > > tighter
> > > > >  fashion for UBS. WOuld be interesting to learn if IEEE for TSN-ATS
> > > (Qcr) was
> > > > >  looking into any of this. E.g.: applying line shaping for the
> > > aggregate of
> > > > >  flows from the same input interface and incorporating the service
> > > curve of
> > > > >  the node.
> > > > >
> > > > >  In any case, whether the tighter latency calculus is you do for
> Qcr,
> > > it
> > > > >  is equally applicable to gLBF.
> > > > >
> > > > >  > It is bounded by (B-L)/r + H(L/r + Lmax/R), where B is the
> flow's
> > > max burst
> > > > >  > size.
> > > > >
> > > > >  See above picture. How could the packet in question not also
> suffer
> > > the
> > > > >  latency introduced by the other 99 packets randomnly arriving at
> almos
> > > > >  the same time, just a tad earlier.
> > > > >
> > > > >  > You can also see that B appears only once, and not multiplied by
> > > the hop
> > > > >  > count H.  So the burst is paid only once.
> > > > >
> > > > >  C-SCORE is a stateless fair queuing (FQ) mechanism, but making FQ
> > > state,
> > > > >  does not change the fact that FQ itself does not eliminate the
> > > > >  burst accumulation on merge points such as shon in the picture.
> > > rfc2212
> > > > >  which for example also recommends FQ independently mandates the
> use of
> > > > >  reshaping.
> > > > >
> > > > >  > Please see inline marked with JJ.
> > > > >
> > > > >  yes, more inline.
> > > > >
> > > > >  > On Thu, Sep 14, 2023 at 3:34 AM Toerless Eckert <[e-mail]>
> > > wrote:
> > > > >  [cutting off dead leafs]
> > > > >
> > > > >  > > I was only talking about the E2E latency bound guaranteeable
> by
> > > DetNet
> > > > >  > > Admission Control (AC).
> > > > >  >
> > > > >  > JJ: Admission control has two aspects.
> > > > >  > First, guaranteeing the total arrival rate (or allocated service
> > > rate) does
> > > > >  > not exceed the link capacity.
> > > > >  > Second, assuring a service level agreement (SLA), or a requested
> > > latency
> > > > >  > bound (RSpec) to a flow.
> > > > >  > By "E2E latency bound guaranteeable by DetNet Admission Control
> > > (AC)",
> > > > >  > I think you mean that negotiating the SLA first, and then,
> based on
> > > the
> > > > >  > negotiated latency, allocating per-hop latency to a flow.
> > > > >  > I suggest this per-hop latency allocation, and enforcing that
> value
> > > in a
> > > > >  > node, is not a good idea,
> > > > >  > since it can harm the advantage of "pay burst only once (PBOO)"
> > > property.
> > > > >
> > > > >  Its the only way AFAIK to achieve scalability in number of hops
> and
> > > number of
> > > > >  flows by being able to calculate latency as a linear composition
> of
> > > per-hop
> > > > >  latencies. This is what rfc2212 does, this is what Qcr does, *CQF,
> > > gLBF and so on.
> > > > >
> > > > >  > PBOO can be interpreted roughly as: If your burst is resolved
> at a
> > > point in
> > > > >  > a network, then it is resolved and does not bother you anymore.
> > > > >  > However, in the process of resolving, the delay is inevitable.
> > > > >
> > > > >  Remember tht the problemtic burstyness is an increase in the
> > > intra-flow
> > > > >  burstyness resulting from merge-point based unexpected latency
> > > > >  to p-1 of the flow followed by no such delay for packet p of the
> same
> > > flow,
> > > > >  and hence clustering p-1 and p closer together, making it exceed
> its
> > > > >  reserved burst size.
> > > > >
> > > > >  This can not be compensated for in a work-conserving way by simply
> > > capturing
> > > > >  the per-hop latency of each packet p-1 and p alone, but it
> requires a
> > > shaper
> > > > >  (or UR) to get rid off.
> > > > >
> > > > >  > JJ: Because you don't know exactly where the burst is resolved,
> > > > >  > when you calculate the per-node latency bound, the latency due
> to
> > > the burst
> > > > >  > has to be added as a portion of the latency bound.
> > > > >  > Thus the sum of per-node bounds is much larger than the E2E
> latency
> > > bound
> > > > >  > calculated with seeing the network as a whole.
> > > > >
> > > > >  Consider a flow passing through 10 hops. On every hop, you have
> > > potentially
> > > > >  merge-point with new traffic flows and new burst collision. All
> that
> > > we do
> > > > >  in the simple UBS/Qcr calculus is to take the worst case into
> account,
> > > > >  where worst case may not even be admitted now, but which could be
> > > admitted
> > > > >  in the future, and at that point in time you do not want to go
> back
> > > and
> > > > >  change the latency guarantee for your already admitted flow.
> > > > >
> > > > >          Src2  Src3  Src4  Src5  Src6  Src7  Src8   Src9 Src10
> > > > >           |     |     |     |     |     |     |     |     |
> > > > >           v     v     v     v     v     v     v     v     v
> > > > >   Src1 -> R1 -> R2 -> R3 -> R4 -> R5 -> R6 -> R7 -> R8 -> R9 ->
> R10 ->
> > > Rcv1
> > > > >                  |     |     |     |     |     |     |     |     |
> > >
> > > > >                  v     v     v     v     v     v     v     v     v
> > > > >                 Rcv2  Rcv3  Rcv4  Rcv5  Rcv6  Rcv7  Rcv8  Rcv9
> Rcv10
> > > > >
> > > > >  Above is example, where Flow 1 from Src1 to Rcv1 will experience
> such
> > > > >  merge-point burst accumulation issue on every hop - worst case.
> And
> > > as mentioned
> > > > >  before, yes, when you use the simple calculus, you're also
> > > overcalculating
> > > > >  the per-hop latency for flows that e.g.: all run in parallel to
> Src1,
> > > but
> > > > >  that is just a matter of using stricter network calculus. And
> because
> > > > >  Network Calculus is complex, and i didn't want to start becoming
> an
> > > expert on
> > > > >  it, i simply built the stateless solution in a way where i can
> reuse a
> > > > >  pre-existing, proven and used-in-standards (Qcr) queuing-model,
> > > calculus.
> > > > >
> > > > >
> > > > >  > JJ: If you enforce resolving it at the network entrance by a
> strict
> > > > >  > regulation, then you may end up resolving while not actually
> needed
> > > to.
> > > > >  > However, this approach is feasible. I will think more about it.
> > > > >
> > > > >  I think the flow-interleaving is a much more fundamental issue of
> > > higher
> > > > >  utilization with large number of flows all with low bitrates.
> Look at
> > > > >  the examples of the draft-eckert-detnet-flow-interleaving, and
> tell me
> > > > >  how else but time-division-multiplexing one would be able to solve
> > > this.
> > > > >  Forget the complex option where flows from diffeent ingres
> routers to
> > > > >  different egres routers are involved. Just the most simple
> problem of
> > > > >  one ingres router PE1, maybe 10 hops through thre network to a
> PE2,
> > > and 10,000
> > > > >  acyclic flows going to thre same egress router PE2.
> > > > >
> > > > >  Seems to me quite obvious tht you can as well resolve the burst on
> > > ingress
> > > > >  PE1 instead of hoping, and getting complex math by trying to do
> this
> > > > >  on further hops along the path.
> > > > >
> > > > >  Cheers
> > > > >      Toerless
> > > > >
> > > > >  > >
> > > > >  > > > It can be a function of many factors, such as number of
> flows,
> > > their
> > > > >  > > > service rates, their max bursts, etc.
> > > > >  > > > The sum of service rates is a deciding factor of
> utilization.
> > > > >  > >
> > > > >  > > Given how queuing latency always occurs from collision of
> packets
> > > in
> > > > >  > > buffers,
> > > > >  > > the sum of burst sizes is the much bigger problem for DetNet
> than
> > > service
> > > > >  > > rates. But this is a side discuss.
> > > > >  > >
> > > > >  >
> > > > >  > JJ: That is a good point. So we should avoid queuing schemes
> whose
> > > latency
> > > > >  > bounds are affected by the sum of bursts.
> > > > >  > Fortunately, C-SCORE's E2E latency bound is NOT affected by the
> sum
> > > of
> > > > >  > bursts, but only by the flow's own burst.
> > > > >  > It is bounded by (B-L)/r + H(L/r + Lmax/R), where B is the
> flow's
> > > max burst
> > > > >  > size.
> > > > >  > You can also see that B appears only once, and not multiplied by
> > > the hop
> > > > >  > count H.
> > > > >  > So the burst is paid only once.
> > > > >  >
> > > > >  >
> > > > >  > > > So, based on an E2E latency bound expression, you can guess
> the
> > > bound at
> > > > >  > > > 100% utilization.
> > > > >  > > > But you can always fill the link with flows of any burst
> sizes,
> > > therefore
> > > > >  > > > the guess can be wrong.
> > > > >  > >
> > > > >  > > "guess" is not a good work for DetNet.
> > > > >  > >
> > > > >  > > A DetNet bounded latency mechanisms needs a latency bound
> > > expression
> > > > >  > > (calculus)
> > > > >  > > to be a guaranteeable (over)estimate of the bounded latency
> > > independent of
> > > > >  > > what other competing traffic there may be in the future. Not a
> > > "guess".
> > > > >  > >
> > > > >  >
> > > > >  > JJ: Right. We should not guess. We should be able to provide an
> > > exact
> > > > >  > mathematical expression for latency bound.
> > > > >  > Because you argued in the previous mail that the latency bound
> > > should be
> > > > >  > obtained based on 100% utilization,
> > > > >  > I was trying to explain why that should not be done.
> > > > >  >
> > > > >  >
> > > > >  > > > Admission control, on the other hand, can be based on
> > > assumption of high
> > > > >  > > > utilization level, but not necessarily 100%.
> > > > >  > > >
> > > > >  > > > You do not assume 100% utilization when you slot-schedule,
> > > don't you?
> > > > >  > >
> > > > >  > > I don't understand what you mean with slot-schedule, can you
> > > please
> > > > >  > > explain ?
> > > > >  > >
> > > > >  >
> > > > >  > JJ: Slot-scheduling, which is a common term in the research
> > > community,
> > > > >  > is a mapping of flow into a slot (or cycle) in a slot (or cycle)
> > > based
> > > > >  > queuing methods, such as CQF.
> > > > >  > When we say a schedulability, it usually means whether we can
> > > allocate
> > > > >  > requesting flows into slots (cycles) with a preconfigured
> cycle's
> > > length
> > > > >  > and number.
> > > > >  >
> > > > >  >
> > > > >  > > > So "incremental scheduling" is now popular.
> > > > >  > >
> > > > >  > > Not sure what you mean with that term.
> > > > >  >
> > > > >  >
> > > > >  > JJ: The incremental scheduling means, when a new flow wants to
> join
> > > the
> > > > >  > network, then the network examines the schedulability of the
> flow,
> > > > >  > without altering the existing flows' schedule.
> > > > >  >
> > > > >  >
> > > > >  > >
> > > > >  > >
> > > > >  > > I am only thinking about "admitting" when it comes to bounded
> > > end-to-end
> > > > >  > > latency, aka: action by the AC of the DetNet
> controller-plane, and
> > > > >  > > yes, that needs to support "on-demand" (incremental?), aka:
> > > whenever
> > > > >  > > a new flow wants to be admitted.
> > > > >  > >
> > > > >  > > > 2) In the example I gave, the two flows travel the same
> path,
> > > thus the
> > > > >  > > > second link's occupancy is identical to the first one.
> > > > >  > > > Thus the competency levels in two links are the same,
> contrary
> > > to your
> > > > >  > > > argument.
> > > > >  > >
> > > > >  > > I guss we started from different assumptions about details not
> > > explicitly
> > > > >  > > mentioned.  For example, i guess we both assume that the
> sources
> > > connect
> > > > >  > > to the
> > > > >  > > first router with arbitrary high interfaces so that we could
> ever
> > > get close
> > > > >  > > to 2B/R on the first interface.
> > > > >  > >
> > > > >  > > But then we differed in a core detail. here is my assumption
> for
> > > > >  > > the topology / admission control:
> > > > >  > >
> > > > >  > >                      Src3          to R4
> > > > >  > >               +----+   \  +----+  /  +----+
> > > > >  > >       Src1 ---|    |    \-|    |-/   |    |
> > > > >  > >       Src2 ---| R1 |x-----| R2 |x----| R3 |
> > > > >  > >               |    |.     |    |.    |    |
> > > > >  > >               +----+.     +----+.    +----+
> > > > >  > >                     |           |
> > > > >  > >                 2B buffer    2B buffer
> > > > >  > >                 R srv rate   R srv rate
> > > > >  > >
> > > > >  > > Aka: in my case, i was assuming that there could be case where
> > > > >  > > the interface from R2 to R3 could have a 2B/R queue (and not
> > > > >  > > assuming further optimizations in calculus). E.g.: in some
> > > > >  > > other possible scenario, Src2 sends to R2, and Src3 and Src1
> to
> > > > >  > > R3 for example.
> > > > >  > >
> > > > >  >
> > > > >  > JJ: You can come up with an example that your scheme works well.
> > > > >  > But that does not negate the counterexample I gave.
> > > > >  >
> > > > >  > JJ: Again, there are only two flows.
> > > > >  > And, B is not the buffer size. B is the max burst size of a
> flow.
> > > > >  > R is the link capacity.
> > > > >  > Please review carefully the example I gave.
> > > > >  >
> > > > >  >
> > > > >  > >
> > > > >  > > You must have assumed that the totality of the DetNet
> admission
> > > control
> > > > >  > > relevant topology is this:
> > > > >  > >
> > > > >  > >               +----+      +----+     +----+
> > > > >  > >       Src1 ---|    |      |    |     |    |
> > > > >  > >       Src2 ---| R1 |x-----| R2 |x----| R3 |
> > > > >  > >               |    |.     |    |.    |    |
> > > > >  > >               +----+.     +----+.    +----+
> > > > >  > >                     |           |
> > > > >  > >                 2B buffer    2B buffer
> > > > >  > >                 R srv rate   R srv rate
> > > > >  > >
> > > > >  > > Aka: DetNet admission control would have to be able to predict
> > > that
> > > > >  > > under no permitted admission scenario, R2 would build a DetNet
> > > queue,
> > > > >  > > so even when Src1 shows up as the first and only flow, the
> > > admission
> > > > >  > > control could permit a latency to R3 of 2B/R - only for the
> > > maximum
> > > > >  > > delay through R1 queue and 0 for R2 queue.
> > > > >  > >
> > > > >  > > But if this is the whole network and the admission control
> logic
> > > > >  > > can come to this conclusion, then of course it could equally
> do
> > > the
> > > > >  > > optimization and not enable gLBF Dampening on R2 output
> > > > >  > > interface, or e.g.: set MAX=0 or the like. An e'voila, gLBF
> > > > >  > > would also give 2B/R - but as said, i think it's not a
> deployment
> > > > >  > > relevant example.
> > > > >  >
> > > > >  >
> > > > >  > JJ: If you can revise and advance the gLBF, that would be great.
> > > > >  > I am willing to join that effort, if you would like to.
> > > > >  >
> > > > >  >
> > > > >  > >
> > > > >  > >
> > > > >  > > Cheers
> > > > >  > >     Toerless
> > > > >  > >
> > > > >  > > > Please see inline with JJ.
> > > > >  > > >
> > > > >  > > > Best,
> > > > >  > > > Jinoo
> > > > >  > > >
> > > > >  > > > On Wed, Sep 13, 2023 at 9:06 AM Toerless Eckert <
> [e-mail]>
> > > wrote:
> > > > >  > > >
> > > > >  > > > > On Fri, Jul 21, 2023 at 08:47:18PM +0900, Jinoo Joung
> wrote:
> > > > >  > > > > > Shaofu, thanks for the reply.
> > > > >  > > > > > It is my pleasure to discuss issues like this with you.
> > > > >  > > > > >
> > > > >  > > > > > The example network I gave is a simple one, but the
> > > scenario is the
> > > > >  > > worst
> > > > >  > > > > > that can happen.
> > > > >  > > > > > The E2E latency bounds are thus,
> > > > >  > > > > >
> > > > >  > > > > > for Case 1: ~ 2B/R
> > > > >  > > > > > for Case 2: ~ 2 * (2B/R)
> > > > >  > > > >
> > > > >  > > > > This is a bit terse, let me try to expand:
> > > > >  > > > >
> > > > >  > > > > Case 1 is FIFO or UBS/ATS, right / Case 2 is gLBF ?
> > > > >  > > > >
> > > > >  > > >
> > > > >  > > > JJ: Correct.
> > > > >  > > >
> > > > >  > > >
> > > > >  > > > >
> > > > >  > > > > Assuming i am interpreting it right, then this is
> > > inconsistent with
> > > > >  > > your
> > > > >  > > > > setup: You said all links are the same so both hops do
> have
> > > the same
> > > > >  > > > > buffer and rates, so the admission controller also expect
> to
> > > > >  > > > > have to put as many flows on second link/queue that it
> fills
> > > up 2B.
> > > > >  > > > >
> > > > >  > > >
> > > > >  > > > JJ: Yes, two links are identical. However, as I have
> mentioned,
> > > > >  > > > an E2E latency bound is calculated based on a given network
> > > environment.
> > > > >  > > > We don't always consider a filled up link capacity.
> > > > >  > > > BTW, B is the max burst size of the flow.
> > > > >  > > >
> > > > >  > > >
> > > > >  > > >
> > > > >  > > > >
> > > > >  > > > > You just then made an example, where there was never such
> an
> > > amount
> > > > >  > > > > of competing traffic on the second hop. But tht does not
> mean
> > > that
> > > > >  > > > > the admission controller could guarantee in UBS/ATS would
> have
> > > > >  > > > > less per-hop latency than 2B/R.
> > > > >  > > >
> > > > >  > > >
> > > > >  > > > JJ: Again, two links are identical and two flows travel both
> > > links.
> > > > >  > > > The difference between Case 1 and Case 2 is not because of
> the
> > > different
> > > > >  > > > competition level (they are identical.)
> > > > >  > > > but because of the non-work conserving behaviour of the
> second
> > > link in
> > > > >  > > Case
> > > > >  > > > 2.
> > > > >  > > >
> > > > >  > > >
> > > > >  > > > >
> > > > >  > > > > If the admission controller knew there would never be a
> queue
> > > on the
> > > > >  > > > > second hop, then gLBF likewise would not need to do a
> Damper
> > > on the
> > > > >  > > > > second hop. Hence as i said previously, the per-hop and
> > > end-to-end
> > > > >  > > > > bounded latency guarantee is the same between UBS and
> gLBF.
> > > > >  > > > >
> > > > >  > > > > > And again, these are the WORST E2E latencies that a
> packet
> > > can
> > > > >  > > experience
> > > > >  > > > > > in the two-hop network in the scenario.
> > > > >  > > > >
> > > > >  > > > > Its not the worst case latency for the UBS case. you just
> did
> > > not have
> > > > >  > > > > an example to create the worst case amount of competing
> > > traffic. Or you
> > > > >  > > > > overestimed the amount of buffering and hence per-hop
> latency
> > > for the
> > > > >  > > > > UBS/ATS casee.
> > > > >  > > > >
> > > > >  > > > > > In any network that is more complex, the E2E latency
> bounds
> > > of two
> > > > >  > > > > schemes
> > > > >  > > > > > are very different.
> > > > >  > > > >
> > > > >  > > > > Counterexample:
> > > > >  > > > >
> > > > >  > > > > You have a network with TSN-ATS. You have an admission
> > > controller.
> > > > >  > > > > You only have one priority for simplicity of example.
> > > > >  > > > >
> > > > >  > > > > You do not want to dymamically signal changed end-to-end
> > > latencies
> > > > >  > > > > to applications... because its difficult. So you need to
> plan
> > > > >  > > > > for worst-case bounded latencies under maximum amount of
> > > traffic
> > > > >  > > > > load. In a simple case this means you give each interface
> > > > >  > > > > a queue size B(i)/r = 10usec. Whenever a new flow needs
> to be
> > > > >  > > > > added to the network, you find a path where all the
> buffers
> > > > >  > > > > have enough space for your new flows burst and you signal
> > > > >  > > > > to the application that the end-t-end guaranteed latency
> is
> > > > >  > > > > P(path)+N*10usec, where P is the physical propagation
> latecy
> > > of
> > > > >  > > > > the path and N is the number of hops it has.
> > > > >  > > > >
> > > > >  > > > > And in result, all packets from the flow will arrive with
> > > > >  > > > > a latency between P(path)...P(path)+N*10usec - depending
> > > > >  > > > > on network load/weather.
> > > > >  > > > >
> > > > >  > > > > Now we replace UBS in the routers with gLBF. What changes
> ?
> > > > >  > > > >
> > > > >  > > > > 1) With UBS the controller still had to signal every new
> and
> > > > >  > > > > to-be-deleted flow to every router along it path to set
> up the
> > > > >  > > > > IR for the flow. This goes away (big win).
> > > > >  > > > >
> > > > >  > > > > 2) The forwarding is in our opinion cheaper/faster to
> > > implement
> > > > >  > > > > (because of lack of memory read/write cycle of IR).
> > > > >  > > > >
> > > > >  > > > > 3) The application now sees all packets arrive at fixed
> > > latency
> > > > >  > > > > of P(path)+N*10usec. Which arguably to the application
> that
> > > > >  > > > > MUST have bounded latency is from all examples i know
> > > > >  > > > > seen rather as a benefit than as a downside.
> > > > >  > > > >
> > > > >  > > > > Cheers
> > > > >  > > > >     Toerless
> > > > >  > > > >
> > > > >  > > > >
> > > > >  > > > > >
> > > > >  > > > > > Best,
> > > > >  > > > > > Jinoo
> > > > >  > > > > >
> > > > >  > > > > > On Fri, Jul 21, 2023 at 8:31 PM <[e-mail].
> .cn>
> > > wrote:
> > > > >  > > > > >
> > > > >  > > > > > >
> > > > >  > > > > > > Hi Jinoo,
> > > > >  > > > > > >
> > > > >  > > > > > >
> > > > >  > > > > > > I tried to reply briefly. If Toerless have free time,
> can
> > > confirm
> > > > >  > > it.
> > > > >  > > > > > >
> > > > >  > > > > > >
> > > > >  > > > > > > Here, when we said latency bound formula, it refers to
> > > worst-case
> > > > >  > > > > latency.
> > > > >  > > > > > >
> > > > >  > > > > > >
> > > > >  > > > > > > Intuitively, the worst-case latency for gLBF (damper +
> > > shaper +
> > > > >  > > > > scheduler)
> > > > >  > > > > > >
> > > > >  > > > > > > is that:
> > > > >  > > > > > >
> > > > >  > > > > > >     damping delay per hop is always 0. (because
> > > scheduling delay =
> > > > >  > > MAX)
> > > > >  > > > > > >
> > > > >  > > > > > >     shaping delay is always 0. (because all are
> > > eligibility
> > > > >  > > arrivals)
> > > > >  > > > > > >
> > > > >  > > > > > >     scheduling delay is always MAX (i.e., concurent
> full
> > > burst
> > > > >  > > from all
> > > > >  > > > > > > eligibility arrivals on each hop)
> > > > >  > > > > > >
> > > > >  > > > > > >
> > > > >  > > > > > > Similarly, the worst-case latency for UBS (shaper +
> > > scheduler) is
> > > > >  > > that:
> > > > >  > > > > > >
> > > > >  > > > > > >     shaping delay is always 0. (because all are
> > > eligibility
> > > > >  > > arrivals)
> > > > >  > > > > > >
> > > > >  > > > > > >     scheduling delay is always MAX (i.e., concurent
> full
> > > burst
> > > > >  > > from all
> > > > >  > > > > > > eligibility arrivals on each hop)
> > > > >  > > > > > >
> > > > >  > > > > > >
> > > > >  > > > > > > Thus, the worst-case latency of gLBF and UBS is the
> same.
> > > > >  > > > > > >
> > > > >  > > > > > > Your example give a minimumal latency that may be
> > > expierenced by
> > > > >  > > UBS,
> > > > >  > > > > but
> > > > >  > > > > > >
> > > > >  > > > > > > it is not the worst-case latency. In fact, your
> example
> > > is a simple
> > > > >  > > > > > > topology that only
> > > > >  > > > > > >
> > > > >  > > > > > > contains a line without fan-in, that cause scheduling
> > > delay almost
> > > > >  > > a
> > > > >  > > > > > > minimumal
> > > > >  > > > > > >
> > > > >  > > > > > > value due to no interfering flows.
> > > > >  > > > > > >
> > > > >  > > > > > >
> > > > >  > > > > > > Regards,
> > > > >  > > > > > >
> > > > >  > > > > > > PSF
> > > > >  > > > > > >
> > > > >  > > > > > >
> > > > >  > > > > > >
> > > > >  > > > > > > Original
> > > > >  > > > > > > *From: *JinooJoung <[e-mail]>
> > > > >  > > > > > > *To: *彭少富10053815;
> > > > >  > > > > > > *Cc: *[e-mail] <[e-mail]>;[e-mail] <
> > > > >  > > [e-mail]>;
> > > > >  > > > > > > [e-mail] <
> > > > >  > > [e-mail]>;
> > > > >  > > > > > > *Date: *2023年07月21日 14:10
> > > > >  > > > > > > *Subject: **Re: [Detnet] FYI:
> > > draft-eckert-detnet-glbf-01.txt*
> > > > >  > > > > > > _______________________________________________
> > > > >  > > > > > > detnet mailing list
> > > > >  > > > > > > [e-mail]
> > > > >  > > > > > > https://www.ietf.org/mailman/listinfo/detnet
> > > > >  > > > > > >
> > > > >  > > > > > > Hello Toerless,
> > > > >  > > > > > > I have a comment on your argument.
> > > > >  > > > > > > This is not a question, so you don't have to answer.
> > > > >  > > > > > >
> > > > >  > > > > > > You argued that the gLBF + SP has the same latency
> bound
> > > formula
> > > > >  > > with
> > > > >  > > > > UBS
> > > > >  > > > > > > (equivalently ATS IR + SP).
> > > > >  > > > > > > The IR is not a generalized gLBF, so they do not have
> the
> > > same
> > > > >  > > bound.
> > > > >  > > > > > >
> > > > >  > > > > > > In short, ATS IR is a rate-based shaper so it enjoys
> "Pay
> > > burst
> > > > >  > > only
> > > > >  > > > > once"
> > > > >  > > > > > > property.
> > > > >  > > > > > > gLBF is not. So it pays burst every node.
> > > > >  > > > > > >
> > > > >  > > > > > > Consider a simplest example, where there are only two
> > > identical
> > > > >  > > flows
> > > > >  > > > > > > travelling the same path.
> > > > >  > > > > > > Every node and link in the path are identical.
> > > > >  > > > > > >
> > > > >  > > > > > > Case 1: Just FIFO
> > > > >  > > > > > > Case 2: gLBF + FIFO
> > > > >  > > > > > >
> > > > >  > > > > > > In the first node, two flows' max bursts arrive
> almost at
> > > the same
> > > > >  > > time
> > > > >  > > > > > > but your flow is just a little late.
> > > > >  > > > > > > Then your last packet in the burst (packet of
> interest,
> > > POI)
> > > > >  > > suffers
> > > > >  > > > > > > latency around 2B/R, where B is the burst size and R
> is
> > > the link
> > > > >  > > > > capacity.
> > > > >  > > > > > > This is true for both cases.
> > > > >  > > > > > >
> > > > >  > > > > > > In the next node:
> > > > >  > > > > > > In Case 1, the POI does not see any packet queued. so
> it
> > > is
> > > > >  > > delayed by
> > > > >  > > > > its
> > > > >  > > > > > > own transmission delay.
> > > > >  > > > > > > In Case 2, the burst from the other flow, as well as
> your
> > > own
> > > > >  > > burst,
> > > > >  > > > > > > awaits the POI. So the POI is again delayed around
> 2B/R.
> > > > >  > > > > > >
> > > > >  > > > > > > In the case of UBS, the max bursts are legitimate, so
> the
> > > regulator
> > > > >  > > > > does
> > > > >  > > > > > > not do anything,
> > > > >  > > > > > > and the forwarding behavior is identical to Case 1.
> > > > >  > > > > > >
> > > > >  > > > > > > Best,
> > > > >  > > > > > > Jinoo
> > > > >  > > > > > >
> > > > >  > > > > > > On Fri, Jul 21, 2023 at 10:58 AM <
> [e-mail]>
> > > wrote:
> > > > >  > > > > > >
> > > > >  > > > > > >>
> > > > >  > > > > > >> Hi Toerless,
> > > > >  > > > > > >>
> > > > >  > > > > > >>
> > > > >  > > > > > >> Thanks for your response, and understand your busy
> > > situation.
> > > > >  > > > > > >>
> > > > >  > > > > > >>
> > > > >  > > > > > >> A quick reply is that gLBF is really an interested
> > > proposal,
> > > > >  > > which is
> > > > >  > > > > > >> very
> > > > >  > > > > > >>
> > > > >  > > > > > >> similar to the function of Deadline on-time per hop.
> > > Our views
> > > > >  > > are
> > > > >  > > > > > >>
> > > > >  > > > > > >> consistent on this point. The key beneficial is to
> avoid
> > > burst
> > > > >  > > > > cumulation.
> > > > >  > > > > > >>
> > > > >  > > > > > >>
> > > > >  > > > > > >> The following example originated from the analysis of
> > > deadline
> > > > >  > > on-time
> > > > >  > > > > > >>
> > > > >  > > > > > >> mode. I believe it also makes sense for gLBF. When
> you
> > > have free
> > > > >  > > time,
> > > > >  > > > > > >>
> > > > >  > > > > > >> may verify it. The result may be helpful both for
> gLBF
> > > and
> > > > >  > > deadline
> > > > >  > > > > > >>
> > > > >  > > > > > >> on-time mode.  Note that I didn't question
> mathematical
> > > proof
> > > > >  > > about
> > > > >  > > > > UBS,
> > > > >  > > > > > >>
> > > > >  > > > > > >> which get the worst-case latency based on the
> > > combination of
> > > > >  > > > > > >>
> > > > >  > > > > > >> "IR shaper + SP scheduler".
> > > > >  > > > > > >>
> > > > >  > > > > > >>
> > > > >  > > > > > >> Regards,
> > > > >  > > > > > >>
> > > > >  > > > > > >> PSF
> > > > >  > > > > > >>
> > > > >  > > > > > >>
> > > > >  > > > > > >>
> > > > >  > > > > > >>
> > > > >  > > > > > >> Original
> > > > >  > > > > > >> *From: *ToerlessEckert <[e-mail]>
> > > > >  > > > > > >> *To: *彭少富10053815;
> > > > >  > > > > > >> *Cc: *[e-mail] <[e-mail]>;
> > > [e-mail] <
> > > > >  > > > > > >> [e-mail]>;[e-mail] <
> > > > >  > > > > > >> [e-mail]>;
> > > > >  > > > > > >> *Date: *2023年07月21日 06:07
> > > > >  > > > > > >> *Subject: **Re: [Detnet] FYI:
> > > draft-eckert-detnet-glbf-01..txt*
> > > > >  > > > > > >>
> > > > >  > > > > > >> Thanks folks for the question and discussion, I have
> > > some WG chair
> > > > >  > > > > vultures hovering over me
> > > > >  > > > > > >>
> > > > >  > > > > > >> making sure i prioritize building slides now (the
> worst
> > > one is
> > > > >  > > myself
> > > > >  > > > > ;-), so i will only
> > > > >  > > > > > >> give a brief answer and will get back to it later
> when i
> > > had more
> > > > >  > > > > time.
> > > > >  > > > > > >>
> > > > >  > > > > > >>
> > > > >  > > > > > >> The calculus that i used is from the [UBS] research
> > > paper from
> > > > >  > > > > Johannes Specht, aka: it has
> > > > >  > > > > > >>
> > > > >  > > > > > >> the mathematical proof, full reference is in the gLBF
> > > draft.
> > > > >  > > There is
> > > > >  > > > > another, later proof of the
> > > > >  > > > > > >>
> > > > >  > > > > > >> calculus from Jean Yves Le Boudec in another research
> > > paper which
> > > > >  > > i'd
> > > > >  > > > > have to dig up, and
> > > > >  > > > > > >>
> > > > >  > > > > > >> depending on whom you ask one or the other is easier
> to
> > > read. I
> > > > >  > > am on
> > > > >  > > > > the UBS research paper
> > > > >  > > > > > >>
> > > > >  > > > > > >> side because i have not studied Jean Yves calculus
> book.
> > > But its
> > > > >  > > > > really beautifully simple
> > > > >  > > > > > >>
> > > > >  > > > > > >> that as soon as you think of flows with only
> burst-size
> > > and rate
> > > > >  > > (or
> > > > >  > > > > period) of those burst,
> > > > >  > > > > > >>
> > > > >  > > > > > >> then your delay through the queue is really just the
> sum
> > > of
> > > > >  > > bursts.
> > > > >  > > > > And i just find beauty
> > > > >  > > > > > >>
> > > > >  > > > > > >> in simplicity. And that can not be the full answer to
> > > Jinoo, but i
> > > > >  > > > > first need to read up more
> > > > >  > > > > > >> on his WRR options.
> > > > >  > > > > > >>
> > > > >  > > > > > >> The need for doing per-hop dampening is really as i
> said
> > > from two
> > > > >  > > > > points:
> > > > >  > > > > > >>
> > > > >  > > > > > >>
> > > > >  > > > > > >> 1. Unless we do per-hop dampening, we will not get
> such
> > > a simple
> > > > >  > > > > calculus and equally low latency.
> > > > >  > > > > > >>
> > > > >  > > > > > >> The two validation slides of the gLBF presentation
> show
> > > that one
> > > > >  > > can
> > > > >  > > > > exceed the simple
> > > > >  > > > > > >>
> > > > >  > > > > > >> calculated bounded latency already with as few as 9
> > > flows across
> > > > >  > > a
> > > > >  > > > > single hop and arriving
> > > > >  > > > > > >>
> > > > >  > > > > > >> into one single queue -  unless there is per-hop
> > > dampening (or
> > > > >  > > > > per-flow-shaper).
> > > > >  > > > > > >>
> > > > >  > > > > > >>
> > > > >  > > > > > >> 2. I can not imagine how to safely sell router
> equipment
> > > and build
> > > > >  > > > > out all desirable topologies without
> > > > >  > > > > > >>
> > > > >  > > > > > >> every node is able to do the dampening. And i also
> see
> > > it as the
> > > > >  > > > > right next-generation challenge
> > > > >  > > > > > >>
> > > > >  > > > > > >> and option to make that happen in high speed
> hardware.
> > > > >  > > Specifically
> > > > >  > > > > in metro rings, every big aggregation
> > > > >  > > > > > >>
> > > > >  > > > > > >> ring node has potentially 100 incoming interfaces and
> > > hence can
> > > > >  > > > > create a lot of bursts onto ring interfaces.
> > > > >  > > > > > >>
> > > > >  > > > > > >> Cheers
> > > > >  > > > > > >>    Toerless
> > > > >  > > > > > >>
> > > > >  > > > > > >>
> > > > >  > > > > > >> P.S.: The validation picture in our slides was from
> our
> > > Springer
> > > > >  > > > > Journal article, so
> > > > >  > > > > > >>
> > > > >  > > > > > >> i can not simply put a copy on the Internet now, but
> > > ping me in
> > > > >  > > PM if
> > > > >  > > > > you want an authors copy.
> > > > >  > > > > > >>
> > > > >  > > > > > >> On Wed, Jul 12, 2023 at 11:48:36AM +0800,
> > > peng.shaofu@zte..com.cn
> > > > >  > > > > wrote:
> > > > >  > > > > > >> > Hi Jinoo, Toerless
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > Also thank Toerless for bringing us this interested
> > > draft.
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > For the question Jinoo pointed out, I guess, based
> on
> > > the
> > > > >  > > similar
> > > > >  > > > > analysis
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > of deadline on-time per hop, that even if all flows
> > > departured
> > > > >  > > from
> > > > >  > > > > the damper
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > and arrived at the queueing subsystem at the same
> > > time, each
> > > > >  > > flow
> > > > >  > > > > can still
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > have its worst-case latency, but just consume the
> next
> > > round of
> > > > >  > > > > budget (i.e.,
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > the MAX value mentioned in the document).
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > However, consuming the next round of budget, means
> > > that it
> > > > >  > > relies
> > > > >  > > > > on the
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > downstream node to compensate latency, and may
> result
> > > a jitter
> > > > >  > > with
> > > > >  > > > > MAX
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > (i.e., worst-case latency). Due to this reason,
> > > deadline on-time
> > > > >  > > > > per hop is
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > temperately removed in version-6, waiting for more
> > > strict
> > > > >  > > proof and
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > optimization.
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > Anyway, gLBF can do same things that deadline
> on-time
> > > per hop
> > > > >  > > done.
> > > > >  > > > > The
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > following instuitive exaple is common for these two
> > > solutions.
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > Assuming that at the last node, all received flows
> have
> > > > >  > > expierenced
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > almost 0 queueding delay on the upstream nodes.
> > > Traffic class-8
> > > > >  > > has
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > per hop worst case latency 80 us (just an example,
> > > similar to
> > > > >  > > delay
> > > > >  > > > > level
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > of deadline),  traffic class-7 has 70 us, ... ...,
> > > traffic
> > > > >  > > class-1
> > > > >  > > > > has 10 us.
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > Then, at time T0, traffic class-8 arrived at the
> last
> > > node, it
> > > > >  > > will
> > > > >  > > > > dampen 80us;
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > at time T0+10us, traffic class-7 arrived, it will
> > > dampen 70us,
> > > > >  > > and
> > > > >  > > > > so on. At
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > T0+80us, all traffic class flows will departure
> from
> > > the damper,
> > > > >  > > > > and send to
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > the same outgoing port. So, an observed packet may
> > > expierence
> > > > >  > > > > another
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > round of worst case lantecy if other higher
> priority
> > > flows
> > > > >  > > > > existing, or
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > expierence best case latency (almost 0) if other
> > > higher priority
> > > > >  > > > > flows not
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > existing. That is, a jitter with value of worst
> case
> > > latency
> > > > >  > > still
> > > > >  > > > > exists.
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > Regards,
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > PSF
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > Original
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > From: JinooJoung <[e-mail]>
> > > > >  > > > > > >> > To: Toerless Eckert <[e-mail]>;
> > > > >  > > > > > >> > Cc: [e-mail] <[e-mail]>;
> > > > >  > > > > [e-mail]
> > > > >  > > > > > >>  <[e-mail]>;
> > > > >  > > > > > >> > Date: 2023年07月09日 09:39
> > > > >  > > > > > >> > Subject: Re: [Detnet] FYI:
> > > draft-eckert-detnet-glbf-01.txt
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > _______________________________________________
> > > > >  > > > > > >> > detnet mailing list
> > > > >  > > > > > >> > [e-mail]
> > > > >  > > > > > >> > https://www.ietf.org/mailman/listinfo/detnet
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > Dear Toerless; thanks for the draft.
> > > > >  > > > > > >>
> > > > >  > > > > > >> > gLBF is an interesting approach, similar in
> concept to
> > > the
> > > > >  > > Buffered
> > > > >  > > > > Network (BN) I have introduced in the ADN Framework
> document.
> > > > >  > > > > > >>
> > > > >  > > > > > >> > The difference seems that the BN buffers only once
> at
> > > the
> > > > >  > > network
> > > > >  > > > > boundary, while gLBF buffers at every node.
> > > > >  > > > > > >>
> > > > >  > > > > > >> > Therefore in the BN, a buffer handles only a few
> > > flows, while in
> > > > >  > > > > the gLBF a buffer needs to face millions of flows.
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > The implementation complexity should be addressed
> in
> > > the future
> > > > >  > > > > draft, I think.
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > I have a quick question below.
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >    +------------------------+
> > > +------------------------+
> > > > >  > > > > > >> >    | Node A                  |        | Node B
> > > > >  > >   |
> > > > >  > > > > > >> >    |   +-+   +-+   +-+      |        |   +-+   +-+
> > >  +-+      |
> > > > >  > > > > > >> >    |-x-|D|-y-|F|---|Q|----z -|------
> > > |-x-|D|-y-|F|---|Q|----z- |
> > > > >  > > > > > >> >    |   +-+   +-+   +-+      | Link |   +-+   +-+
> > >  +-+      |
> > > > >  > > > > > >> >    +------------------------+
> > > +------------------------+
> > > > >  > > > > > >> >            |<- A/B in-time latency ->|
> > > > >  > > > > > >> >            |<--A/B on-time latency ------->|
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >                Figure 3: Forwarding with Damper and
> > > measuring
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > In Figure 3, how can F and Q guarantee the nodal
> > > latency below
> > > > >  > > MAX?
> > > > >  > > > > > >>
> > > > >  > > > > > >> > Does the gLBF provide the same latency bound as
> that
> > > of UBS, as
> > > > >  > > it
> > > > >  > > > > is argued?
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > In UBS, an interleaved regulator (IR) works as the
> > > damper D in
> > > > >  > > the
> > > > >  > > > > gLBF.
> > > > >  > > > > > >>
> > > > >  > > > > > >> > IR is essentially a FIFO, whose HoQ packet is
> examined
> > > and
> > > > >  > > leaves
> > > > >  > > > > if eligible.
> > > > >  > > > > > >>
> > > > >  > > > > > >> > A packet's eligible time can be earlier than the
> time
> > > that it
> > > > >  > > > > became the HoQ.
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > However, in gLBF, a packet has a precise moment
> that
> > > it needs
> > > > >  > > to be
> > > > >  > > > > forwarded from D.
> > > > >  > > > > > >> > (Therefore, UBS is not a generalized gLBF.)
> > > > >  > > > > > >>
> > > > >  > > > > > >> > In the worst case, all the flows may want to send
> the
> > > packets
> > > > >  > > from
> > > > >  > > > > D to F at the same time.
> > > > >  > > > > > >>
> > > > >  > > > > > >> > If it can be implemented as such, bursts may
> > > accumulate, and the
> > > > >  > > > > latency cannot be guaranteed.
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> > If it cannot be implemented that way, you may
> > > introduce another
> > > > >  > > > > type of delay.
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > Don't you need an additional mechanism for latency
> > > guarantee?
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > Thanks a lot in advance, I support this draft.
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > Best,
> > > > >  > > > > > >> > Jinoo
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > On Sat, Jul 8, 2023 at 12:05 AM Toerless Eckert <
> > > [e-mail]>
> > > > >  > > > > wrote:
> > > > >  > > > > > >> >
> > > > >  > > > > > >> > Dear DetNet WG,
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  FYI on a newly posted  bounded latency
> > > method/proposal draft
> > > > >  > > that
> > > > >  > > > > we call gLBF.
> > > > >  > > > > > >> >  (guaranteed Latency Based Forwarding).
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  gLBF, as compared to TCQF and CSQF is proposed
> from
> > > our side
> > > > >  > > to be
> > > > >  > > > > a more long-term
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  solution, because it has not been validated with
> > > high-speed
> > > > >  > > > > forwarding hardware and requires
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  new network header information for the damper
> value,
> > > whereas
> > > > >  > > > > TCQF/CSQF of course can operate
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  without new headers, have proven high-speed
> > > implementations PoC
> > > > >  > > > > and are therefore really
> > > > >  > > > > > >> >  ready for adoption now.
> > > > >  > > > > > >> >
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  gLBF is a specific variant of the damper idea
> that is
> > > meant to
> > > > >  > > be
> > > > >  > > > > compatible with the
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  TSN-ATS latency calculus so that it can use the
> same
> > > > >  > > > > controller-plane/path-computation
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  algorithms and implementations one would use for
> > > TSN-ATS. It
> > > > >  > > also
> > > > >  > > > > allows to eliminate
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  the need for hop-by-hop clock synchronization and
> (we
> > > hope)
> > > > >  > > should
> > > > >  > > > > be well implementable
> > > > >  > > > > > >> >  in high-speed hardware.
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >  Any feedback welcome.
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >  Cheers
> > > > >  > > > > > >> >      Toerless
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >  In-Reply-To: <
> > > > >  > > > > [e-mail]>
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >  On Fri, Jul 07, 2023 at 07:37:56AM -0700,
> > > > >  > > [e-mail]
> > > > >  > > > > > >>  wrote:
> > > > >  > > > > > >> >  >
> > > > >  > > > > > >> >  > A new version of I-D,
> > > draft-eckert-detnet-glbf-01.txt
> > > > >  > > > > > >> >  > has been successfully submitted by Toerless
> Eckert
> > > and
> > > > >  > > posted to
> > > > >  > > > > the
> > > > >  > > > > > >> >  > IETF repository.
> > > > >  > > > > > >> >  >
> > > > >  > > > > > >> >  > Name:         draft-eckert-detnet-glbf
> > > > >  > > > > > >> >  > Revision:     01
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  > Title:                Deterministic Networking
> > > (DetNet) Data
> > > > >  > > > > Plane - guaranteed Latency Based Forwarding (gLBF) for
> > > bounded latency
> > > > >  > > with
> > > > >  > > > > low jitter and asynchronous forwarding in Deterministic
> > > Networks
> > > > >  > > > > > >> >  > Document date:        2023-07-07
> > > > >  > > > > > >> >  > Group:                Individual Submission
> > > > >  > > > > > >> >  > Pages:                39
> > > > >  > > > > > >> >  > URL:
> > > > >  > > > > > >>
> > > https://www.ietf.org/archive/id/draft-eckert-detnet-glbf-01.txt
> > > > >  > > > > > >> >  > Status:
> > > > >  > > > > > >>
> > > https://datatracker.ietf.org/doc/draft-eckert-detnet-glbf/
> > > > >  > > > > > >> >  > Htmlized:
> > > > >  > > > > > >>
> > > https://datatracker.ietf.org/doc/html/draft-eckert-detnet-glbf
> > > > >  > > > > > >> >  > Diff:
> > > > >  > > > > > >>
> > > > >  > >
> > > https://author-tools.ietf.org/iddiff?url2=draft-eckert-detnet-glbf-01
> > > > >  > > > > > >> >  >
> > > > >  > > > > > >> >  > Abstract:
> > > > >  > > > > > >> >  >    This memo proposes a mechanism called
> > > "guaranteed Latency
> > > > >  > > > > Based
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  >    Forwarding" (gLBF) as part of DetNet for
> > > hop-by-hop packet
> > > > >  > > > > forwarding
> > > > >  > > > > > >> >  >    with per-hop deterministically bounded
> latency
> > > and minimal
> > > > >  > > > > jitter.
> > > > >  > > > > > >> >  >
> > > > >  > > > > > >> >  >    gLBF is intended to be useful across a wide
> > > range of
> > > > >  > > networks
> > > > >  > > > > and
> > > > >  > > > > > >> >  >    applications with need for high-precision
> > > deterministic
> > > > >  > > > > networking
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  >    services, including in-car networks or
> networks
> > > used for
> > > > >  > > > > industrial
> > > > >  > > > > > >> >  >    automation across on factory floors, all the
> way
> > > to
> > > > >  > > ++100Gbps
> > > > >  > > > > > >> >  >    country-wide networks.
> > > > >  > > > > > >> >  >
> > > > >  > > > > > >> >  >    Contrary to other mechanisms, gLBF does not
> > > require
> > > > >  > > network
> > > > >  > > > > wide
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  >    clock synchronization, nor does it need to
> > > maintain
> > > > >  > > per-flow
> > > > >  > > > > state at
> > > > >  > > > > > >> >  >    network nodes, avoiding drawbacks of other
> known
> > > methods
> > > > >  > > while
> > > > >  > > > > > >> >  >    leveraging their advantages.
> > > > >  > > > > > >> >  >
> > > > >  > > > > > >> >  >    Specifically, gLBF uses the queuing model and
> > > calculus of
> > > > >  > > > > Urgency
> > > > >  > > > > > >> >  >    Based Scheduling (UBS, [UBS]), which is used
> by
> > > TSN
> > > > >  > > > > Asynchronous
> > > > >  > > > > > >> >  >    Traffic Shaping [TSN-ATS]. gLBF is intended
> to
> > > be a
> > > > >  > > plug-in
> > > > >  > > > > > >> >  >    replacement for TSN-ATN or as a parallel
> > > mechanism beside
> > > > >  > > > > TSN-ATS
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  >    because it allows to keeping the same
> > > controller-plane
> > > > >  > > design
> > > > >  > > > > which
> > > > >  > > > > > >> >  >    is selecting paths for TSN-ATS, sizing
> TSN-ATS
> > > queues,
> > > > >  > > > > calculating
> > > > >  > > > > > >> >  >    latencies and admitting flows to calculated
> > > paths for
> > > > >  > > > > calculated
> > > > >  > > > > > >> >  >    latencies.
> > > > >  > > > > > >> >  >
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  >    In addition to reducing the jitter compared
> to
> > > TSN-ATS by
> > > > >  > > > > additional
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  >    buffering (dampening) in the network, gLBF
> also
> > > eliminates
> > > > >  > > > > the need
> > > > >  > > > > > >> >  >    for per-flow, per-hop state maintenance
> required
> > > by
> > > > >  > > TSN-ATS.
> > > > >  > > > > This
> > > > >  > > > > > >> >  >    avoids the need to signal per-flow state to
> > > every hop
> > > > >  > > from the
> > > > >  > > > > > >> >  >    controller-plane and associated scaling
> > > problems.  It also
> > > > >  > > > > reduces
> > > > >  > > > > > >> >  >    implementation cost for high-speed networking
> > > hardware
> > > > >  > > due to
> > > > >  > > > > the
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  >    avoidance of additional high-speed speed
> > > read/write memory
> > > > >  > > > > access to
> > > > >  > > > > > >> >  >    retrieve, process and update per-flow state
> > > variables for
> > > > >  > > a
> > > > >  > > > > large
> > > > >  > > > > > >> >  >    number of flows.
> > > > >  > > > > > >> >  >
> > > > >  > > > > > >>
> > > > >  > > > > > >> >  >
> > > > >  > > > > > >> >  >
> > > > >  > > > > > >> >  >
> > > > >  > > > > > >> >  > The IETF Secretariat
> > > > >  > > > > > >> >
> > > > >  > > > > > >> >  _______________________________________________
> > > > >  > > > > > >> >  detnet mailing list
> > > > >  > > > > > >> >  [e-mail]
> > > > >  > > > > > >> >  https://www.ietf.org/mailman/listinfo/detnet
> > > > >  > > > > > >>
> > > > >  > > > > > >>
> > > > >  > > > > > >>
> > > > >  > > > > > >> --
> > > > >  > > > > > >> ---
> > > > >  > > > > > >> [e-mail]
> > > > >  > > > > > >>
> > > > >  > > > > > >> _______________________________________________
> > > > >  > > > > > >> detnet mailing list
> > > > >  > > > > > >> [e-mail]
> > > > >  > > > > > >> https://www.ietf.org/mailman/listinfo/detnet
> > > > >  > > > > > >>
> > > > >  > > > > > >>
> > > > >  > > > > > >>
> > > > >  > > > > > >
> > > > >  > > > >
> > > > >  > > > > --
> > > > >  > > > > ---
> > > > >  > > > > [e-mail]
> > > > >  > > > >
> > > > >  > >
> > > > >  > > --
> > > > >  > > ---
> > > > >  > > [e-mail]
> > > > >  > >
> > > > >
> > > > >  --
> > > > >  ---
> > > > >  [e-mail]
> > > >
> > > >
> > > >
> > > > --
> > > > ---
> > > > [e-mail]
> > >
> > >
> > >
> > > --
> > > ---
> > > [e-mail]
> > >
>
> --
> ---
> [e-mail]
>