Re: [bmwg] Martin Duke's Discuss on draft-ietf-bmwg-b2b-frame-03: (with DISCUSS and COMMENT)

["MORTON, ALFRED C (AL)" <[e-mail]>]

Hi Scott,

Please see my replies below, marked [acm], with a couple of questions.
I hope I'm not missing something obvious, so trying to be very clear in all replies!
But I could be overlooking something, and if so I will learn something very soon...

> -----Original Message-----
> From: Scott O. Bradner [mailto:[e-mail]]
> Sent: Tuesday, December 15, 2020 9:06 AM
> To: MORTON, ALFRED C (AL) <[e-mail]>
> Cc: Martin Duke <[e-mail]>; The IESG <[e-mail]>;
> [e-mail]; [e-mail]; [e-mail]
> Subject: Re: [bmwg] Martin Duke's Discuss on draft-ietf-bmwg-b2b-frame-03:
> (with DISCUSS and COMMENT)
> 
> I basically understood that but it seemed to me that using a fixed (2
> second) extra time, which is unrelated
> to whatever time that the burst might have taken to be sent seemed risky
> since I could
> imagine cases where the play out speed was less than the receive speed

[acm] 
I guess I don't understand your example, where (buffer?) play-out speed plays a role in the results, and how play-out speed could be less than the receive speed in the multi-second time scale of the buffer-bloat example. I think (buffer) play-out speed and receive speed should be nominally the same.

Although RFC 2544 Throughput definition is based on offered load delivered loss-free to the receiver, we use it here as the best approximation available for packet header processing rate (equal to playout rate from the buffer?), egress from the DUT, and the speed at which the test system receives packets. 

So, in our diagram from the memo:

                        |------------ DUT --------|
   Generator -> Ingress -> Buffer -> HeaderProc -> Egress -> Receiver

Is your play-out speed the HeaderProc speed, or Egress speed?

And how can the (buffer) play-out speed be less than the speed at a subsequent interface (for very long)?

help me understand the mechanics I'm overlooking, my friend!

> 
> but if you are convinced that the 2 seconds extra time would cover all
> possible cases then go to it
[acm] 

Well, we say "at least 2 seconds" and allow for customization if necessary.

As you know, I've conducted LOTS of production network testing, where we have used static waiting times to distinguish packet loss from long delay, and prescribed the same in IPPM RFCs, etc. A static waiting time "Tmax" has served us well.

Here, we have the added stability of the Isolated Test Environment (ITE, as Kevin Dubray called it), and the three time-component definition of trial duration, where we wait 2 seconds after the last packet on seen egress (it is more like a cool-down interval between trials). I think all the adaptation we need comes from explicit recognition that the time for the Test Receiver to receive the entire burst depends on the buffer size, the DUT header processing rate, the actual interface speed, etc. IOW, all the unknown variables.

Thanks again for your time, Scott!
Al

> 
> Scott
> 
> 
> > On Dec 14, 2020, at 7:24 PM, MORTON, ALFRED C (AL)
> <[e-mail]> wrote:
> >
> > Hi Scott, thanks for helping with this discussion.
> >
> > I'm trying to formulate adaptive extra time based on the time it takes
> to *receive* the burst, with the additional "at least 2 seconds"  waiting
> time to be sure we received all the packets that might arrive.  Let me try
> drawing the timeline that's in my mind, and I'll use a buffer-bloat case
> example of a 1 second buffer (which dominates all other buffers in the
> DUT).
> >
> > One of the key contributions of this memo is recognizing that the buffer
> is being emptied while the burst of back-to-back frames is simultaneously
> trying to fill the buffer.
> >
> > Assume that the RFC 2544 Throughput is only half of the back-to-back
> frame rate for the frame size used.
> >
> > From the draft:
> >   4.  A helpful concept is the buffer filling rate, which is the
> >       difference between the Max Theoretical Frame Rate (ingress) and
> >       the Measured Throughput (HeaderProc on egress).  If the actual
> >       buffer size in frames was known, the time to fill the buffer
> >       during a measurement can be calculated using the filling rate as
> >       a check on measurements.  However, the Buffer in the model
> >       represents many buffers of different sizes in the DUT data path.
> >
> > So (danger: calculating while typing and drawing!), a 1 second burst of
> B2B frames only raises the occupation buffer to 50%, and another second of
> transmission is needed before reaching 100% occupation.
> >
> > Trial
> > Time, sec: 0          1          2          3         4          5
> 6
> >
> > Sender:    |==========|==========|
> > Receiver:  |= = = = = |= = = = = |= = = = = |= = = = =|
> > Waiting Time                                          |          |
> |
> >
> Trial
> >
> Ends
> >
> > In the ideal example timeline above, the back-to-back burst stopped
> exactly when the buffer reached capacity, so there is no loss. The buffer
> fill rate is half the back-to-back rate. Also, it takes 2 seconds to
> deplete the buffer and for frames to stop arriving at the receiver. Only
> then do we start the 2 second waiting time to ensure no more frames will
> arrive!
> >
> > While we're here, let's look at a calculation from the memo:
> >
> >   Corrected DUT Buffer Time =
> >                          /                                         \
> >           Implied DUT    |Implied DUT       Measured Throughput    |
> >        =  Buffer Time -  |Buffer Time * -------------------------- |
> >                          |              Max Theoretical Frame Rate |
> >                          \                                         /
> >        =  2 - [ 2 * 0.5 ] seconds
> >        =  1 second
> >
> > and we avoid the error of calculating buffer time based on the sender's
> burst duration alone.
> >
> > hope this helps,
> > Al
> >
> >
> >> -----Original Message-----
> >> From: Scott O. Bradner [mailto:[e-mail]]
> >> Sent: Saturday, December 12, 2020 5:18 PM
> >> To: MORTON, ALFRED C (AL) <[e-mail]>
> >> Cc: Martin Duke <[e-mail]>; The IESG <[e-mail]>;
> >> [e-mail]; [e-mail]; [e-mail]
> >> Subject: Re: [bmwg] Martin Duke's Discuss on draft-ietf-bmwg-b2b-frame-
> 03:
> >> (with DISCUSS and COMMENT)
> >>
> >> this would seem to work if 2 seconds is significantly longer than it
> takes
> >> to send the burst - but if it takes 2 second to send the burst
> >> then 2 seconds extra buffer could easily lose packets - seems to me
> that
> >> he extra time should be related to the time it takes to send the burst
> >>
> >> e.g 50% of the burst time but not less than 2 seconds
> >>
> >> Scott
> >>
> >>
> >>> On Dec 12, 2020, at 10:18 AM, MORTON, ALFRED C (AL)
> >> <[e-mail]> wrote:
> >>>
> >>> Hi Martin, thanks for your review and comment,
> >>> please see my reply, [acm] below,
> >>> Al
> >>>
> >>>> -----Original Message-----
> >>> ...
> >>>>
> >>>> ---------------------------------------------------------------------
> -
> >>>> DISCUSS:
> >>>> ---------------------------------------------------------------------
> -
> >>>>
> >>>> Thank you for engaging with the TSVART review. Despite the
> wordsmithing
> >> that
> >>>> has gone on, I am not sure that we have captured the correct text.
> >>>>
> >>>> The proposed change is:
> >>>>> I clarified:
> >>>>> The duration of the trial MUST include at least 2 seconds in
> addition
> >> to the time
> >>>>> required to send and receive each burst of frames, to ensure that
> DUT
> >> buffers to deplete.
> >>>>> and I'll add:
> >>>>> The upper search limit for the time to send each burst MUST be
> >> configurable as
> >>>>> high as 30 seconds (buffer time results
> >>>>> reported at the configured upper limit are likely invalid, and the
> >> test MUST
> >>>>> be repeated with a higher search limit).
> >>>>
> >>>> But IIUC it's the additional time that needs to scale up.
> >>> [acm]
> >>>
> >>> In the revised text where David and I reached agreement, we identified
> 3
> >> time components of the trial duration, making the duration variable: no
> >> longer static and at "at least 2 seconds".
> >>>
> >>> 1. the time to send the burst of frames (at the back-to-back rate),
> >> determined by the search algorithm
> >>> 2. the time to receive the transferred burst of frames (at the RFC2544
> >> Throughput rate), possibly truncated by buffer overflow, but certainly
> >> including the latency of the DUT with or without buffer-bloat
> >>> 3. at least 2 seconds in addition to the time to receive the burst
> (2.),
> >> to ensure that DUT buffers have depleted.
> >>>
> >>> So, both components 1. and 2. are variables, and the burst receive
> time
> >> component (2.) compensates for large buffers, non-back-to-back burst
> >> egress, and anything else that contributes to DUT latency. The final
> "at
> >> least 2 seconds" is simply about making sure the trial is really over
> >> before moving on in an automated test - we won't make an error if
> frames
> >> trickle-out very late for some unfortunate reason.
> >>>
> >>>> A layman's reading of
> >>>> the document, IMO, suggests that the burst length has a binary search
> >> but the 2
> >>>> seconds of waiting can be fixed.
> >>> [acm]
> >>> Yes, that's right, plus all the other factors above.
> >>>
> >>> So, let's try this, but I'm trying not to extend or complicate the
> >> buffer time << 2 seconds testing for the sake of the buffer-bloat case:
> >>>
> >>> -=-=-=-=-=-=-
> >>>
> >>> The duration of the trial includes three REQUIRED components:
> >>>
> >>> 1. the time to send the burst of frames (at the back-to-back rate),
> >> determined by the search algorithm
> >>> 2. the time to receive the transferred burst of frames (at the RFC2544
> >> Throughput rate), possibly truncated by buffer overflow, and certainly
> >> including the latency of the DUT
> >>> 3. at least 2 seconds not overlapping the time to receive the burst
> >> (2.), to ensure that DUT buffers have depleted.
> >>>
> >>> The upper search limit for the time to send each burst MUST be
> >> configurable as high as 30 seconds (buffer time results reported at or
> >> near the configured upper limit are likely invalid, and the test MUST
> be
> >> repeated with a higher search limit).
> >>>
> >>> -=-=-=-=-=-=-=-=-
> >>>
> >>> Does that wording do it?
> >>>
> >>>>
> >>>> ---------------------------------------------------------------------
> -
> >>>> COMMENT:
> >>>> ---------------------------------------------------------------------
> -
> >>>>
> >>>> Other than that, this a well-written document. Thanks!
> >>> [acm]
> >>> Thank you!
> >>>
> >>>>
> >>>>
> >>>
> >>> _______________________________________________
> >>> bmwg mailing list
> >>> [e-mail]
> >>>
> >>
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/bmwg__;!
> >> !BhdT!1uRJDJBUadSunB4ZCkgOTzg3ZssPtiufcyrsTcxEc1F67df5q4YNUa9IYHacnsA$
> >