Re: [Tsv-art] Tsvart last call review of draft-ietf-bmwg-b2b-frame-03

["Black, David" <[e-mail]>]

Al,

> [acm] I agree with your observation that there are cases where trial duration
> should be increased to accommodate the encountered in the DUT, but not as a
> mandate for all testing. I have four factors in mind:
> 1. Some of the virtual network DUTs we are testing now have very small buffers,
> and the B2B stream of frames is quite short -- less than 2000 frames@10GE in
> some cases -- so 2 seconds fully sufficient.
> 2. The trial duration is a factor in total test duration, where each trial is one step in
> the Binary Search. We need to manage the tension between the time needed to
> reach a search result and confidence that we have depleted the queues.
> 3. The RFC 2544 Latency benchmark will tell us if bufferbloat is present.
> 4. The current text says "at least 2 seconds".
> 
> So I suggest adding the following text:
> 
>     The duration of the trial MUST be at least 2 seconds, to allow DUT
>     buffers to deplete. When RFC2544 Latency measurements indicate that
>     large buffers are present in the DUT, the trial duration SHOULD be
>     increased to ensure that buffer depletion takes place, without unduly
>     extending the total test time.

The overall approach of collecting evidence that there is a problem before increasing the 2 second minimum duration is fine, but the details appear to need more attention:

1. Obtaining the RFC 2544 Latency measurements would need to be added to Section 4 (Prerequisites) of this draft to ensure that the buffer size information is available.

2. I did not see any requirements in the RFC 2544 Latency test (Section 26.2) to deplete buffers.  Did I miss something?

3. I would think that the trial duration MUST be increased, not just SHOULD be increased if there is evidence of large buffer size, as buffer depletion appears to be a necessary characteristic of this B2B measurement.

It also looks like the link to the entire slide deck didn't make it into my original review correctly - that slide deck is at: http://www.taht.net/~d/lca_tcp3.odp .  In addition to slide 6 from this slide deck (Figure 1 in the APNIC blog), slide 14 is also relevant to this discussion.

Thanks, --David

> -----Original Message-----
> From: MORTON, ALFRED C (AL) <[e-mail]>
> Sent: Tuesday, November 24, 2020 11:11 AM
> To: Black, David; [e-mail]
> Cc: [e-mail]; [e-mail]; [e-mail]
> Subject: RE: Tsvart last call review of draft-ietf-bmwg-b2b-frame-03
> 
> 
> [EXTERNAL EMAIL]
> 
> Hi David, Thanks for your review and comment!
> 
> Please see a proposed resolution below, [acm] Al
> 
> > -----Original Message-----
> > From: David Black via Datatracker [mailto:[e-mail]]
> > Sent: Tuesday, November 24, 2020 10:27 AM
> > To: [e-mail]
> > Cc: [e-mail]; [e-mail]; last-
> > [e-mail]
> > Subject: Tsvart last call review of draft-ietf-bmwg-b2b-frame-03
> >
> > Reviewer: David Black
> > Review result: Ready with Issues
> >
> ...
> >
> > This draft updates the back-to-back frame testing procedure in RFC
> > 2544 to take account of experience.
> >
> > The draft is in good shape, with one notable exception in Section 5.2:
> >
> >    The duration of the trial MUST be at least 2 seconds, to allow DUT
> >    buffers to deplete.
> >
> > That duration of 2 seconds has been carried forward from RFC 2544
> > without change.  A 2 second duration may have been sufficient to
> > deplete buffers in 1999, but that is no longer reliably the case.  For
> > example, on-site measurement of the network for the 2020 Linux
> > Conference in Australia indicated a at least 1.6 seconds of buffering,
> > as indicated by Figure 1 at
> > https://urldefense.com/v3/__https://blog.apnic.net/2020/01/22/bufferbloat-may-be-solved-but-its-not-over-yet/__;!!BhdT!wOuQE0NajXs4dT7tdIMVQU5FFpb0JiU0-yK2DOVVn0ecoYjf7mFEABLmlwDk$ ,
> > which is slide 6 in from the complete slide deck at:
> > https://urldefense.com/v3/__https://blog.apnic.net/2020/01/22/bufferbloat-may-be-solved-but-its-not-over-yet/__;!!BhdT!wOuQE0NajXs4dT7tdIMVQU5FFpb0JiU0-yK2DOVVn0ecoYjf7mFEABLmlwDk$
> > .  Experience with bufferbloat suggests that one network device was
> > primarily responsible.  Also, see slide 14 in that slide deck.
> >
> > That 1.6 seconds measured on an actual network is entirely too close
> > to 2 seconds for confidence that buffers will be depleted in any tested device.
> > Hence, the 2 second minimum duration ought to be increased by at least
> > a factor of 10.  I'd suggest changing it to 30 seconds or 60 seconds
> > as convenient round numbers, and providing the rationale that
> > increased buffering in WiFi devices, e.g., home "routers," as
> > indicated by experience with bufferbloat measurements, is the reason for the
> increased duration.
> >
> [acm]
> [acm] I agree with your observation that there are cases where trial duration
> should be increased to accommodate the encountered in the DUT, but not as a
> mandate for all testing. I have four factors in mind:
> 1. Some of the virtual network DUTs we are testing now have very small buffers,
> and the B2B stream of frames is quite short -- less than 2000 frames@10GE in
> some cases -- so 2 seconds fully sufficient.
> 2. The trial duration is a factor in total test duration, where each trial is one step in
> the Binary Search. We need to manage the tension between the time needed to
> reach a search result and confidence that we have depleted the queues.
> 3. The RFC 2544 Latency benchmark will tell us if bufferbloat is present.
> 4. The current text says "at least 2 seconds".
> 
> So I suggest adding the following text:
> 
>     The duration of the trial MUST be at least 2 seconds, to allow DUT
>     buffers to deplete. When RFC2544 Latency measurements indicate that
>     large buffers are present in the DUT, the trial duration SHOULD be
>     increased to ensure that buffer depletion takes place, without unduly
>     extending the total test time.
> 
> I hope this suggestion resolves your issue; thanks for highlighting it!
> Al