Re: [bmwg] AD Review of draft-ietf-bmwg-b2b-frame

["MORTON, ALFRED C (AL)" <[e-mail]>]

Hi Warren,

Thanks for your comments on the b2b-frame draft. Please see replies below, [acm].

Al

> -----Original Message-----
> From: Warren Kumari [mailto:[e-mail]]
> Sent: Monday, November 9, 2020 10:22 AM
> To: [e-mail]; [e-mail]
> Subject: AD Review of draft-ietf-bmwg-b2b-frame
> 
> Hi there ADs and WG.
> 
> Thank you for this document, I found it clear, useful, and an easy read.
> I do have a few comments/nits that addressing now should help the IETF
> LC and iesg eval go more smoothly. Some of the plenary discussions at
> IETF108 brought to light a desire from the community that nits and
> similar be better addressed before LC/eval/the RFC Ed, and so I'm
> trying to be especially diligent about noting them.
> 
> Please SHOUT loudly once you've had a chance to address these. We are
> during a posting-blackout, but seeing as the document is already in
> "Publication Requested" state, and these are nits/readability, I'm
> happy to approve posting a new version anyway...

[acm]
<SHOUTING>
I will send a separate message about this => The .txt and .xml files are ready!
</SHOUTING>

> 
> Comments:
> 1: RFC2119 was updated/replaced with RFC8174. Replacing RFC2119 with
> RFC8174 in:
> "Thus, the requirements presented in this memo are expressed in
> [RFC2119] terms,... " would probably be a good idea (or explain in the
> document why not). This will require some text massaging in the prior
> sentence[0].
[acm] 
Yes, I have updated both sentences to reflect the arrival of RFC 8174 and its clarifications.

Please see the reply to [0] as well.

> 
> 2: "Fortunately, the Open Platform for Network Function Virtualization
> (OPNFV) VSPERF project’s Continuous Integration (CI) testing ..."
> Is there a link/reference that we can insert for this?
[acm] 
Yes, there's a wiki page:
https://wiki.opnfv.org/display/vsperf/VSPERF+CI


> 
> 3: "Back-to-back Frame Benchmark was very consistent for some fixed
> frame sizes, and somewhat variable for others."
> s/for others/for other frame sizes/ ? It might just be me, but this
> took me a few tries to understand. This might be a nit...
[acm] 
Yes, you are correct, I made the substitution above.

> 
> 4: "It’s important that the buffering time was used in this analysis."
> This reads like a fragment -- *why* is it important? Perhaps "it's
> important to note that...", but I must admit I still don't quite
> understand what this sentence is adding - that 30 seconds is
> unrealistically long and someone should have asked "...what, what?!"?
[acm] 
Yes, I noticed this difficult sentence just recently while putting the memo to use :-) in a current study. I think some of it may have arrived by way of welcome comments/ideas, but I didn't do a good job of fitting all together at the first attempt: 

It was important that the buffering time calculations were part the referenced testing and analysis [VSPERF-b2b], because the calculated buffer times of 30 seconds for some frame sizes were clearly wrong or highly suspect.


> 
>  5: Similarly, I'm having a hard time parsing "using results" in:
> "It was found that the actual extent of buffer time in the DUT could
> be estimated using results to measure the longest burst"
> Using *what* results?
> Perhaps:
> "It was found that the actual extent of buffer time in the DUT could
> be estimated by measuring the longest burst in frames without loss and
> results from the Throughput tests conducted according to Section 26.1
> of [RFC2544]." ? But I actually have no idea, and so am just
> guessing...
[acm]
Sorry about that.  This version may help:

It was found that a better estimate of the DUT buffer time could be calculated using measurements of both the longest burst in frames without loss and results from the Throughput tests conducted according to Section 26.1 of [RFC2544]. 


> 
> 6: Section 7 - Security Considerations
> I *think* that it might be useful to add:
> "[RFC Editor, please remove before publication. This is the default
> security considerations text which is included in BMWG documents - see
> rfc6815 for more info.] "
> or similar. This may help having to explain this again....
[acm]
Well, if we remove the entire section at publication, then there will be no requirements like:

The benchmarking network topology will be an independent test setup and MUST NOT be connected to devices that may forward the test traffic into a production network, or misroute traffic to the test management network.

which the SEC-DIR reviewers consider a big deal, and rightfully so. BMWG's test conditions often require link utilization of 100% for extended periods of time, and the traffic should not appear in the Internet. In fact, BMWG's dedicated IPv4 and IPv6 address space assignments allow production network devices to discard matching packets when received.

I think we leave this section as-is.


> 
> 
> Nits:
> 1: "The IETF’s fundamental Benchmarking Methodologies are defined
> in[RFC2544],..."
> There is a missing space before the reference.
[acm] 
thanks - my "old" version of XMLMind made it appear as though there was a space ...

> 
> 
> 2: "In particular, analysis of the results indicates that buffers size
> matters when compensating for disruptions in the software packet
> processor,  "
> s/buffers/buffer/ (or "that the size of buffers matters", or similar)
[acm]

I went with:
In particular, analysis of the results indicates that buffer size matters when compensating for interruptions of software packet processing,...

> 
> 3: "The tests described in this memo address the cases where the
> maximum frame rate"
> s/cases where/cases in which/? (readability)
[acm]

yeah, that sounds better, thanks.

> 
> 4: The [RFC8239] methods measure
>    buffer latency directly with traffic on multiple ingress ports that
>    overload an egress port on the Device Under Test (DUT), and are not ...
> s/(DUT),./(DUT)/ spurious comma.
[acm] 
ok

> 
> 
> 5: To summarize, there are several reasons that
>    devices on a network produce bursts of frames at the minimum allowed
>    spacing, and it is therefore worthwhile to understand the Device
> [O]  spacing, and it is therefore worthwhile
> [P] spacing; and it is, therefore, worthwhile
[acm]
ok

> 
> 
> 6:  3. Calculation of the extent of buffer time in the DUT helped to
>        explain the results observed with all frame sizes (for example,
>        some frame sizes cannot exceed the frame header processing rate
>        of the DUT and therefore no buffering occurs, therefore the
> [O] occurs, therefore
> [P] occurs; therefore,
[acm]
I suddenly saw that "therefore" appears twice in close proximity.
Here's what I wrote:
... tests with some frame sizes cannot exceed the frame header processing rate of the DUT and thus no buffering occurs; therefore, the results depended on the test equipment and not the DUT).

> 
> 7: The presentation of OPNFV VSPERF evaluation and development of
>    enhanced search alogorithms [VSPERF-BSLV] was discussed at IETF-102.
> 
> [O] alogorithms
> [P] algorithms
[acm]
Sorry about these, it's also a disadvantage that my version of XMLMind lacks spell-check. 
 
> 
>    The enhancements are intended to compensate for transient inerrrupts
> 
> [O] inerrrupts
> [P] interrupts
[acm]
ok

> 
> 8: Therefore, sources of packet loss that are un-related
> 
> [O] un-related
> [P] unrelated
[acm]
ok

> 
> 9: Fortunately, the adaptation of Binary
>    Search, and an enhanced Binary Search with Loss Verification have
>    been specified in clause 12.3 of [TST009].  These alogorithms can
> 
> [O] alogorithms
> [P] algorithms
[acm]
Hmm, I didn't know I was mis-typing this so often...
> 
>    easily be used for Back-to-back Frame benchmarking by replacing the
>    Offered Load level with burst length in frames.  [TST009] Annex B
>    describes the theory behind the enhanced Binary Search with Loss
>    Verification algorithm.
> 
>    There is also promising work-in-progress that may prove useful in for
> 
> [O]  useful in for
> [P] useful in
[acm]
ok

> 
> 10: The "Measured Throughput" is the [RFC2544] Throughput Benchmark
>        for the frame size tested, as augmented by methods including the
>        Binary Search with Loss Verification aglorithm in [TST009] where
> 
> [O] aglorithm
> [P] algorithm
[acm] 
... and that someone named "al" would avoid this typo more easily!

> 
> 
> Thank you again,
> W
> 
> [0]: Some people take the 2119->8174 change more seriously than
> others. I don't really care, but it's a hot-button for some, and so
> addressing it now seems cleaner....
[acm]
Yes, I know that Barry is one of them. BMWG also takes the Requirements Language seriously (as described in the Intro, Scott Bradner created it for our use, despite our work being limited to IETF-stream Informational RFCs).


> 
> 
> --
> I don't think the execution is relevant when it was obviously a bad
> idea in the first place.
> This is like putting rabid weasels in your pants, and later expressing
> regret at having chosen those particular rabid weasels and that pair
> of pants.
>    ---maf