[bmwg] AD Review of draft-ietf-bmwg-b2b-frame

Warren Kumari <warren@kumari.net> Mon, 09 November 2020 15:22 UTC

Return-Path: <warren@kumari.net>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id 646ED3A1135 for <bmwg@ietfa.amsl.com>; Mon, 9 Nov 2020 07:22:51 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=kumari-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id fC546tVoc2uC for <bmwg@ietfa.amsl.com>; Mon, 9 Nov 2020 07:22:49 -0800 (PST)
Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com [IPv6:2a00:1450:4864:20::136]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8FB923A1132 for <bmwg@ietf.org>; Mon, 9 Nov 2020 07:22:49 -0800 (PST)
Received: by mail-lf1-x136.google.com with SMTP id d17so9567769lfq.10 for <bmwg@ietf.org>; Mon, 09 Nov 2020 07:22:49 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kumari-net.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to :content-transfer-encoding; bh=+ZuYlfO8NhYEe3KFfAdmjrqFssDRPQY++REixLayAd8=; b=lFuV4V89rCCghg0cNx9zAMukRfKBbbTEMGoRgtnHDPXijyaANm5o2nZNrhukOhHVJk RC6o7DGitTm6y8wn6NQNziULu2arxkWsoP/k+sR50+DuQnlnjPFoM9XDaIdRy2ECM9W1 M38zTPSRzhsMdOxO3WsJgJaKSQtGZ7mzVjC4Me9w6g86jwc9IgjRauPchm4aZNvMGiCo Jx/joGKsfyKHIR1UIEuchag9Aq9jfzrJvXP+qq+34bAh302wYzzg7RPioDjqpZ6iAQSC AIWUolqYnhinnZsjlVQ/QSk5/vhjHVYyksexoWqjTodc0re97qnY3Vk4NErFXieGYTaP YBkw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to :content-transfer-encoding; bh=+ZuYlfO8NhYEe3KFfAdmjrqFssDRPQY++REixLayAd8=; b=AeFWk3ihA3eLc5jYs8wsvQGOI1QT8ebsi8SWFiFYUerjycUrJe/ZJmj6J9I5E9f1DO Z+z35oIR8MNVH5U+5T3k7WCBGBqZlLfjDdceb/D8od14/3TqGvcErvsRWP4SNwJeQRYE 6eZ94aKhCUQIFkJXBoK6misn4JXJWlq5VeGGXVhRUb9CQG6bqSaVVCrfQYcnHEhLomWb oxHnv2uWecZM7ObMPoeWi6H0JjLVOe/66FfJLhapFe4ikBunJVu0WQYXuZBaQ54zhigh X7Q4yzAz6jRyl7fcxX+1DcQyhVCSzisLuWD4MmicO6M0qcWWe/oXaZr0fQbhnc4J+W5D xUfA==
X-Gm-Message-State: AOAM533h08W/e4grFAtBpmbIiXXbPA8laiPbLJmvbw07lfNyeyvWJ0SU HxQlyZ53VhJa28lMaUrY2VGytzh4Yd6dApJN5awggw==
X-Google-Smtp-Source: ABdhPJyTBvTM50RhRNlsjG2lPCx2Cdz68n8QHiHFSmovAWfjAq3LL7NGrBBcEvwIWA5N7CgX3HBPKBBw2Z0T29P0L+Q=
X-Received: by 2002:a05:6512:314d:: with SMTP id s13mr2739555lfi.464.1604935366839; Mon, 09 Nov 2020 07:22:46 -0800 (PST)
MIME-Version: 1.0
From: Warren Kumari <warren@kumari.net>
Date: Mon, 9 Nov 2020 10:22:10 -0500
Message-ID: <CAHw9_iLgv9Mqe42_H==O1rR4zXNaS+QtXJaP-Zgg_cE6c2pXUA@mail.gmail.com>
To: draft-ietf-bmwg-b2b-frame@ietf.org, bmwg@ietf.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/O0NvvzR7E83rgcHe9MYYFtweNPY>
Subject: [bmwg] AD Review of draft-ietf-bmwg-b2b-frame
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 09 Nov 2020 15:22:51 -0000

Hi there ADs and WG.

Thank you for this document, I found it clear, useful, and an easy read.
I do have a few comments/nits that addressing now should help the IETF
LC and iesg eval go more smoothly. Some of the plenary discussions at
IETF108 brought to light a desire from the community that nits and
similar be better addressed before LC/eval/the RFC Ed, and so I'm
trying to be especially diligent about noting them.

Please SHOUT loudly once you've had a chance to address these. We are
during a posting-blackout, but seeing as the document is already in
"Publication Requested" state, and these are nits/readability, I'm
happy to approve posting a new version anyway...

1: RFC2119 was updated/replaced with RFC8174. Replacing RFC2119 with RFC8174 in:
"Thus, the requirements presented in this memo are expressed in
[RFC2119] terms,... " would probably be a good idea (or explain in the
document why not). This will require some text massaging in the prior

2: "Fortunately, the Open Platform for Network Function Virtualization
(OPNFV) VSPERF project’s Continuous Integration (CI) testing ..."
Is there a link/reference that we can insert for this?

3: "Back-to-back Frame Benchmark was very consistent for some fixed
frame sizes, and somewhat variable for others."
s/for others/for other frame sizes/ ? It might just be me, but this
took me a few tries to understand. This might be a nit...

4: "It’s important that the buffering time was used in this analysis."
This reads like a fragment -- *why* is it important? Perhaps "it's
important to note that...", but I must admit I still don't quite
understand what this sentence is adding - that 30 seconds is
unrealistically long and someone should have asked "...what, what?!"?

 5: Similarly, I'm having a hard time parsing "using results" in:
"It was found that the actual extent of buffer time in the DUT could
be estimated using results to measure the longest burst"
Using *what* results?
"It was found that the actual extent of buffer time in the DUT could
be estimated by measuring the longest burst in frames without loss and
results from the Throughput tests conducted according to Section 26.1
of [RFC2544]." ? But I actually have no idea, and so am just

6: Section 7 - Security Considerations
I *think* that it might be useful to add:
"[RFC Editor, please remove before publication. This is the default
security considerations text which is included in BMWG documents - see
rfc6815 for more info.] "
or similar. This may help having to explain this again....

1: "The IETF’s fundamental Benchmarking Methodologies are defined
There is a missing space before the reference.

2: "In particular, analysis of the results indicates that buffers size
matters when compensating for disruptions in the software packet
processor,  "
s/buffers/buffer/ (or "that the size of buffers matters", or similar)

3: "The tests described in this memo address the cases where the
maximum frame rate"
s/cases where/cases in which/? (readability)

4: The [RFC8239] methods measure
   buffer latency directly with traffic on multiple ingress ports that
   overload an egress port on the Device Under Test (DUT), and are not ...
s/(DUT),./(DUT)/ spurious comma.

5: To summarize, there are several reasons that
   devices on a network produce bursts of frames at the minimum allowed
   spacing, and it is therefore worthwhile to understand the Device
[O]  spacing, and it is therefore worthwhile
[P] spacing; and it is, therefore, worthwhile

6:  3. Calculation of the extent of buffer time in the DUT helped to
       explain the results observed with all frame sizes (for example,
       some frame sizes cannot exceed the frame header processing rate
       of the DUT and therefore no buffering occurs, therefore the
[O] occurs, therefore
[P] occurs; therefore,

7: The presentation of OPNFV VSPERF evaluation and development of
   enhanced search alogorithms [VSPERF-BSLV] was discussed at IETF-102.

[O] alogorithms
[P] algorithms

   The enhancements are intended to compensate for transient inerrrupts

[O] inerrrupts
[P] interrupts

8: Therefore, sources of packet loss that are un-related

[O] un-related
[P] unrelated

9: Fortunately, the adaptation of Binary
   Search, and an enhanced Binary Search with Loss Verification have
   been specified in clause 12.3 of [TST009].  These alogorithms can

[O] alogorithms
[P] algorithms

   easily be used for Back-to-back Frame benchmarking by replacing the
   Offered Load level with burst length in frames.  [TST009] Annex B
   describes the theory behind the enhanced Binary Search with Loss
   Verification algorithm.

   There is also promising work-in-progress that may prove useful in for

[O]  useful in for
[P] useful in

10: The "Measured Throughput" is the [RFC2544] Throughput Benchmark
       for the frame size tested, as augmented by methods including the
       Binary Search with Loss Verification aglorithm in [TST009] where

[O] aglorithm
[P] algorithm

Thank you again,

[0]: Some people take the 2119->8174 change more seriously than
others. I don't really care, but it's a hot-button for some, and so
addressing it now seems cleaner....

I don't think the execution is relevant when it was obviously a bad
idea in the first place.
This is like putting rabid weasels in your pants, and later expressing
regret at having chosen those particular rabid weasels and that pair
of pants.