Re: [bmwg] draft-ietf-bmwg-dcbench-methodology-07

["MORTON, ALFRED C (AL)" <[e-mail]>]

Hi Lucien, Scott, and all,

When I read this material, I assumed the goal was to
limit the maximum sending rate of the test traffic generator
in accordance with the Ethernet clock tolerances,
effectively choosing the low-end of clock tolerance
as the maximum sending rate with a factor of 2 margin.
99.98% of the nominal line rate would be -200 ppm,
and any conforming interface clock should work fine.

If I understood your goal correctly,
I wonder if a small clarification will help future
readers.

When section 2.2 says:
   For all tests, the percentage of traffic per port capacity sent MUST
                                                ^^^^^^^^^^^^^^^^^^
   be 99.98% at most, with no PPM adjustment to ensure stressing the DUT
      +++++++++++++++
   in worst case conditions.

You’ve defined the concept of Port Capacity in terminology Section 5,
under the Line Rate definition, as

   The term "port capacity" defines the maximum speed capability for the
   given port; for example 1GE, 10GE, 40GE, 100GE etc.

Actually, the port capacity examples are the *nominal value of Line Rate*,
before the clock tolerances are considered.

Here’s a suggestion for Section 2.2 of the meth:

   For all tests, the test traffic generator sending rate MUST be
   less than or equal to 99.98% of the nominal value of Line Rate
   (with no further PPM adjustment to account for interface clock
   tolerances), to ensure stressing the DUT in reasonable worst case
   conditions.

Changing
s/port capacity/nominal value of Line Rate/
here, requires an additional substitution in section 2.3 of the meth draft,
and a single substitution in section 5.1 of the term draft.

Does this help?
Al


From: Lucien Avramov [mailto:[e-mail]]
Sent: Saturday, June 17, 2017 1:49 PM
To: Scott Bradner
Cc: [e-mail]; [e-mail]; [e-mail]
Subject: Re: draft-ietf-bmwg-dcbench-methodology-07

Hi Scott!
·         The draft you are reviewing is the ''methodology'' draft which explains the methodology for line rate performance testing. The exact reason onto why it is 99.98% with 0 PPM adjustment is covered extensively in the terminology draft, section 5: https://tools.ietf.org/html/draft-ietf-bmwg-dcbench-terminology-12#section-5<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Dbmwg-2Ddcbench-2Dterminology-2D12-23section-2D5&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=XTSYKMpGlG-drxu-CVqDhTNx8YcVWdMmypOAISknLvc&s=UkXcmruEslwhl7CcJhOFERz_27CZSfAfR-0WHqZl13g&e=>. That draft dependency is covered in the introduction and also in the reference lists and is referenced throughout the document
·         There is a clock drift difference between tester and DUT explained in section 5.2 of the terminology draft
In our view BMWG has not morphed, our goal has been to be very precise and accurate with performance testing, and this is not about conformance testing at all. These definitions that you see, have not changed in 4 years and have been proven themselves in a very large amount of testing done since we started publishing.

As you may remember, we have been working on this draft actively for over 4 years now. We've incorporated numerous of your feedbacks on these drafts over the IETF sessions (since IETF 86 - captured in the meeting minutes of the IETF sessions).  On April 2016, we had your blessing on the list also regarding our drafts.

We appreciate as always your feedback, Scott!

Lucien


On Fri, Jun 16, 2017 at 7:33 PM, Scott Bradner <[e-mail]<mailto:[e-mail]>> wrote:
try again - my new mac book often sends mail before I want it to


I have been asked to do a OPS-DIR review of draft-ietf-bmwg-dcbench-methodology-07

I quickly became confused - a paragraph in section 2.2 reads

For all tests, the percentage of traffic per port capacity sent MUST
  be 99.98% at most, with no PPM adjustment to ensure stressing the DUT
  in worst case conditions. Tests results at a lower rate MAY be
  provided for better understanding of performance increase in terms of
  latency and jitter when the rate is lower than 99.98%. The receiving
  rate of the traffic SHOULD be captured during this test in % of line
  rate.


Thus reads to me like a conformance test not a performance test (saying what is “passing”)  unless
BMWG has morphed a lot since I was last involved conformance and pass/fail are not things that BMWG
should be doing -

what am I missing?

Scott