Re: [bmwg] draft-ietf-bmwg-dcbench-methodology-07

["MORTON, ALFRED C (AL)" <[e-mail]>]

adding OPS-DIR,
> -----Original Message-----
> From: bmwg [mailto:[e-mail]] On Behalf Of Scott Bradner
> Sent: Sunday, June 18, 2017 9:53 AM
> To: MORTON, ALFRED C (AL)
> Cc: [e-mail]; bmwg-
> [e-mail]; [e-mail]
> Subject: Re: [bmwg] draft-ietf-bmwg-dcbench-methodology-07
> 
> my issue was that I read the current text as talking about the result of
> the testing not the offered load
> 
> so I was just going to suggest coming similar to what Al suggested
> 
> but I’d add specific pointers to the explanation in the terminology
> document as to why
> 
> 
> e.g.
> 
>    For all tests, the test traffic generator sending rate MUST be
>    less than or equal to 99.98% of the nominal value of Line Rate
>    (with no further PPM adjustment to account for interface clock
>    tolerances), to ensure stressing the DUT in reasonable worst case
>    conditions. (see RFC XXX section ??? - note to RFC editor replace XXX
>    with number of terminology RFC)
> 
[ACM] 
good addition, Scott.

> I have not reviewed the rest of the ID to see if similar misreading is
> possible so that
> a similar change would be useful
[ACM] 
Let's assume for now that other Area reviewers and IESG review will
catch some more of these, if needed. 

thanks,
Al

> 
> Scott
> 
> 
> > On Jun 18, 2017, at 1:03 AM, MORTON, ALFRED C (AL) <[e-mail]>
> wrote:
> >
> > Hi Lucien, Scott, and all,
> >
> >
> >
> > When I read this material, I assumed the goal was to
> >
> > limit the maximum sending rate of the test traffic generator
> >
> > in accordance with the Ethernet clock tolerances,
> >
> > effectively choosing the low-end of clock tolerance
> >
> > as the maximum sending rate with a factor of 2 margin.
> >
> > 99.98% of the nominal line rate would be -200 ppm,
> >
> > and any conforming interface clock should work fine.
> >
> >
> >
> > If I understood your goal correctly,
> >
> > I wonder if a small clarification will help future
> >
> > readers.
> >
> >
> >
> > When section 2.2 says:
> >
> >    For all tests, the percentage of traffic per port capacity sent
> MUST
> >
> >                                                 ^^^^^^^^^^^^^^^^^^
> >
> >    be 99.98% at most, with no PPM adjustment to ensure stressing the
> DUT
> >
> >       +++++++++++++++
> >
> >    in worst case conditions.
> >
> >
> >
> > You’ve defined the concept of Port Capacity in terminology Section 5,
> >
> > under the Line Rate definition, as
> >
> >
> >
> >    The term "port capacity" defines the maximum speed capability for
> the
> >
> >    given port; for example 1GE, 10GE, 40GE, 100GE etc.
> >
> >
> >
> > Actually, the port capacity examples are the *nominal value of Line
> Rate*,
> >
> > before the clock tolerances are considered.
> >
> >
> >
> > Here’s a suggestion for Section 2.2 of the meth:
> >
> >
> >
> >    For all tests, the test traffic generator sending rate MUST be
> >
> >    less than or equal to 99.98% of the nominal value of Line Rate
> >
> >    (with no further PPM adjustment to account for interface clock
> >
> >    tolerances), to ensure stressing the DUT in reasonable worst case
> >
> >    conditions.
> >
> >
> >
> > Changing
> >
> > s/port capacity/nominal value of Line Rate/
> >
> > here, requires an additional substitution in section 2.3 of the meth
> draft,
> >
> > and a single substitution in section 5.1 of the term draft.
> >
> >
> >
> > Does this help?
> >
> > Al
> >
> >
> >
> >
> >
> > From: Lucien Avramov [mailto:[e-mail]]
> > Sent: Saturday, June 17, 2017 1:49 PM
> > To: Scott Bradner
> > Cc: [e-mail]; [e-mail];
> [e-mail]
> > Subject: Re: draft-ietf-bmwg-dcbench-methodology-07
> >
> >
> >
> > Hi Scott!
> >
> > ·         The draft you are reviewing is the ''methodology'' draft
> which explains the methodology for line rate performance testing. The
> exact reason onto why it is 99.98% with 0 PPM adjustment is covered
> extensively in the terminology draft, section 5:
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__tools.ietf.org_html_draft-2Dietf-2Dbmwg-2Ddcbench-2Dterminology-
> 2D12-23section-2D5&d=DwIGaQ&c=LFYZ-
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=cY2eIWfAaujzPr7A49lWCjuJ_Vx
> 0ywhEDDmsWPBcTJA&s=96T3Kqxhm3c-gASDa1w_XDRAcXYh30m7fyjPEm28HCE&e= . That
> draft dependency is covered in the introduction and also in the
> reference lists and is referenced throughout the document
> >
> > ·         There is a clock drift difference between tester and DUT
> explained in section 5.2 of the terminology draft
> >
> > In our view BMWG has not morphed, our goal has been to be very precise
> and accurate with performance testing, and this is not about conformance
> testing at all. These definitions that you see, have not changed in 4
> years and have been proven themselves in a very large amount of testing
> done since we started publishing.
> >
> >
> >
> > As you may remember, we have been working on this draft actively for
> over 4 years now. We've incorporated numerous of your feedbacks on these
> drafts over the IETF sessions (since IETF 86 - captured in the meeting
> minutes of the IETF sessions).  On April 2016, we had your blessing on
> the list also regarding our drafts.
> >
> >
> >
> > We appreciate as always your feedback, Scott!
> >
> >
> >
> > Lucien
> >
> >
> >
> >
> >
> > On Fri, Jun 16, 2017 at 7:33 PM, Scott Bradner <[e-mail]> wrote:
> >
> > try again - my new mac book often sends mail before I want it to
> >
> >
> > I have been asked to do a OPS-DIR review of draft-ietf-bmwg-dcbench-
> methodology-07
> >
> > I quickly became confused - a paragraph in section 2.2 reads
> >
> > For all tests, the percentage of traffic per port capacity sent MUST
> >   be 99.98% at most, with no PPM adjustment to ensure stressing the
> DUT
> >   in worst case conditions. Tests results at a lower rate MAY be
> >   provided for better understanding of performance increase in terms
> of
> >   latency and jitter when the rate is lower than 99.98%. The receiving
> >   rate of the traffic SHOULD be captured during this test in % of line
> >   rate.
> >
> >
> > Thus reads to me like a conformance test not a performance test
> (saying what is “passing”)  unless
> > BMWG has morphed a lot since I was last involved conformance and
> pass/fail are not things that BMWG
> > should be doing -
> >
> > what am I missing?
> >
> > Scott
> >
> >
> >
> >
> 
> _______________________________________________
> bmwg mailing list
> [e-mail]
> https://urldefense.proofpoint.com/v2/url?u=https-
> 3A__www.ietf.org_mailman_listinfo_bmwg&d=DwIGaQ&c=LFYZ-
> o9_HUMeMTSQicvjIg&r=OfsSu8kTIltVyD1oL72cBw&m=cY2eIWfAaujzPr7A49lWCjuJ_Vx
> 0ywhEDDmsWPBcTJA&s=NwNczwTUN_yQqjl4Aw4jBTqk3iMbJpTrJMDiuFT8g20&e=