[bmwg] Zaheduzzaman Sarker's Discuss on draft-ietf-bmwg-evpntest-09: (with DISCUSS and COMMENT)

[Zaheduzzaman Sarker via Datatracker <[e-mail]>]

Zaheduzzaman Sarker has entered the following ballot position for
draft-ietf-bmwg-evpntest-09: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)


Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about DISCUSS and COMMENT positions.


The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-evpntest/



----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------


Thanks for the attempt here, defining test cases are tricky.

I think to use the test cases in this document effectively, followings need to
correctly specified/addressed -

* All the tests are said to be run for N times and collect N samples. Are those
N same? What should be the default value for the N samples? How do we decide
that there are sufficient samples? I find it a bit odd that there is no
recommendation provided here. There should be a min and max range of values the
N can take so that we know we have done enough tests to conclude on the
results. Is there are specific waiting time required between running those
tests N times? This feel like very under specified to me.

* Section 3.10: what is the exit criteria of this test? how do we know the test
was successful?

* Section 3.12: I am missing a recommended duration of running the test here. I
am supposed to take hourly reading but for how long time I should do that to
get to an conclusion?

* May be I missed it but I haven't found any mention of non-test traffic if
they are allowed during the test, if yes what kind of considerations should be 
taken when interpreting the results. If they are not allowed during the test it
should be written as well.

I also support Martin and Eric's discuss.


----------------------------------------------------------------------
COMMENT:
----------------------------------------------------------------------


Some non-blocking comments -

* Section 2:
   ** please expand abbreviated words in their first occurrence if not listed
   in well known terminologies. See the list :
   https://www.rfc-editor.org/materials/abbrev.expansion.txt ** why is this
   called topology 1? are expected to have more typologies? Where are they? and
   it is actually not clear to me what is topology 1.

* Section 3.8: what is PPS?

* Section 3.9: how do you measure packet loss? is this loss measured for the
whole run or with with a time window?

* Section 4.9 : "Then increment the scale of N by 5% of N till the limit is
reached." Please clarify if the increment steps always 5% of initial value or N
or N = N+5% of N.

Nits :

* Section 2 : Figure 2 - s/SHPFE3/SHPE3 or define SHPFE3.