Re: [ippm] Zaheduzzaman Sarker's No Objection on draft-ietf-ippm-capacity-metric-method-10: (with COMMENT)

[Zaheduzzaman Sarker <[e-mail]>]

Thanks Al, this looks good to me .

BR
Zahed

________________________________
From: MORTON JR., AL <[e-mail]>
Sent: Wednesday, June 2, 2021 12:17 AM
To: Zaheduzzaman Sarker; The IESG
Cc: [e-mail]; [e-mail]; [e-mail]; Ian Swett; [e-mail]
Subject: RE: Zaheduzzaman Sarker's No Objection on draft-ietf-ippm-capacity-metric-method-10: (with COMMENT)

Hi Zahed,

Please see [acm] replies below.

I think we have reached resolution of all your comments with the added requirement on concurrent tests,
so I'm going to submit version 11 now.

Al

> -----Original Message-----
> From: Zaheduzzaman Sarker <[e-mail]>
> Sent: Tuesday, June 1, 2021 8:51 AM
> To: MORTON JR., AL <[e-mail]>; The IESG <[e-mail]>
> Cc: [e-mail]; [e-mail];
> [e-mail]; Ian Swett <[e-mail]>; [e-mail]
> Subject: Re: Zaheduzzaman Sarker's No Objection on draft-ietf-ippm-capacity-
> metric-method-10: (with COMMENT)
>
>
>
> On 2021-05-30, 04:00, "MORTON JR., AL" <[e-mail]> wrote:
>
>     ...
>     > ----------------------------------------------------------------------
>     > COMMENT:
>     > ----------------------------------------------------------------------
>     >
>     > Thanks for the work on the document. I consider this document to be an
>     > important one when it comes to IP layer capacity for access networks.
>     >
>     > After my review and observing the IPPM working group discussions, I
> consider
>     > the valid discuss points raised by Magnus Westerlund is now addressed in
> the
>     > -10 version of this document.
>     >
>     > please find my comments and observations below -
>     >
>     > * Major Issue: I didn't find any restriction imposed by the memo on the
> number
>     > of allowed concurrent tests between same Source and Destination. My
>     > understanding is this memo should either strictly prohibit the
> concurrent tests
>     > between same Source and Destination or provide a maximum limit of
> concurrent
>     > tests where the algorithm has been tested to provide valid results. I am
> not
>     > going to hold a discuss on this but I would like to be corrected if I am
> wrong
>     > in the understanding.
>     [acm]
>     I'm trying to understand your comment in order to address it properly.
>     Let's consider the existing requirements:
>
>     We have text to restrict concurrent non-measurement traffic, in the Scope:
>
>        It is RECOMMENDED to discontinue non-measurement traffic that shares a
>        subscriber's dedicated resources while testing: measurements may not be
>        accurate and throughput of competing elastic traffic may be greatly
> reduced.
>
> My comment was on the measurement traffic
>
>     We allow a single test to use concurrent flows, and the default is one
> flow
>     (text below is near the end of section 8.3):
>
>        In general, results depend on the sending stream characteristics;
>        the measurement community has known this for a long time, and needs to
> keep
>        it front of mind. Although the default is a single flow (F=1) for
> testing,
>        use of multiple flows may be advantageous for the following reasons:
>        ...
> This is fine.
>
>     We don't use the term "concurrent" in the draft, and it is obviously
>     counter-productive to run more than one test independently attempting to
>     measure the *maximum* capacity between the same Source and Destination.
>     The number of concurrent tests between same Source and Destination
>     that we have tested is one.
> I was looking for some text in the spirit of what you wrote above.
>
>     The maximum limit on concurrent tests *at an on-net server* is a function
>     of the server host characteristics: CPU power, network interface speed,
> and
>     the external Internet access capacity the host can expect to use.
>     None of these characteristics are the subject of standardization
>     in the form of numerical limits. However, in Section 10 we already
> specified:
>
>     - Hosts MUST limit the number of simultaneous tests to avoid resource
>       exhaustion and inaccurate results.
>
>     Perhaps you have a better view of the current text now, and limits on
> testing
>     that makes sense in the context of Maximum capacity measurement.
>     Was it your intent that we mandate what seems to be obvious (one test
> only)?
>     Or something else?
>
> Yes, I think it is logical to explicitly mandate the number of independent
> concurrent test between same destination and server for the counter-productive
> part. And in case the server identifies this it should reject the start of the
> concurrent.
[acm]
Ok, I added a requirement near the end of section 8.3, so that the discussion on
concurrent flows precedes it and can easily be distinguished from the punch-line:

        It is obviously counter-productive to run more than one independent test (regardless of the number of flows in the test stream) attempting to measure the *maximum* capacity between the same Source and Destination. The number of concurrent, independent tests between the same Source and Destination SHALL be limited to one.


>
>     >
>     > * Minor Issues , nits:
>     >
>     >    ** I found it a bit odd to not find Magnus Westerlund's name in the
>     >    acknowledgement section as his review and comments has improved the
>     >    algorithms and the document a lot. I would encourage the author's to
> include
>     >    names of the individuals who has provided any sort of review
> (including the
>     >    directorate reviews) that improved this document.
>     [acm]
>     OK
>     >
>     >    ** Section 2:
>     >
>     >       *** I am not getting the context of "local goal", what does it
>     >       really mean?
>     [acm]
>     Lars questioned this word choice too. We changed it to "secondary goal".
>
>     >
>     >       *** I kind of feel that the paragraph 2 and 3 should swap their
> order to
>     >       focus the goal of "defining efficient test procedure" and then
>     >       "harmonizing the metric and method across the industry" becomes
> "another
>     >       goal". This is based on the current text which makes me feel there
> are
>     >       number of goals and the ranks (if there is any) of the goals are
> not
>     >       clear. If there is not rank among the goals then simply putting
> them in a
>     >       bullet list would help the readability.
>     [acm]
>     The section reads differently with the wording changes related to
> replacement
>     of "local goal".
>     >
>     >    ** Section 8.1:
>     >
>     >       *** paragraph 1: s/ agressiveness / aggressiveness .
>     [acm]
>     OK
>
>     >
>     >       *** paragraph 2: It will be helpful if it is mentioned who pre-
> builts the
>     >       table and  who (and where) it will be used.
>     [acm]
>     ... (by the test initiator)...
>
>     >
>     >    ** Section 10 : I find 4, 5, 6 of the new security considerations
>     > introduced
>     >    in this section not entirely security related rather general
>     > considerations
>     >    on how to run the tests. I think those mentioned considerations are
>     > more
>     >    suitable to put it in section 8 (in section 8.3). I understand the
>     > feeling
>     >    the everyone should read the security considerations but there is no
>     >    guarantee that it is the case specially while implementing the tests.
>     > Hence,
>     >    putting then in section 8 likely will bring these to attention early.
>     [acm]
>     We already make reference to these requirements much earlier than Section
> 8.3,
>     in the text of Section 2, Scope:
>
>     - MUST only be used in circumstances consistent with Section 10,
>       Security Considerations
>
>     We can't make the reference much earlier than that!
>
> Right! That is perhaps fine.
>
>     Changes above have been implemented in the working text for version 11.
>
>     >
>     >
>