Re: [bmwg] Genart last call review of draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06

[Marius Georgescu <[e-mail]>]

Hello Robert,

Thank you very much for your detailed review.

We have revised the draft according to your comments.

The result was uploaded as:

https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07

Please find inline a punctual overview of the changes.

On 4/26/2017 7:05 PM, Robert Sparks wrote:
> Reviewer: Robert Sparks
> Review result: Ready with Issues
>
> I am the assigned Gen-ART reviewer for this draft. The General Area
> Review Team (Gen-ART) reviews all IETF documents being processed
> by the IESG for the IETF Chair.  Please treat these comments just
> like any other last call comments.
>
> For more information, please see the FAQ at
>
> <https://trac.ietf.org/trac/gen/wiki/GenArtfaq>.
>
> Document: draft-ietf-bmwg-ipv6-tran-tech-benchmarking-06
> Reviewer: Robert Sparks
> Review Date: 2017-04-26
> IETF LC End Date: 2017-05-02
> IESG Telechat date: Not scheduled for a telechat
>
> Summary: Essentially Ready for publication as an Informational RFC,
> but with some minor issues to address before publication
>
> This document is (with exceptions noted below) straightforward and
> easy to follow
>
> Minor Issues:
>
> Section 8 is very confusing - I suspect it has been made so by
> removing things that were in it earlier. Right now it claims to
> provide additional tests, but the only content is about testing things
> with firewalls, along with a statement that this document is only
> targeting network devices that do not have a firewall function. I
> think you can keep most of the text here (except the statement that
> you aren't talking about things with firewalls) and remove the
> confusion by changing the section heading to something like "Tests in
> the presence of a firewall function".
- we have removed the second paragraph , which was indeed confusing.
The point, which may be clearer in Section 1.1 is that we need
additional benchmarks for stateful IPv6 transition technologies,
which may be (or not) labeled as having a firewall function.  
> It's unclear how to apply the formula in Section 10.2.1 to the results
> that come out of, say, Section 7.3.2, where you are reporting a
> (minimum, median, maximum) tuple. Some discussion about the
> applicability of the tests where you recommend against reporting a
> single number to the methods in this section would help.
+ we added a note in the reporting format part stating: "

*In the case of benchmarks for which more than one value is reported
(e.g. IPDV Section 7.3.2), a column for each of the values SHOULD be
included.*

"
>  It would also
> help to point out that the Xpd result can be go negative (it will go
> negative for things that become smaller as the number of flows
> increase, and positive for things that get bigger). If I read this
> correctly, if throughput (for example) goes to 0 as n increases, Xpd
> will go to -100%. Similary if latency doubles as n increases, Xpd will
> go to +100% (and will go to +200% if latency triples).
+ This was one of the important points which needed clarification. Thank
you for pointing it out.
In order to keep the  performance degradation results in the Lower is
better  direction, we proposed a secondary calculation formula for
higher is better benchmarks (e.g. Throughput). This makes, in our
opinion, the results easier to interpret.

> Nits:
>
> There are several places where you point to Section 6 where I think
> you meant to point to Section 7. See the Procedure: line in 10.2.1 for
> an example. Also make sure 9 is correct where you say "Sections 6
> through 9" in section 12.
+Corrected
>
> Please double-check that you meant "larger MTU" in the last paragraph
> of 5.1. It might be correct, but I find the paragraph confusing.
We find this part to be correct. However, keeping only the 1280 +
overhead recommendation might make it less confusing.
Do you think that would be a viable solution?
> In 8.1 you meant to point to 5.2, not 5.3
>
> Please try to simplify this sentence: 
>   "The duration of each trial SHOULD be at least 60 seconds to reduce
> the potential gain of a DNS64 server, which is able to exhibit higher
> performance by storing the requests and thus utilizing also the
> timeout time for answering them."
+corrected
>
> (Style comment - please feel free to ignore): Consider deleting "as
> well" everywhere it occurs in the document - most of the places it is
> used, the sentence works just as well, and sometimes better, without
> it.
+corrected
> Typos:
> end of section 5.1.1: Appendix A1 should be Appendix A
+corrected

Best regards,
Marius