[bmwg] draft-ietf-bmwg-ipv6-tran-tech-benchmarking-00.txt

["Marius Georgescu" <[e-mail]>]

Hello BMWG,

Thank you very much for your support and feedback so far, and I hope I can
count on it as well in the future. With Al's help the version 00 of the WG
draft is now uploaded
(https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-00)
.
Please find below a changelog of the update.

### Tentatively ADDRESSED Comments

Comments from Al Morton:

    - clarify CE, PE terminology
		+ redefined the generic classes of IPv6 transition
technologies and gave -up the Customer-provider model in favour of
IP-specific domains model suggested by Fred Baker
	- use higher percentile for PDV calculation
		+ revised formula for PDV as:
		PDV=D99.9thPercentile - Dmin
		Where: 	D99.9thPercentile - the 99.9th Percentile (as it was
described in [RFC5481]) of the One-way delay for the stream
		 Dmin - the minimum One-way delay in the stream
	- additions to 2. Conventions used in this document
		+ added the following text:
		"although these terms are usually associated
		with protocol requirements, in this doc the terms are
requirements for users and
		systems that intend to implement the test conditions and
claim conformance with
		this specification."
	- SONET's almost historical status
+2 Similar comments from Fred Baker and Nalini Elkins
			- removed the recommendations for SONET as example
media
	- MTU specifications 
+1 Similar comment from Masataka Mawatari 
		+ Added the following text:
		"To avoid this situation, the larger MTU between the
physical NICs and virtual encapsulation/translation interfaces SHOULD be set
for all interfaces of the DUT and tester. 
		To be more specific, the minimum IPv6 MTU size (1280 bytes)
plus the encapsulation/translation overhead is the RECOMMENDED value for the
physical interfaces as well as virtual ones. "
	- rewording for Frame Sizes to Be Used over Ethernet:
		+ Text modified to:
		"the following frame sizes SHOULD be used for benchmarking
IPvX/IPvY traffic on Ethernet links: 64, 128, 256, 512, 1024, 1280, 1518,
1522, 2048, 4096, 8192 and 9216."
	- Extension headers use
+1 Similar comment from Fred Baker
		+Text modified to:
		"The selected protocol addresses should follow the
recommendations of [RFC5180](Section 5) for IPv6 and [RFC2544](Section 12)
for IPv4.
		Note: testing traffic with extension headers might not be
possible for the transition technologies which employ translation. 
		Proposed IPvX/IPvY translation algorithms such as IP/ICMP
translation [RFC6145] do not support the use of extension headers."
	- RECOMMEND Units of Measure for PDV
		+Added the text:
		"Following the recommendations of [RFC5481], the RECOMMENDED
units of measurement are milliseconds."
	- update Reset benchmark 
		+added RFC6201 as reference
	- add scalability detaills in the introduction
		+added the following text in Section 1
		"The document also includes an approach to quantify load
scalability. Load scalability can be defined as a system's ability to
gracefully accommodate higher loads. 
		Because poor scalability usually leads to poor performance,
the proposed approach is to quantify the load scalability by measuring the
performance degradation 
		created by a higher number of network flows. "

Comments from Scott Bradner

	- add minor wording on how to set up dynamic routing
		+Added text to section 3:
		"In terms of route setup, the recommendations of [RFC2544]
Section 13 are valid for this document as well assuming that an IPv6 version
of the routing packets shown in appendix C.2.6.2 is used."
	- clarify the 2-DUTs setup limitations
		+Added text to section 3.2-DUTs
		"One of the limitations of the dual DUT setup is the
inability to reflect asymmetries in behavior between the DUTs. 
		Considering this, additional performance tests SHOULD be
performed using the single DUT setup.
		Note: For encapsulation IPv6 transition technologies, in the
single DUT setup, in order to test the decapsulation efficiency, the tester
SHOULD be able to send IPvX packets encasulated as IPvY."
		
Comments from Fred Baker:
	- reference mishap Normative vs Informative
		+ Rearranged references section
	- Better reflect scope in the abstract:
		+ Added text to the abstract:
		"This document provides complementary guidelines for
evaluating the performance of IPv6 transition technologies.  
		More specifically, this document targets IPv6 transition
technologies that employ encapsulation or translation mechanisms, 
		as dual-stack nodes can be very well tested using the
recommendations of RFC2544 and RFC5180. "
	- Redefine the generic categories of IPv6 transition-technologies
		+modifications in Section 1.1 and throughout the
draft-ietf-bmwg-ipv6-tran-tech-benchmarking-00
	- Stop at Ethernet and mention Layer 2 scenario in the test report
		+modified text to
		"To prevent exceeding the limitations imposed by the media,
the frame size overhead needs to be taken into account when calculating the
maximum theoretical frame rates. 
		The calculation method for the Ethernet, as well as a
calculation example are detailed in Appendix A. The details of the media
employed for the benchmarking tests MUST be noted in all test reports."
	- add RFC5180 as additional reference for the tester setup
		+modified text in Section 3 to:
		"In the case of the tester setup, the options presented in
[RFC2544] and [RFC5180] can be applied here as well."

Comment from Nalini Elkins:
	- propose a methodology for DNS performance
	+ Added Section 8. DNS Resolution Performance - contributed by Prof.
Gabor Lencse
	
Comment for Kaname Nishizuka:
	- Consider the existence of DNS cache
		+ Defined caching parameter
		"Details and parameters:
		1. Caching 
		First, all the DNS queries MUST contain different domain
names (or domain names MUST NOT be repeated before the cache of the DUT is
exhausted). 
		Then new tests MAY be executed with 10%, 20%, 30%, etc.
domain names which are repeated (early enough to be still in the cache)."

Comment from Kostas Pentikousis:
	- Rationale for using Mean and not median
		+Added section 10. Summarizing function and repeatability
	
	
### NOT ADDRESSED YET Comments

Comments from Fred Baker:
	-Network Performance Degradation: Incremental vs Simultaneous (State
Creation vs State Use)
	-Benchmarking NATXX using the same methodology

Best regards,
Marius

-----Original Message-----
From: bmwg [mailto:[e-mail]] On Behalf Of
[e-mail]
Sent: Thursday, October 15, 2015 1:18 AM
To: [e-mail]
Cc: [e-mail]
Subject: [bmwg] I-D Action:
draft-ietf-bmwg-ipv6-tran-tech-benchmarking-00.txt


A New Internet-Draft is available from the on-line Internet-Drafts
directories.
 This draft is a work item of the Benchmarking Methodology Working Group of
the IETF.

        Title           : Benchmarking Methodology for IPv6 Transition
Technologies
        Authors         : Marius Georgescu
                          Gabor Lencse
	Filename        : draft-ietf-bmwg-ipv6-tran-tech-benchmarking-00.txt
	Pages           : 22
	Date            : 2015-10-14

Abstract:
   There are benchmarking methodologies addressing the performance of
   network interconnect devices that are IPv4- or IPv6-capable, but the
   IPv6 transition technologies are outside of their scope. This
   document provides complementary guidelines for evaluating the
   performance of IPv6 transition technologies.  More specifically,
   this document targets IPv6 transition technologies that employ
   encapsulation or translation mechanisms, as dual-stack nodes can be
   very well tested using the recommendations of RFC2544 and RFC5180.
   The methodology also includes a tentative metric for benchmarking
   load scalability.


The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-ietf-bmwg-ipv6-tran-tech-benchmarking
/

There's also a htmlized version available at:
https://tools.ietf.org/html/draft-ietf-bmwg-ipv6-tran-tech-benchmarking-00


Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
bmwg mailing list
[e-mail]
https://www.ietf.org/mailman/listinfo/bmwg