Re: [bmwg] Meeting Minutes Review: IPsec Terminology
Tim Van Herck <herckt@cisco.com> Wed, 29 October 2003 21:13 UTC
Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged)) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA06510 for <bmwg-archive@lists.ietf.org>; Wed, 29 Oct 2003 16:13:27 -0500 (EST)
Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AExcj-0004uv-0x; Wed, 29 Oct 2003 16:13:01 -0500
Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AExcT-0004tx-9h for bmwg@optimus.ietf.org; Wed, 29 Oct 2003 16:12:45 -0500
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA06500 for <bmwg@ietf.org>; Wed, 29 Oct 2003 16:12:34 -0500 (EST)
Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AExcR-0003UK-00 for bmwg@ietf.org; Wed, 29 Oct 2003 16:12:43 -0500
Received: from ams-iport-1.cisco.com ([144.254.74.5]) by ietf-mx with esmtp (Exim 4.12) id 1AExcQ-0003UH-00 for bmwg@ietf.org; Wed, 29 Oct 2003 16:12:43 -0500
Received: from cisco.com (171.71.177.254) by ams-iport-1.cisco.com with ESMTP; 29 Oct 2003 22:10:04 +0100
Received: from mira-sjc5-f.cisco.com (IDENT:mirapoint@mira-sjc5-f.cisco.com [171.71.163.13]) by sj-core-2.cisco.com (8.12.9/8.12.6) with ESMTP id h9TLC6j0006285; Wed, 29 Oct 2003 13:12:08 -0800 (PST)
Received: from cisco.com (dhcp-10-34-44-35.cisco.com [10.34.44.35]) by mira-sjc5-f.cisco.com (Mirapoint Messaging Server MOS 3.3.6-GR) with ESMTP id ALZ82410; Wed, 29 Oct 2003 13:17:27 -0800 (PST)
Message-ID: <3FA02D25.6030007@cisco.com>
Date: Wed, 29 Oct 2003 13:12:05 -0800
From: Tim Van Herck <herckt@cisco.com>
Organization: Cisco Systems, Inc.
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax)
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: "Talbert, Brian" <brian.talbert@mci.com>
CC: bmwg@ietf.org
Subject: Re: [bmwg] Meeting Minutes Review: IPsec Terminology
References: <F4CCF42A318CE3448ACA172CB087AF230612993B@DGEXCH04.mcilink.com>
In-Reply-To: <F4CCF42A318CE3448ACA172CB087AF230612993B@DGEXCH04.mcilink.com>
Content-Type: text/plain; charset="us-ascii"; format="flowed"
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: bmwg-admin@ietf.org
Errors-To: bmwg-admin@ietf.org
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit
Brian, >>>1. Imix is a strict defined packetmix >>> - poses problems with newly introduced links and also IPv6, it >>> will work with old data >>> - has the advantage of providing an apples-to-apples comparison > > This does provide apples-to-apples comparison of device performance but as > has already been suggested, one of the primary drivers for an IMIX number is > for marketing/sales ease of use. And as such the number is not just used to > compare one vendor to another but also to provide some notion of customer > experience for a given service. So, the strictly defined mix does start to > become apples-to-orange from the perspective of trying to understand how a > given device will perform for a given application. Of particular note in > this discussion is for VPN applications, for at least in my experience the > Ipsec devices that we benchmark are used to build VPNs and to forwad data > for enterprise applications securelty across the Internet and not merely to > send generalized Internet data through an Ipsec tunnel. It is indeed correct that traffic over an IPSEC tunnel will not be the same as generalized Internet traffic since more emphasis will be on enterprise client-server application generated data as described in EMIX. >>>2. Imix is -a- mix, user reports what it is >>> - hard to compare results >>> - probably will still use the old style IMIX distributions > > If IMIX must be defined at all this would probably get my vote as it > provides the freedom to create custome mixes that suite the application of > the device. Now my perspective is that of the service provider who is > characterizing service expectations and it is not the perspective of an > equipment manufacturer, who could conceivably create a mix that is favors > the nature of their device rather than any particular application (we > already see vendors using simple IMIX streams in the VPN space that omit the > 1500 byte packet in favor of a 1400 byte packet because it exposes the > inefficiency with which they reassemble fragments). So this is an admitted > down side but I don't see a more positive approach other than perhaps to > standardize many different mixes suitable for a variety of applications, but > I don't see this as being feasible. The more we discuss about this the more I also tend to go with this solution where we only loosely define IMIX as -a- mix and suggest a few types but make sure that during reporting the mix is described since this will highly influence the results. Next we will also stress that this is an optional test and that this is not so much of a engineering style test but rather a marketing gimmick that it used and that it is not a prefered traffic stream for testing (as Scott suggested). But I still feel we need to get the industry in line about IMIX since the term is used often without any reference to what is behind it and as you just mentioned IMIX can take all forms and shapes at present. >>>3. Imix is a packetsweep (after renaming the mix) >>> - good chance of comparable results >>> - can be adjusted to the needs of the network >>> - exercises the full datapath >>> - more futureproof > > One of the downsides to this is that you will essentially create a strictly > defined packet mix, with all of it's downsides. For instance, if as > suggested you sweep from min to max and do so in fixed n-byte increments, > you may end up yielding results that unfairly favor one vendor over another > simply based upon the nature of their internal architecture, which may for > instance use n-byte cells to move data chuncks through the switching fabric. > Modify the sweep to n+1 increments and you get vastly different results. So > how do you choose n? There are potential solutions, like randomizing the > packet size, etc., but .... And then the discussion over which packetsizes to include start all over again. So this is not an option ! A sweep with 1 byte increments would solve the architecture dependency but it will have to be linked to the MTU and that again changes the average packetsize of the mix which then creates incomparable results all over. >>>Let the voting begin ! > > ... I still vote for number 2, simply define the general concept of a mixed > packet stream (in a general benchmarking terminology document so that the > definition carries across all BMWG efforts) and require that the stream be > clearly defined to include not only packet size distribution but the > interleaving pattern and any other parameters that may further characterize > the nature of the stream. We'll do that ! Unless anybody has another opinion about it ... Thanks, Tim- _______________________________________________ bmwg mailing list bmwg@ietf.org https://www1.ietf.org/mailman/listinfo/bmwg
- Re: [bmwg] Meeting Minutes Review: IPsec Terminol… Kevin Dubray
- [bmwg] Meeting Minutes Review: IPsec Terminology Michele Bustos
- RE: [bmwg] Meeting Minutes Review: IPsec Terminol… Talbert, Brian
- RE: [bmwg] Meeting Minutes Review: IPsec Terminol… sporetsky
- RE: [bmwg] Meeting Minutes Review: IPsec Terminol… Merike Kaeo
- RE: [bmwg] Meeting Minutes Review: IPsec Terminol… sporetsky
- Re: [bmwg] Meeting Minutes Review: IPsec Terminol… Tim Van Herck
- RE: [bmwg] Meeting Minutes Review: IPsec Terminol… Scott Bradner
- Re: [bmwg] Meeting Minutes Review: IPsec Terminol… Tim Van Herck
- RE: [bmwg] Meeting Minutes Review: IPsec Terminol… Merike Kaeo
- RE: [bmwg] Meeting Minutes Review: IPsec Terminol… Michele Bustos
- RE: [bmwg] Meeting Minutes Review: IPsec Terminol… Michele Bustos
- RE: [bmwg] Meeting Minutes Review: IPsec Terminol… Michele Bustos
- Re: [bmwg] Meeting Minutes Review: IPsec Terminol… Tim Van Herck
- Re: [bmwg] Meeting Minutes Review: IPsec Terminol… Scott Bradner
- Re: [bmwg] Meeting Minutes Review: IPsec Terminol… Tim Van Herck
- Re: [bmwg] Meeting Minutes Review: IPsec Terminol… Scott Bradner
- Re: [bmwg] Meeting Minutes Review: IPsec Terminol… Tim Van Herck
- RE: [bmwg] Meeting Minutes Review: IPsec Terminol… Sunil Kalidindi
- RE: [bmwg] Meeting Minutes Review: IPsec Terminol… Talbert, Brian
- Re: [bmwg] Meeting Minutes Review: IPsec Terminol… Tim Van Herck
- Re: [bmwg] Meeting Minutes Review: IPsec Terminol… Al Morton
- Re: [bmwg] Meeting Minutes Review: IPsec Terminol… Al Morton