RE: [bmwg] Meeting Minutes Review: IPsec Terminology

"Talbert, Brian" <brian.talbert@mci.com> Wed, 29 October 2003 15:32 UTC

Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged)) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA13988 for <bmwg-archive@lists.ietf.org>; Wed, 29 Oct 2003 10:32:25 -0500 (EST)
Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AEsIj-0005eq-R3; Wed, 29 Oct 2003 10:32:01 -0500
Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AEri7-0007wU-34 for bmwg@optimus.ietf.org; Wed, 29 Oct 2003 09:54:11 -0500
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA09888 for <bmwg@ietf.org>; Wed, 29 Oct 2003 09:53:59 -0500 (EST)
Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AEri5-0001vi-00 for bmwg@ietf.org; Wed, 29 Oct 2003 09:54:09 -0500
Received: from dgesmtp01.wcom.com ([199.249.16.16]) by ietf-mx with esmtp (Exim 4.12) id 1AEri4-0001vZ-00 for bmwg@ietf.org; Wed, 29 Oct 2003 09:54:08 -0500
Received: from pmismtp05.wcomnet.com ([166.38.62.53]) by firewall.wcom.com (Iplanet MTA 5.2) with ESMTP id <0HNI0083KX3871@firewall.wcom.com> for bmwg@ietf.org; Wed, 29 Oct 2003 14:47:32 +0000 (GMT)
Received: from pmismtp05.wcomnet.com by pmismtp05.wcomnet.com (iPlanet Messaging Server 5.1 HotFix 0.7 (built May 7 2002)) with SMTP id <0HNI00J01X37IJ@pmismtp05.wcomnet.com>; Wed, 29 Oct 2003 14:47:32 +0000 (GMT)
Received: from dgexch50.wcomnet.com ([166.38.58.238]) by pmismtp05.wcomnet.com (iPlanet Messaging Server 5.1 HotFix 0.7 (built May 7 2002)) with ESMTP id <0HNI00IGLX37HD@pmismtp05.wcomnet.com>; Wed, 29 Oct 2003 14:47:31 +0000 (GMT)
Received: by DGEXCH50.mcilink.com with Internet Mail Service (5.5.2653.19) id <VDBQW1AF>; Wed, 29 Oct 2003 14:47:31 +0000
Content-return: allowed
Date: Wed, 29 Oct 2003 14:47:30 +0000
From: "Talbert, Brian" <brian.talbert@mci.com>
Subject: RE: [bmwg] Meeting Minutes Review: IPsec Terminology
To: "'Tim Van Herck'" <herckt@cisco.com>, sporetsky@quarrytech.com
Cc: bmwg@ietf.org, ipsec-term@external.cisco.com
Message-id: <F4CCF42A318CE3448ACA172CB087AF230612993B@DGEXCH04.mcilink.com>
MIME-version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-type: text/plain; CHARSET=us-ascii
Sender: bmwg-admin@ietf.org
Errors-To: bmwg-admin@ietf.org
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>

>>-----Original Message-----
>>From: bmwg-admin@ietf.org [mailto:bmwg-admin@ietf.org] On 
>>Behalf Of Tim Van Herck
>>Sent: Tuesday, October 28, 2003 6:20 PM
>>To: sporetsky@quarrytech.com
>>Cc: bmwg@ietf.org; ipsec-term@external.cisco.com
>>Subject: Re: [bmwg] Meeting Minutes Review: IPsec Terminology
>>

...

>>1. Imix is a strict defined packetmix
>>    - poses problems with newly introduced links and also IPv6, it
>>      will work with old data
>>    - has the advantage of providing an apples-to-apples 
>>      comparison 

This does provide apples-to-apples comparison of device performance but as
has already been suggested, one of the primary drivers for an IMIX number is
for marketing/sales ease of use.  And as such the number is not just used to
compare one vendor to another but also to provide some notion of customer
experience for a given service.  So, the strictly defined mix does start to
become apples-to-orange from the perspective of trying to understand how a
given device will perform for a given application.  Of particular note in
this discussion is for VPN applications, for at least in my experience the
Ipsec devices that we benchmark are used to build VPNs and to forwad data
for enterprise  applications securelty across the Internet and not merely to
send generalized Internet data through an Ipsec tunnel.  

>>2. Imix is -a- mix, user reports what it is
>>    - hard to compare results
>>    - probably will still use the old style IMIX 
>>distributions 

If IMIX must be defined at all this would probably get my vote as it
provides the freedom to create custome mixes that suite the application of
the device.  Now my perspective is that of the service provider who is
characterizing service expectations and it is not the perspective of an
equipment manufacturer, who could conceivably create a mix that is favors
the nature of their device rather than any particular application (we
already see vendors using simple IMIX streams in the VPN space that omit the
1500 byte packet in favor of a 1400 byte packet because it exposes the
inefficiency with which they reassemble fragments).   So this is an admitted
down side but I don't see a more positive approach other than perhaps to
standardize many different mixes suitable for a variety of applications, but
I don't see this as being feasible.

>>3. Imix is a packetsweep (after renaming the mix)
>>    - good chance of comparable results
>>    - can be adjusted to the needs of the network
>>    - exercises the full datapath
>>    - more futureproof

One of the downsides to this is that you will essentially create a strictly
defined packet mix, with all of it's downsides.  For instance, if as
suggested you sweep from min to max and do so in fixed n-byte increments,
you may end up yielding results that unfairly favor one vendor over another
simply based upon the nature of their internal architecture, which may for
instance use n-byte cells to move data chuncks through the switching fabric.
Modify the sweep to n+1 increments and you get vastly different results.  So
how do you choose n?  There are potential solutions, like randomizing the
packet size, etc., but ....


>>Let the voting begin !

... I still vote for number 2, simply define the general concept of a mixed
packet stream (in a general benchmarking terminology document so that the
definition carries across all BMWG efforts) and require that the stream be
clearly defined to include not only packet size distribution but the
interleaving pattern and any other parameters that may further characterize
the nature of the stream.

Brian Talbert

_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www1.ietf.org/mailman/listinfo/bmwg