Re: [bmwg] Meeting Minutes Review: IPsec Terminology

Tim Van Herck <herckt@cisco.com> Wed, 29 October 2003 00:01 UTC

Received: from optimus.ietf.org (ietf.org [132.151.1.19] (may be forged)) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA09110 for <bmwg-archive@odin.ietf.org>; Tue, 28 Oct 2003 19:01:21 -0500 (EST)
Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AEdlp-0002NX-1q for bmwg-archive@odin.ietf.org; Tue, 28 Oct 2003 19:01:05 -0500
Received: (from exim@localhost) by www1.ietf.org (8.12.8/8.12.8/Submit) id h9T015KQ009134 for bmwg-archive@odin.ietf.org; Tue, 28 Oct 2003 19:01:05 -0500
Received: from localhost.localdomain ([127.0.0.1] helo=www1.ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AEdln-0002Mm-Qk; Tue, 28 Oct 2003 19:01:03 -0500
Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by optimus.ietf.org with esmtp (Exim 4.20) id 1AEdlP-0002Jo-3s for bmwg@optimus.ietf.org; Tue, 28 Oct 2003 19:00:39 -0500
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id TAA09033 for <bmwg@ietf.org>; Tue, 28 Oct 2003 19:00:25 -0500 (EST)
Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 1AEdlL-0003GM-00 for bmwg@ietf.org; Tue, 28 Oct 2003 19:00:35 -0500
Received: from sj-iport-4.cisco.com ([171.68.10.86]) by ietf-mx with esmtp (Exim 4.12) id 1AEdlL-0003FI-00 for bmwg@ietf.org; Tue, 28 Oct 2003 19:00:35 -0500
Received: from mira-sjc5-f.cisco.com (IDENT:mirapoint@mira-sjc5-f.cisco.com [171.71.163.13]) by sj-core-3.cisco.com (8.12.6/8.12.6) with ESMTP id h9T003eu029189; Tue, 28 Oct 2003 16:00:03 -0800 (PST)
Received: from cisco.com (dhcp-10-34-44-35.cisco.com [10.34.44.35]) by mira-sjc5-f.cisco.com (Mirapoint Messaging Server MOS 3.3.6-GR) with ESMTP id ALY93960; Tue, 28 Oct 2003 16:05:24 -0800 (PST)
Message-ID: <3F9F0302.8080504@cisco.com>
Date: Tue, 28 Oct 2003 16:00:02 -0800
From: Tim Van Herck <herckt@cisco.com>
Organization: Cisco Systems, Inc.
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax)
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Scott Bradner <sob@harvard.edu>
CC: bmwg@ietf.org
Subject: Re: [bmwg] Meeting Minutes Review: IPsec Terminology
References: <200310282318.h9SNIc8W005479@newdev.harvard.edu>
In-Reply-To: <200310282318.h9SNIc8W005479@newdev.harvard.edu>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Transfer-Encoding: 7bit
Sender: bmwg-admin@ietf.org
Errors-To: bmwg-admin@ietf.org
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
Content-Transfer-Encoding: 7bit

	Scott,

> fwiw
> it is my opinion that doing a perfomance test of a stream of
> a mix of packet sizes is almost useless - very small changes in
> the mix can produce large changes in the result and any time that
> small changes in conditions can produce large changes in output
> you get a test that is quite hard to 1/ repeate and 2/ find any meaning in

Useless is maybe too strong of a word here but I 100% agree that it is 
far more difficult the reproduce previously obtained result with traffic 
mixes versus a single packet size stream.

But the reality of the matter is that people, whether you like it or 
not, will use it. Like Brian already mentioned, people want to see a 
single number in a spec and don't usually want to deal with looking at a 
long result table that doesn't fit in the spec layout. IMIX is NOT a 
technical requirement but rather a marketing prop that is used without 
any sound foundation.

Although testing with a single packetsize and just walking through all 
packetsizes, if required, would be much more easier, I hate to admit 
that people still request & report IMIX datapoints.

> This is someting that the WG has talked about from time to time, starting
> in Vancover a long time ago, and always decided that such a test was
> not one that would produce clear enough information to be meaningful 
> to do.
> 
> I think it is far more important to test multiple sizes one after the other

Correct, IMIX testing will be optional !

> I can think of one build-to-be-tested case that could be overcome 
> by a mix of sizes and that is an auto tuning of buffer sizes based
> on packet size but that hardly seems worth dealing with.

Sounds too platform specific and we're trying to include all IPSec 
devices, from PDA's to large military style encryption engines.

What if we do something like this :

A prerequisite will be that ingress and egress MTU have to be identical.

Then we send a round robin stream of the following packets :
  - TCP (minimum allowed packetsize on PHY) bytes
  - UDP 128 bytes
  - TCP 256 bytes
  - UDP 512 bytes
  - keep on alternating UDP/TCP, increase by 256 bytes
  - ....
  - UDP (max MTU) bytes

	Tim-


_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www1.ietf.org/mailman/listinfo/bmwg