RE: [bmwg] New WG Deliverable Proposal - Core Router Software Acc elerated Life Testing

Jim McQuaid <jim.mcquaid@netiq.com> Wed, 07 May 2003 12:46 UTC

Received: from www1.ietf.org (ietf.org [132.151.1.19] (may be forged)) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA28322 for <bmwg-archive@lists.ietf.org>; Wed, 7 May 2003 08:46:33 -0400 (EDT)
Received: from www1.ietf.org (localhost.localdomain [127.0.0.1]) by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h47CtK812676; Wed, 7 May 2003 08:55:20 -0400
Received: from ietf.org (odin.ietf.org [132.151.1.176]) by www1.ietf.org (8.11.6/8.11.6) with ESMTP id h47Csu812630 for <bmwg@optimus.ietf.org>; Wed, 7 May 2003 08:54:56 -0400
Received: from ietf-mx (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id IAA28275 for <bmwg@ietf.org>; Wed, 7 May 2003 08:45:27 -0400 (EDT)
Received: from ietf-mx ([132.151.6.1]) by ietf-mx with esmtp (Exim 4.12) id 19DOKZ-0000pf-00 for bmwg@ietf.org; Wed, 07 May 2003 08:47:31 -0400
Received: from [65.222.200.22] (helo=ralexch01.netiq.local) by ietf-mx with esmtp (Exim 4.12) id 19DOKZ-0000pI-00 for bmwg@ietf.org; Wed, 07 May 2003 08:47:31 -0400
Received: by ralexch01.netiq.local with Internet Mail Service (5.5.2653.19) id <JQ9D800Q>; Wed, 7 May 2003 12:50:38 -0000
Message-ID: <613E3F060982754CBF2FC6751E82679B040DA17A@ralexch01.netiq.local>
From: Jim McQuaid <jim.mcquaid@netiq.com>
To: 'Kevin Dubray' <kdubray@juniper.net>, bmwg@ietf.org
Cc: "SENGUPTA,ANANDA (A-NewEngland,ex1)" <ananda_sengupta@agilent.com>
Subject: RE: [bmwg] New WG Deliverable Proposal - Core Router Software Acc elerated Life Testing
Date: Wed, 07 May 2003 12:49:37 -0000
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain; charset="iso-8859-1"
Sender: bmwg-admin@ietf.org
Errors-To: bmwg-admin@ietf.org
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.0.12
Precedence: bulk
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>

I haven't read the ID so this is semi-off the wall, I admit, but I think a
fundamental distinction that ought to be made is between DUT behavior and
DUT performance.  Behavior includes all sorts of operations (like
hotswap-ability and configuration mngt) that are important to operations BUT
not performance metrics in any reasonable sense of the word.  Performance is
more related to capacity planning in a sense, while behavior is related to
manageability and such.

Jim McQuaid
NetIQ Raleigh

-----Original Message-----
From: Kevin Dubray [mailto:kdubray@juniper.net]
Sent: Wednesday, May 07, 2003 8:36 AM
To: bmwg@ietf.org
Cc: SENGUPTA,ANANDA (A-NewEngland,ex1)
Subject: Re: [bmwg] New WG Deliverable Proposal - Core Router Software
Accelerated Life Testing


In section 3.4, the I-D proposes "evaluation benchmarks". (Since the
proponents produced an I-D, I'm going to assume the I-D's a more complete
statement of intent and scope. My question is, do these tests and
their collection scenario present good BMWG benchmarks?

Consider the ol' manual, cable pull test or its higher-end relative, the
module hotswap test. Experience tells us that these two tests are some
of the finest system-level tests known to routerkind.  You can even measure
some aspect of "recovery time".  But do they lend themselves to good 
"benchmarks"?

-Kevin



SENGUPTA,ANANDA (A-NewEngland,ex1) wrote:
> Well, this is the reason I was readily agreeing to the work on the
nomenclature draft, and then debating what tests fit in this WG in the
actual test plan draft. It is clear that there is value to this work, and
though all the test cases may not fit in this WG, some will.
> Ananda

_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www1.ietf.org/mailman/listinfo/bmwg
_______________________________________________
bmwg mailing list
bmwg@ietf.org
https://www1.ietf.org/mailman/listinfo/bmwg