Re: [bmwg] I-D Action: draft-ietf-bmwg-ca-bench-meth-02.txt

Chuck McAuley <cmcauley@breakingpoint.com> Thu, 02 August 2012 16:37 UTC

Return-Path: <cmcauley@breakingpoint.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3AFD611E80F6 for <bmwg@ietfa.amsl.com>; Thu, 2 Aug 2012 09:37:08 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.599
X-Spam-Level:
X-Spam-Status: No, score=-2.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FC+rJ1nAcdWE for <bmwg@ietfa.amsl.com>; Thu, 2 Aug 2012 09:37:07 -0700 (PDT)
Received: from mail.breakingpoint.com (mail.breakingpoint.com [65.36.7.12]) by ietfa.amsl.com (Postfix) with ESMTP id 890E011E808E for <bmwg@ietf.org>; Thu, 2 Aug 2012 09:37:07 -0700 (PDT)
Received: from EXCHANGE.securitytestsystems.com ([::1]) by EXCHANGE.securitytestsystems.com ([::1]) with mapi id 14.01.0355.002; Thu, 2 Aug 2012 11:37:00 -0500
From: Chuck McAuley <cmcauley@breakingpoint.com>
To: Al Morton <acmorton@att.com>, Tom Alexander <talexander@ixiacom.com>, Mike Hamilton <mhamilton@breakingpoint.com>, "bmwg@ietf.org" <bmwg@ietf.org>, "Sarah Banks (sabanks)" <sabanks@cisco.com>
Thread-Topic: [bmwg] I-D Action: draft-ietf-bmwg-ca-bench-meth-02.txt
Thread-Index: AQHNcE9DvuVKWBCiEESe/HdbK9J9k5dGyjYA
Date: Thu, 02 Aug 2012 16:37:00 +0000
Message-ID: <CC401E78.167CD%cmcauley@breakingpoint.com>
In-Reply-To: <201208020137.q721b9BA002992@alpd052.aldc.att.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
user-agent: Microsoft-MacOutlook/14.2.3.120616
x-originating-ip: [172.16.10.90]
Content-Type: text/plain; charset="us-ascii"
Content-ID: <4794E6A42B496D46BD3359DE479F53B7@breakingpoint.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Subject: Re: [bmwg] I-D Action: draft-ietf-bmwg-ca-bench-meth-02.txt
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 02 Aug 2012 16:37:08 -0000

Al,

There are other things that can impact application traffic behavior beyond
a dropped packet.

Off the top of my head:
1. Out of order packets from load balanced connections
2. Layer 7 Content Manipulation
3. Higher average latency of one stateful connection over another.  IE one
transaction can get "ahead" at the cost of another, altering all traffic
behavior after that point. This is similar to #1.

Chuck McAuley

On 8/1/12 9:36 PM, "Al Morton" <acmorton@att.com> wrote:

>At 02:21 PM 7/31/2012, Tom Alexander wrote:
>>Hello, Mike,
>>
>>Thanks for incorporating my feedback. The -03 draft looks very good
>>and I have no further comments.
>>
>>Best regards,
>>
>>- Tom
>
>Mike and Sarah,
>
>I read the draft on the flight-out, and have the following comments.
>(these are comments on the version 02 text, sorry I couldn't keep
>up with your really fast update...)
>
>Al (as a participant)
>
>-=-=-=-=-
>
>end of Section 1:
>    ...
>    The nature of application-driven networks is such that a single
>    dropped packet inherently changes the input stimulus from a network
>    perspective.  While application flows will be specified in great
>    detail, it simply is not practical to require totally repeatable
>    input stimulus.
>
>It seems that the input stimulus *should* be repeatable when there
>is no loss.  Whether the measured results are repeatable or not depends
>on the DUT.
>
>When there is loss (in overload) the input will change (TCP connections
>will react, etc.), and the offered load will change thus the measured
>result will change. Therefore several trials at the same conditions will
>be needed along with averaging.
>
>