Re: [bmwg] Feedback for Benchmarking Methodology for OpenFlow SDN Controller Performance

"MORTON, ALFRED C (AL)" <acmorton@att.com> Sat, 21 June 2014 12:30 UTC

Return-Path: <acmorton@att.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 551E11B2A6F for <bmwg@ietfa.amsl.com>; Sat, 21 Jun 2014 05:30:19 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -4.852
X-Spam-Level:
X-Spam-Status: No, score=-4.852 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.651, SPF_HELO_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F9NX0WJ-12bV for <bmwg@ietfa.amsl.com>; Sat, 21 Jun 2014 05:30:17 -0700 (PDT)
Received: from mail-red.research.att.com (mail-red.research.att.com [204.178.8.23]) by ietfa.amsl.com (Postfix) with ESMTP id C0F291B2799 for <bmwg@ietf.org>; Sat, 21 Jun 2014 05:30:17 -0700 (PDT)
Received: from mail-blue.research.att.com (unknown [135.207.178.11]) by mail-red.research.att.com (Postfix) with ESMTP id 6BC9A554BB9; Sat, 21 Jun 2014 08:33:44 -0400 (EDT)
Received: from njfpsrvexg8.research.att.com (unknown [135.207.255.243]) by mail-blue.research.att.com (Postfix) with ESMTP id EA842EF85B; Sat, 21 Jun 2014 08:30:16 -0400 (EDT)
Received: from NJFPSRVEXG8.research.att.com ([fe80::cdea:b3f6:3efa:1841]) by njfpsrvexg8.research.att.com ([fe80::cdea:b3f6:3efa:1841%13]) with mapi; Sat, 21 Jun 2014 08:30:16 -0400
From: "MORTON, ALFRED C (AL)" <acmorton@att.com>
To: "Castelli, Brian" <Brian.Castelli@spirent.com>, "Banks, Sarah" <sbanks@akamai.com>, "Bhuvan (Veryx Technologies)" <bhuvaneswaran.vengainathan@veryxtech.com>, "bmwg@ietf.org" <bmwg@ietf.org>
Date: Sat, 21 Jun 2014 08:30:15 -0400
Thread-Topic: [bmwg] Feedback for Benchmarking Methodology for OpenFlow SDN Controller Performance
Thread-Index: AQHPhQ0wPhnX4rvCTUmBdMpJ6z8vUpt45mmAgAIWN4CAAI1nkA==
Message-ID: <2845723087023D4CB5114223779FA9C801896A83AD@njfpsrvexg8.research.att.com>
References: <CFB4B28A.541E%brian.castelli@spirent.com> <000001cf80d4$2fabbd00$8f033700$@veryxtech.com> <D21535D16E7F464EBA75B9CA12A7DFFD205C1B43@SPCCOREXCMBX01.AD.SPIRENTCOM.COM> <CFC82A01.5BC1%sbanks@akamai.com> <CFC86AB3.6288%brian.castelli@spirent.com>
In-Reply-To: <CFC86AB3.6288%brian.castelli@spirent.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
acceptlanguage: en-US
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Archived-At: http://mailarchive.ietf.org/arch/msg/bmwg/Glchqbvg6F7vOUc0ug3ACyztcNg
Cc: 'Anton Basil' <anton.basil@veryxtech.com>, 'Vishwas Manral' <vishwas.manral@gmail.com>
Subject: Re: [bmwg] Feedback for Benchmarking Methodology for OpenFlow SDN Controller Performance
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 21 Jun 2014 12:30:19 -0000

Hi Brian, Sarah, Bhuvan, and all,

a couple of quick answers and a comment:

> -----Original Message-----
> From: bmwg [mailto:bmwg-bounces@ietf.org] On Behalf Of Castelli, Brian
...
> My two main concerns are getting the terminology right and making sure
> that the proposed set of tests will achieve our goals. I believe those
> goals ought to include a common taxonomy, enabling apples-to-apples
> comparisons, and minimizing the time/work required to execute the test
> cases.

Great, that's exactly what we are chartered to do in BMWG.
http://datatracker.ietf.org/wg/bmwg/charter/


> 
> I am willing to help improve the benchmark. I understand that it is an
> early draft. The time is right to make sure that we are developing the
> best specification that we can.
> 
> How do we continue this process? Via email to this list? I am willing to
> help.

It's principally e-mail (just as you've been doing) and face-to-face meetings
three times a year.  Sarah mentions that our next meeting is in her home town:
http://www.ietf.org/meeting/90/index.html

Make some text proposals for the draft, and we can discuss them on the list.

Having said that, and recognizing that there appears to be a comparable 
activity in ONF that (I assume) is OpenFlow-specific, perhaps a way to 
add value to the industry is to approach the SDN controller problem
more generically, such that different controller designs performing the 
same tasks could be benchmarked as black-boxes.  I realize this approach
has substantial implications for the draft, but the benefit of wider 
applicability.

food for thought,
Al
(as a participant)