Re: [bmwg] Spencer Dawkins' No Objection on draft-ietf-bmwg-sdn-controller-benchmark-meth-08: (with COMMENT)
Spencer Dawkins at IETF <spencerdawkins.ietf@gmail.com> Mon, 16 April 2018 20:07 UTC
Return-Path: <spencerdawkins.ietf@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id BFF4112704A; Mon, 16 Apr 2018 13:07:38 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.699
X-Spam-Level:
X-Spam-Status: No, score=-2.699 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 9k-j6n7Kvpeh; Mon, 16 Apr 2018 13:07:36 -0700 (PDT)
Received: from mail-yw0-x233.google.com (mail-yw0-x233.google.com [IPv6:2607:f8b0:4002:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id DAC4D126FB3; Mon, 16 Apr 2018 13:07:35 -0700 (PDT)
Received: by mail-yw0-x233.google.com with SMTP id q12so7079334ywj.0; Mon, 16 Apr 2018 13:07:35 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=8BtqLYU3uQLUJs+EYbUVW3udcE4uVXlABykTige65kE=; b=HjkWoaHBYfP9vq6AHz/oyMenFWxpohvaPcdGclU5VhV96M5GsjEKucOFO6EueC6PPa sON+JgzEEohoM6beQ/6wx3o2vju/KJcYMQgMdqhj0qEN4dKfr9FxlQpNbyIKMo5UDabj eiVwxjZZx7rH4MBZESHvYY6taEUDxudy4Zrpriv6235kaPFgDhtE8a4dMi9w8URczEEb /QhHjl87rwZQiMZgfnH6oqcYd/P2XgXQROcpRy/uoiUJEvU/Y3II8lqh+2agnnraAtdJ Lp36qgUsmr5sprAAfQ0vg55yaD2RT21Qz+g3gcLnMd3VJTonps6J9yuby41V7FI/KGMg 6I+g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=8BtqLYU3uQLUJs+EYbUVW3udcE4uVXlABykTige65kE=; b=pV9u8zpAm9PHKO5fYFxUD9mVTjQvd31SDCGGKsk3qgc8lbxcokM6+fpyx+keDgexMW IzJNu1IFe5x+5zM3GK8Gid0bt5FUx3V6AleZPSqinshe0gDBvlPX3NyKmU9PVEVEcGw4 eOJHco9045vcu4d3aRx71GhGxXhE8mUqDOBGrRZpSBSgbFiuFwUd9W3bKGHAPK8bCIrp GVCvreyktimNCOCFz1INH9lD5kyQN42lzwcn5++3xDl1Xt6izFtkfnE6S/5L7cvSB+4W nFHtRCV01j3bciAFvVc3geLv9s1NOwkx30mpGWf86k4TJOUgbZZvP+VrlBGOlbd7LIAJ jd7Q==
X-Gm-Message-State: ALQs6tA6zVrTMGClFkQJsItIJCrfJj901QnEdYJn7TfB8tt9xNDFAHTJ pg3Gt9e5KB+RiwjqEVRiKmVJJY++Pz4LOB5So54=
X-Google-Smtp-Source: AIpwx4+1zg+AChJi6rQciYNZ1+KHYJvWyz8k4vEO1OZVsu/kAz5XMtERkQcFsR//FFonJJyR38C6Gopajt8RyGRoEtc=
X-Received: by 10.129.236.20 with SMTP id j20mr2266710ywm.311.1523909254724; Mon, 16 Apr 2018 13:07:34 -0700 (PDT)
MIME-Version: 1.0
Received: by 2002:a5b:c06:0:0:0:0:0 with HTTP; Mon, 16 Apr 2018 13:07:34 -0700 (PDT)
In-Reply-To: <152390796412.19681.526742517809994752.idtracker@ietfa.amsl.com>
References: <152390796412.19681.526742517809994752.idtracker@ietfa.amsl.com>
From: Spencer Dawkins at IETF <spencerdawkins.ietf@gmail.com>
Date: Mon, 16 Apr 2018 15:07:34 -0500
Message-ID: <CAKKJt-dC2CBUoOy6raYjN7PfW3M8Qrg5o4edkJVA28zoXXdF_g@mail.gmail.com>
To: The IESG <iesg@ietf.org>
Cc: draft-ietf-bmwg-sdn-controller-benchmark-meth@ietf.org, bmwg-chairs@ietf.org, Al Morton <acmorton@att.com>, bmwg@ietf.org
Content-Type: multipart/alternative; boundary="089e082d6cd8cc42270569fcc5d6"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/mdNa6dKrT3-SQ2siQ0cDl8Cco9Y>
Subject: Re: [bmwg] Spencer Dawkins' No Objection on draft-ietf-bmwg-sdn-controller-benchmark-meth-08: (with COMMENT)
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 16 Apr 2018 20:07:39 -0000
"Text is hard" ... please see correction below. On Mon, Apr 16, 2018 at 2:46 PM, Spencer Dawkins < spencerdawkins.ietf@gmail.com> wrote: > Spencer Dawkins has entered the following ballot position for > draft-ietf-bmwg-sdn-controller-benchmark-meth-08: No Objection > > When responding, please keep the subject line intact and reply to all > email addresses included in the To and CC lines. (Feel free to cut this > introductory paragraph, however.) > > > Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html > for more information about IESG DISCUSS and COMMENT positions. > > > The document, along with other ballot positions, can be found here: > https://datatracker.ietf.org/doc/draft-ietf-bmwg-sdn- > controller-benchmark-meth/ > > > > ---------------------------------------------------------------------- > COMMENT: > ---------------------------------------------------------------------- > > I have a few questions, at the No Objection level ... do the right thing, > of > course. > > I apologize for attempting to play amateur statistician, but it seems to me > that this text > > 4.7. Test Repeatability > > To increase the confidence in measured result, it is recommended > that each test SHOULD be repeated a minimum of 10 times. > > is recommending a heuristic, when I'd think that you'd want to repeat a > test > until the results seem to be converging on some measure of central > tendency, > given some acceptable margin of error, and this text > > Procedure: > > 1. Establish the network connections between controller and network > nodes. > 2. Query the controller for the discovered network topology > information and compare it with the deployed network topology > information. > 3. If the comparison is successful, increase the number of nodes by 1 > and repeat the trial. > If the comparison is unsuccessful, decrease the number of nodes by > 1 and repeat the trial. > 4. Continue the trial until the comparison of step 3 is successful. > 5. Record the number of nodes for the last trial (Ns) where the > topology comparison was successful. > > seems to beg for a binary search, especially if you're testing whether a > controller can support a large number of controllers ... > "Can support a large number of NODES" ... in all caps, so I remember it ... Spencer > This text > > Reference Test Setup: > > The test SHOULD use one of the test setups described in section 3.1 > or section 3.2 of this document in combination with Appendix A. > > or some variation is repeated about 16 times, and I'm not understanding why > this is using BCP 14 language, and if BCP 14 language is the right thing > to do, > I'm not understanding why it's always SHOULD. > > I get the part that this will help compare results, if two researchers are > running the same tests. Is there more to the requirement than that? > > In this text, > > Procedure: > > 1. Perform the listed tests and launch a DoS attack towards > controller while the trial is running. > > Note: > > DoS attacks can be launched on one of the following interfaces. > > a. Northbound (e.g., Query for flow entries continuously on > northbound interface) > b. Management (e.g., Ping requests to controller's management > interface) > c. Southbound (e.g., TCP SYN messages on southbound interface) > > is there a canonical description of "DoS attack" that researchers should be > using, in order to compare results? These are just examples, right? > > Is the choice of > > [OpenFlow Switch Specification] ONF,"OpenFlow Switch Specification" > Version 1.4.0 (Wire Protocol 0x05), October 14, 2013. > > intentional? I'm googling that the current version of OpenFlow is 1.5.1, > from > 2015. > > >
- [bmwg] Spencer Dawkins' No Objection on draft-iet… Spencer Dawkins
- Re: [bmwg] Spencer Dawkins' No Objection on draft… Spencer Dawkins at IETF
- Re: [bmwg] Spencer Dawkins' No Objection on draft… bhuvaneswaran.vengainathan