[bmwg] Review of draft-ietf-bmwg-sdn-controller-benchmark-meth-07

Henning Rogge <hrogge@gmail.com> Mon, 29 January 2018 11:52 UTC

Return-Path: <hrogge@gmail.com>
X-Original-To: bmwg@ietfa.amsl.com
Delivered-To: bmwg@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 83C5612D95B; Mon, 29 Jan 2018 03:52:04 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2
X-Spam-Level:
X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id MskGhNTOVUmY; Mon, 29 Jan 2018 03:51:57 -0800 (PST)
Received: from mail-qt0-x235.google.com (mail-qt0-x235.google.com [IPv6:2607:f8b0:400d:c0d::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 13DCB12EABC; Mon, 29 Jan 2018 03:51:48 -0800 (PST)
Received: by mail-qt0-x235.google.com with SMTP id s39so12253779qth.7; Mon, 29 Jan 2018 03:51:48 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=A6OjCUeVCmQ182aNjXY2gtIS4eyNqhp8q5RaDdr3bn4=; b=Cx66RJeEjgRJTANodzAN2MU0MKkscVncuHaXZZzT+TGvfXo3gIPNkNKzH9AlpStqIu 0US2s9EFXFuWrJR7pB3uf4xiEWDTpRjpDvTnSiO5Z8cYgonzJnj01U2aCcJiCgSpKbOq eKQKCDHt/985cMvFs+UyCKwkiyd4mmQom+/gR9N7eqcmphUP7Ae0f85aUkY38N+UpPBE RmKpca5zeV5Vn2JA2ShiK835q4pciKu0R7Tt2WddouPQ5dnbYjhi3K4eIEAe2GkHa6UR CRb2jiSQxh4CP+4g2VADFPlYKoLQGM/zpAuuebmV9JHfeDfDFg4Y9e+nJ7RhKYcmbgFK R9KA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=A6OjCUeVCmQ182aNjXY2gtIS4eyNqhp8q5RaDdr3bn4=; b=oHnWH2LK3XFscKpeeHw9z9GTVf9KRPJ+BrSklnyClM57D2u44RTLVrvlJBvwk0UXIP 2tjxfYH185Aa78vH3ned+WttxIOV7RKCsOYhXdhfE8iXGddna02U7b00koe+0GUxWN0z hvPBvD9VLTRdD4Z9kS6Bad3R6N8mz2pd/xFzQ/fJu/pBIv8+qfaUiFpUAzX6Ize80Ted BUf4x+cA43sP7j/glA/kjIVb04FA6sRMY2fhvpAC0GiIbjGfFjjuKF0aizWJIDnBtTGh GqvN9gqOIzHaBUCUS29xJvpHrlphsFFqTmkStDZ02BDpVRc6SO7aeVOYbW1acoOvJHkB NwLA==
X-Gm-Message-State: AKwxytdIz/EUEfBC1G07m5x3iApR8EGzzy9UjmYFOsMfsa0kWkaVWrYW KbR+vPJlEkZQvOgemDWF6HudKYhgMZy1Yw7+ETJ9em12
X-Google-Smtp-Source: AH8x227af7rsTcEcEIXqjyzn8FgUoGCQ4Tn8rUdpcvHPoLYCaqL8N2es3dTRJUXeBI8UQqLuMWnjTSmDxoQuS0gZhW4=
X-Received: by 10.200.13.139 with SMTP id s11mr39565029qti.287.1517226706958; Mon, 29 Jan 2018 03:51:46 -0800 (PST)
MIME-Version: 1.0
Received: by 10.237.53.23 with HTTP; Mon, 29 Jan 2018 03:51:16 -0800 (PST)
From: Henning Rogge <hrogge@gmail.com>
Date: Mon, 29 Jan 2018 12:51:16 +0100
Message-ID: <CAGnRvuqNP5X5=WqqghjqiERgU-F6fCQvJYun07OfN5Yb0Fhwtg@mail.gmail.com>
To: "<rtg-ads@ietf.org>" <rtg-ads@ietf.org>
Cc: rtg-dir@ietf.org, draft-ietf-bmwg-sdn-controller-benchmark-meth@ietf.org, bmwg@ietf.org
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://mailarchive.ietf.org/arch/msg/bmwg/Ork44BqLNteFdTBjQmzBKliLmzg>
Subject: [bmwg] Review of draft-ietf-bmwg-sdn-controller-benchmark-meth-07
X-BeenThere: bmwg@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Benchmarking Methodology Working Group <bmwg.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/bmwg>, <mailto:bmwg-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/bmwg/>
List-Post: <mailto:bmwg@ietf.org>
List-Help: <mailto:bmwg-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/bmwg>, <mailto:bmwg-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 29 Jan 2018 11:52:04 -0000

Hi,

Min Ye asked me to do a review of
draft-ietf-bmwg-sdn-controller-benchmark-meth-07.


I first have to say I like the fact that the draft has a "sequence
diagram" for each of the test cases in the appendix. This will make it
easier to design test software without forgetting small things or
misinterpreting the text version of the test.

Second, in chapter 5.1.1 you might want to consider that topology
discovery can have transient results that might be (accidentally) the
correct one. Just because the you get the right topology once it
doesn't mean that the algorithm is already in a stable state. I would
suggest keeping the "test three times for same result" also for the
correct topology.

Similar in chapter 5.2.2, it might be good to compare the results of
procedure step 3 and 4... see if they have a different of one of more.
I am not sure if a test result should be just invalid if the
difference is larger or if the difference should recorded in the
results.

Last, the Table of Content link to chapter 3 and the "section 10" link
in chapter 4.1 seem to be broken.

Henning Rogge