Re: [secdir] [bmwg] Secdir telechat review of draft-ietf-bmwg-b2b-frame-03

Mališa Vučinić <> Tue, 15 December 2020 14:21 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 658713A1146; Tue, 15 Dec 2020 06:21:11 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: 0.001
X-Spam-Status: No, score=0.001 tagged_above=-999 required=5 tests=[MIME_QP_LONG_LINE=0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id 4QUndsTNkAKx; Tue, 15 Dec 2020 06:21:09 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 11A1C3A1142; Tue, 15 Dec 2020 06:21:07 -0800 (PST)
X-IronPort-AV: E=Sophos;i="5.78,421,1599516000"; d="scan'208";a="483101263"
Received: from (HELO []) ([]) by with ESMTP/TLS/AES256-GCM-SHA384; 15 Dec 2020 15:21:01 +0100
User-Agent: Microsoft-MacOutlook/
Date: Tue, 15 Dec 2020 15:20:59 +0100
From: =?UTF-8?B?TWFsacWhYQ==?= =?UTF-8?B?IFZ1xI1pbmnEhw==?= <>
To: "MORTON, ALFRED C (AL)" <>, "" <>
CC: "" <>, "" <>, "" <>
Message-ID: <>
Thread-Topic: [bmwg] Secdir telechat review of draft-ietf-bmwg-b2b-frame-03
References: <> <>
In-Reply-To: <>
Mime-version: 1.0
Content-type: text/plain; charset="UTF-8"
Content-transfer-encoding: quoted-printable
Archived-At: <>
Subject: Re: [secdir] [bmwg] Secdir telechat review of draft-ietf-bmwg-b2b-frame-03
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Security Area Directorate <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Tue, 15 Dec 2020 14:21:12 -0000

Hi Al,

Thanks, that is clear. I think that discussing the assumption of honesty among the parties involved in benchmarking  would be a useful addition to the Security Considerations section in the draft.


On 15/12/2020 14:45, "MORTON, ALFRED C (AL)" <> wrote:

    Hi Mališa, 
    thanks for your review, please see below for one reply to your question (acm].
    > -----Original Message-----
    > From: bmwg [] On Behalf Of Mališa Vucinic via
    > Datatracker
    > Sent: Tuesday, December 15, 2020 6:30 AM
    > To:
    > Cc:;; draft-ietf-bmwg-b2b-
    > Subject: [bmwg] Secdir telechat review of draft-ietf-bmwg-b2b-frame-03
    > Reviewer: Mališa Vučinić
    > Review result: Ready
    > I reviewed this document as part of the Security Directorate's ongoing
    > effort
    > to review all IETF documents being processed by the IESG. These comments
    > were
    > written primarily for the benefit of the Security Area Directors. Document
    > authors, document editors, and WG chairs should treat these comments just
    > like
    > any other IETF Last Call comments.
    > Thank you for this well-written document, it was a pleasure to read and I
    > think
    > it is ready to proceed. Since the document updates RFC2544 benchmarking
    > procedure for estimating the buffer time of a Device Under Test (DUT), it
    > does
    > not raise any security issues. Security Considerations section is quite
    > clear
    > and it stresses that these tests are performed in a lab environment.
    > I do have a question regarding the last paragraph of the Security
    > Considerations on special capabilities of DUTs for benchmarking purposes.
    > Currently, the sentence reads: "Special capabilities SHOULD NOT exist in
    > the
    > DUT/SUT specifically for benchmarking purposes." Why is this a SHOULD NOT
    > and
    > not a MUST NOT? Could you give an example when such special capabilities
    > in a
    > DUT are appropriate?
    We can only make a strong recommendation in this area. As testers/benchmarkers are often independent from the DUT developers and conduct testing external to the DUT, we assume honesty among other parties but we cannot require it. If someone constructed a DUT that recognized test conditions and operated differently to perform better somehow, our tests would measure the intended "better" performance. It takes a special/additional test effort to prove that a DUT has "designed to the test" (consider Volkswagen and fuel efficiency testing [0]).
    We simply do not have any authority in this matter, but we can let all parties know that gaming the test can be discovered and reported (albeit with more testing that we do not describe).
    > _______________________________________________
    > bmwg mailing list
    > !BhdT!1JFeLsENzMU-ew89jxmJKxfp4wj5Zo3AZ6V8iULU3hWAentH1dymqJmDOvw7$