Re: [bmwg] Secdir telechat review of draft-ietf-bmwg-b2b-frame-03

["MORTON, ALFRED C (AL)" <[e-mail]>]

Hi Mališa,

Thanks for your proposed wording, it seems sufficiently neutral and with a few small tweaks, WFM.

I see that Roman's COMMENT also supports this additional text.

So, consider it part of the next version, and thanks for your help!
Al


> -----Original Message-----
> From: Mališa Vučinić [mailto:[e-mail]]
> Sent: Wednesday, December 16, 2020 7:22 AM
> To: MORTON, ALFRED C (AL) <[e-mail]>; [e-mail]
> Cc: [e-mail]; [e-mail]; draft-ietf-bmwg-b2b-
> [e-mail]
> Subject: Re: [bmwg] Secdir telechat review of draft-ietf-bmwg-b2b-frame-03
> 
> Al,
> 
> I don't have a strong opinion on using the term "honesty" here. How about
> this phrasing, just before the last paragraph in Security Considerations:
> 
> The DUT developers are commonly independent from the personnel and
> institutions conducting the benchmarking.
> The DUT developers might have incentives to alter the performance of the
> DUT if the test conditions are detected.
> Procedures described in this document are not designed to detect such
> activity.
> Additional testing, outside of the scope of this document, is needed and
> has been successfully used in the past to discover such malpractices.
> 
> Mališa
> 
> On 15/12/2020 20:22, "MORTON, ALFRED C (AL)" <[e-mail]> wrote:
> 
>     Hi Mališa,
>     please see below...
> 
>     > -----Original Message-----
>     > From: Mališa Vučinić [mailto:[e-mail]]
>     > Sent: Tuesday, December 15, 2020 9:21 AM
>     > To: MORTON, ALFRED C (AL) <[e-mail]>; [e-mail]
>     > Cc: [e-mail]; [e-mail]; draft-ietf-bmwg-b2b-
>     > [e-mail]
>     > Subject: Re: [bmwg] Secdir telechat review of draft-ietf-bmwg-b2b-
> frame-03
>     >
>     > Hi Al,
>     >
>     > Thanks, that is clear. I think that discussing the assumption of
> honesty
>     > among the parties involved in benchmarking  would be a useful
> addition to
>     > the Security Considerations section in the draft.
>     [acm]
> 
>     I don't mind explaining the requirement using the term "honesty", but
> I can only imagine raised eyebrows and subsequent DISCUSS/comments if we
> try to assert a need for/assumption of honesty anywhere in the memo.
> 
>     Do you have suggested wording?
> 
>     Do others have opinions whether or not this is needed?
> 
>     thanks,
>     Al
> 
>     >
>     > Mališa
>     >
>     > On 15/12/2020 14:45, "MORTON, ALFRED C (AL)" <[e-mail]>
> wrote:
>     >
>     >     Hi Mališa,
>     >     thanks for your review, please see below for one reply to your
>     > question (acm].
>     >     Al
>     >
>     >     > -----Original Message-----
>     >     > From: bmwg [mailto:[e-mail]] On Behalf Of Mališa
>     > Vucinic via
>     >     > Datatracker
>     >     > Sent: Tuesday, December 15, 2020 6:30 AM
>     >     > To: [e-mail]
>     >     > Cc: [e-mail]; [e-mail]; draft-ietf-bmwg-b2b-
>     >     > [e-mail]
>     >     > Subject: [bmwg] Secdir telechat review of draft-ietf-bmwg-b2b-
> frame-
>     > 03
>     >     >
>     >     > Reviewer: Mališa Vučinić
>     >     > Review result: Ready
>     >     >
>     >     > I reviewed this document as part of the Security Directorate's
>     > ongoing
>     >     > effort
>     >     > to review all IETF documents being processed by the IESG.
> These
>     > comments
>     >     > were
>     >     > written primarily for the benefit of the Security Area
> Directors.
>     > Document
>     >     > authors, document editors, and WG chairs should treat these
> comments
>     > just
>     >     > like
>     >     > any other IETF Last Call comments.
>     >     >
>     >     > Thank you for this well-written document, it was a pleasure to
> read
>     > and I
>     >     > think
>     >     > it is ready to proceed. Since the document updates RFC2544
>     > benchmarking
>     >     > procedure for estimating the buffer time of a Device Under
> Test
>     > (DUT), it
>     >     > does
>     >     > not raise any security issues. Security Considerations section
> is
>     > quite
>     >     > clear
>     >     > and it stresses that these tests are performed in a lab
> environment.
>     >     >
>     >     > I do have a question regarding the last paragraph of the
> Security
>     >     > Considerations on special capabilities of DUTs for
> benchmarking
>     > purposes.
>     >     > Currently, the sentence reads: "Special capabilities SHOULD
> NOT
>     > exist in
>     >     > the
>     >     > DUT/SUT specifically for benchmarking purposes." Why is this a
>     > SHOULD NOT
>     >     > and
>     >     > not a MUST NOT? Could you give an example when such special
>     > capabilities
>     >     > in a
>     >     > DUT are appropriate?
>     >     [acm]
>     >     We can only make a strong recommendation in this area. As
>     > testers/benchmarkers are often independent from the DUT developers
> and
>     > conduct testing external to the DUT, we assume honesty among other
> parties
>     > but we cannot require it. If someone constructed a DUT that
> recognized
>     > test conditions and operated differently to perform better somehow,
> our
>     > tests would measure the intended "better" performance. It takes a
>     > special/additional test effort to prove that a DUT has "designed to
> the
>     > test" (consider Volkswagen and fuel efficiency testing [0]).
>     >
>     >     We simply do not have any authority in this matter, but we can
> let all
>     > parties know that gaming the test can be discovered and reported
> (albeit
>     > with more testing that we do not describe).
>     >
>     >     [0]
> https://urldefense.com/v3/__https://www.consumerreports.org/fuel-
>     > economy-efficiency/volkswagen-used-special-software-to-exaggerate-
> fuel-
>     > economy/__;!!BhdT!0KS_VCF5ZQfIGkVyPLoJXuAxdcoS3-
>     > xJTE0LoKZPWuSiHjQZM1u0H9M36YXByCk$
>     >
>     >     >
>     >     >
>     >     >
>     >     > _______________________________________________
>     >     > bmwg mailing list
>     >     > [e-mail]
>     >     >
>     >
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/bmwg__;!
>     >     > !BhdT!1JFeLsENzMU-
> ew89jxmJKxfp4wj5Zo3AZ6V8iULU3hWAentH1dymqJmDOvw7$
>     >
>     >
> 
> 
>