Re: [secdir] secdir last call review of draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07.txt

["MORTON, ALFRED C (AL)" <[e-mail]>]

Hi Taylor,
please see in-line reply, thanks,
Al

> -----Original Message-----
> From: Taylor Yu [mailto:[e-mail]]
> Sent: Thursday, May 11, 2017 6:55 PM
> To: Marius Georgescu
> Cc: MORTON, ALFRED C (AL); [e-mail]; [e-mail]; draft-ietf-
> [e-mail]
> Subject: Re: secdir last call review of draft-ietf-bmwg-ipv6-tran-tech-
> benchmarking-07.txt
> 
> Marius Georgescu <[e-mail]> writes:
> 
> >> On May 8, 2017, at 4:34 AM, Taylor Yu <[e-mail]> wrote:
> 
> >> I'm sorry, I guess I misinterpreted, then.  Do you mean that the
> >> implementations tested in the benchmarking lab should not materially
> >> differ from ones intended for production?  As in the tested
> >> implementations should be have neither stronger nor weaker security
> >> properties than their deployed counterparts?  I might be able to
> suggest
> >> improved wording if I understand your intentions correctly.
> 
> > Indeed, we meant that the implementations tested in the lab should not
> > differ from the ones intended for production.
> 
> In that case, what do you think of the following change?
> 
> OLD
> 
>    Further, benchmarking is performed on a "black-box" basis, relying
>    solely on measurements observable external to the DUT/SUT. Special
>    capabilities SHOULD NOT exist in the DUT/SUT specifically for
>    benchmarking purposes. Any implications for network security arising
>    from the DUT/SUT SHOULD be identical in the lab and in production
>    networks.
> 
> NEW
> 
>    Further, benchmarking is performed on a "black-box" basis, relying
>    solely on measurements observable external to the DUT.  Special
>    capabilities SHOULD NOT exist in the DUT specifically for
>    benchmarking purposes.  Testers and implementors SHOULD ensure that
>    the DUT has identical security properties to its production version.
[ACM] 
This new sentence has redundant effect with second sentence (unchanged).
Furthermore, Testing personnel have no means to ensure the identical 
properties, and implementers may not yet have a production version available
(Benchmarking often takes place with very early products, whose
non-tested aspects may change before reaching production).

>    For example, any security hardening or instrumentation present on the
>    DUT SHOULD also be present in the production version, and vice versa.
[ACM] 
Again, this cannot be ensured when it is not a tested feature, 
and there may be *many* (later) production releases that differ 
from the DUT. The example goes too far.

At this point, I feel it is appropriate to point out that BMWG has 
used the original wording of the paragraphs in this section for years
(~10), and their intent has been sufficiently clear to the community.


> 
> Best regards,
> -Taylor