Re: [secdir] secdir last call review of draft-ietf-bmwg-ipv6-tran-tech-benchmarking-07.txt

[Marius Georgescu <[e-mail]>]

Hello Taylor,

Thank you very much for reviewing the document.
Please find my comments inline.

> On May 8, 2017, at 4:34 AM, Taylor Yu <[e-mail]> wrote:
> 
> "MORTON, ALFRED C (AL)" <[e-mail]> writes:
> 
>> you wrote:
>>> Please consider replacing it with lowercase "should".  (I read it as
>>> predicting a correlation between the network security properties of the
>>> DUT in the lab environment and its behavior in a production environment,
>>> not as a guideline for implementors.)
>> 
>> This *is* a guideline to implementors, who are part of the intended
>> audience. We don't want testers to waste time benchmarking 
>> implementations that are for the lab-only; it is recommended
>> to test the systems intended for production (and such testing
>> will be safer in the isolated lab, of course).
> 
> I'm sorry, I guess I misinterpreted, then.  Do you mean that the
> implementations tested in the benchmarking lab should not materially
> differ from ones intended for production?  As in the tested
> implementations should be have neither stronger nor weaker security
> properties than their deployed counterparts?  I might be able to suggest
> improved wording if I understand your intentions correctly.
Indeed, we meant that the implementations tested in the lab should not differ from the ones intended for production.

> 
>> Also, if implementations have run-time error instrumentation,
>> so be it, but collecting this info is normally beyond the scope
>> of the blackbox lab texting of external phenomena.
> 
> I think there is an opportunity here to use instrumented versions of
> implementations to detect security-relevant errors and anomalies as they
> operate near their performance limits.  I also concede that sort of work
> might be outside the scope of this document.  (I can envision a related
> document that would cover such security-related topics.)
This sounds like an interesting benchmarking subject. However, I think it's outside the scope of this document and should remain that way.
Maybe a specific document dedicated to this subject is a better idea.

Best regards,
Marius