Re: [bmwg] AD Review of draft-ietf-bmwg-ngfw-performance

I also, wanted to chime in regarding this and again, I also apologize for
not reviewing earlier.

I've not really been too involved in security devices until the last couple
years, but boy has it changed
drastically from when I was.  Also, I noted that this document is basically
following prior standards that
are also a bit out dated.

Some of the evolution, I've noticed which I think should at least be
addressed in.  I think you tried
to exclude these in the Scope.?  However I'm not really sure you are
covering but maybe 1/2 of the devices
that are being deployed at this time without addressing:

Virtualization. /  Remote FWs in the next gen.

*) Large FWs are compartmentalizing Customers and serving multiple
customers with the same firewall.  So while
you have said what would happen to a customer when The Firewall is UT, you
don't really address what's going
on with all the other customers on this DUT.  If there are 10 Customers
going through 1 large FW, what are the
other 9 experiencing when this is under Test.  Would be interesting to have
multiple parallel tests underway
and verify that behaviour is consistent across every "VDOM"  within the
system?

*) Time to Change.  If things are changed during your Load profile.perhaps
even on a different virtual Firewall
within the system,

*) Sandbox dips.  Several of the firewall vendors send stuff through labs /
temp work spaces either online or
offline that allow them to test against Zero Day scenarios.  What happens
to traffic flows during anomalies
in the traffic flows.  Does it change the throughput of the DUT?  I know
this is kind of an open ended question.
but not sure you have good benchmark stats for a FW without it...

*) Cloud based scenarios.  If the firewall is removed, and I'm not really
sure how you would test this with
consistent results, but if the FW is offsite and the traffic is being
routed through a Cloud based FW  What does
that do to your traffic results?

TimC
Fruth Group
tcopley@fruthgroup.com

On Wed, Dec 15, 2021 at 8:38 AM Warren Kumari <warren@kumari.net> wrote:

> Hi there all,
>
> I'd like to apologize to the authors and WG for how long it has taken me
> to post this review; I read the document a while back, but forgot to post
> it :-(
>
> I only had a few editorial / readability suggestions -- they are
> sufficiently minor (and I'm embarrassed by how long my review took!) that
> I'll kick off LC without asking for a new version -- authors, please
> consider them if you post a new version addressing LC comments, or to
> address IESG Eval comments.
>
> Chairs / Al: The document state is listed as: "Doc Shepherd Follow-up
> Underway" - I'm assuming that this is from:
> https://mailarchive.ietf.org/arch/msg/bmwg/EonLD10nf0xQfWzHt0YT0QDK6lU -
> can you please confirm (LOUDLY) that this is OK now and I'll kick off LC.
>
>
> W
>
>
> 1.  Introduction
>
>    18 years have passed since IETF recommended test methodology and
>    terminology for firewalls initially ([RFC3511]).  The requirements
>    for network security element performance and effectiveness have
>    increased tremendously since then.  Security function implementations
>
> [O] 18 years have passed since IETF recommended test methodology and
>
>    terminology for firewalls initially ([RFC3511]).  The requirements
>    for network security element performance and effectiveness have
>    increased tremendously since then.
> [P] In the eighteen years since [RFC3511] was published, recommending test methodology and terminology for firewalls, requirements and expectations for network security elements has increased tremendously.
>
>    have evolved to more advanced areas and have diversified into
>    intrusion detection and prevention, threat management, analysis of
>    encrypted traffic, etc.  In an industry of growing importance, well-
>    defined, and reproducible key performance indicators (KPIs) are
>    increasingly needed as they enable fair and reasonable comparison of
>
> [O] as they enable
> [P] to enable
>
>    network security functions.
>
>
>
>
> --
> Perhaps they really do strive for incomprehensibility in their specs.
> After all, when the liturgy was in Latin, the laity knew their place.
> -- Michael Padlipsky
> _______________________________________________
> bmwg mailing list
> bmwg@ietf.org
> https://www.ietf.org/mailman/listinfo/bmwg
>

-- 
Timothy Copley
602.350.0633