Re: [bmwg] WGLC on New version of draft-ietf-bmwg-ngfw-performance

[Sarah Banks <[e-mail]>]

Hi Brian,
   I apologize, I'm super swamped at work right now and haven't had the bandwidth to read through these. I will aim to complete the review by Friday. My apologies for the delay, and thank you for the patience. 

Thanks,
Sarah


> On Jun 28, 2021, at 10:53 AM, <[e-mail]> <[e-mail]> wrote:
> 
> Sarah and Al,
> 
> Given that 15 days have gone by, may I safely assume that all of our
> comments/edits are accepted, and we can submit a new draft? That is hoping
> the new draft will pass muster easily.
> 
> Brian
> 
> -----Original Message-----
> From: [e-mail] <[e-mail]> 
> Sent: Saturday, June 12, 2021 7:25 AM
> To: 'Sarah Banks' <[e-mail]>; 'ALFRED MORTON' <[e-mail]>
> Cc: 'Gabor LENCSE' <[e-mail]>; 'Bala Balarajah'
> <[e-mail]>; 'Bala Balarajah' <[e-mail]>;
> [e-mail]; 'MORTON, ALFRED C (AL)' <[e-mail]>;
> [e-mail]; [e-mail]; 'Carsten
> Rossenhoevel' <[e-mail]>; 'Jack, Mike' <[e-mail]>;
> 'Christopher Brown' <[e-mail]>; 'Ryan Liles (ryliles)'
> <[e-mail]>
> Subject: RE: [bmwg] WGLC on New version of draft-ietf-bmwg-ngfw-performance
> 
> Hi Sarah,
> 
> As I mentioned in the previous message, we will remove reference to IDS from
> the draft. Given that, none of the IDS related comments/questions are being
> addressed.
> 
> Sorry for the delay in responding. I was unexpectedly out of the office. 
> 
> Our responses are below, preceded by [bpm]. However, please note that this
> is not a response from a single person, but from multiple people,
> representing security product vendors, test labs and test tool vendors.
> 
> Brian
> 
>>>> 
> 
> - The draft aims to replace RFC3511, but expands scope past Firewalls, to
> "next generation security devices". I'm not finding a definition of what a
> "next generation security device is", nor an exhaustive list of the devices
> covered in this draft. A list that includes is nice, but IMO not enough to
> cover what would be benchmarked here - I'd prefer to see a definition and an
> exhaustive list.
> 
> [bpm] "We avoid limiting the draft by explicitly adding a list of NG
> security devices currently available in the market only. In the future,
> there may be more and more new types of NG security devices that will appear
> on the market.
> 
> [bpm] This draft includes a list of security features that the security
> device can have ( RFC 3511 doesn't have such a list). Also, we will describe
> in the draft that the security devices must be configured ""in-line"" mode.
> We believe these two points qualifying the definition of next generation
> security.
> 
> - What is a NGIPS or NGIDS? If there are standardized definitions pointing
> to them is fine, otherwise, there's a lot of wiggle room here.
> 
> [bpm] See above. We are removing NGIDS from the draft.
> 
> - I still have the concern I shared at the last IETF meeting, where here,
> we're putting active inline security devices in the same category as passive
> devices. On one hand, I'm not sure I'd lump these three together in the
> first place; on the other, active inline devices typically include
> additional functions to allow administrators to control what happens to
> packets in the case of failure, and I don't see those test cases included
> here.
> 
> [bpm] This draft focuses on ""in-line"" mode security devices only. We will
> describe this in section 4 in more detail.
> 
> [bpm] Additionally, the draft focuses mainly on performance tests. The DUT
> must be configured in ""fail close"" mode. We will describe this under
> section 4. Any failure scenarios like ""fail open"" mode is out of scope.
> 
> - Section 4.1 - it reads as if ANY device in the test setup cannot
> contribute to network latency or throughput issues, including the DUTs - is
> that what you intended?
> 
> [bpm] "Our intention is, if the external devices (routers and switches) are
> used in the test bed, they should not negatively impact DUT/SUT performance.
> To address this, we added a section ( section 5 ""Test Bed Considerations"")
> which recommends a pre-test.  We can rename this as reference test or
> baseline test. "
> 
> - Option 1: It'd be nice to see a specific, clean, recommended test bed.
> There are options for multiple emulated routers. As a tester, I expect to
> see a specific, proscribed test bed that I should configure and test
> against. 
> 
> [bpm] The draft describes that Option 1 is the recommended test setup.
> However. We added emulated routers as optional in option 1. The reason for
> that:
> Some type of security devices for some deployment scenarios requires routers
> between test client/server and the DUT (e.g., NGFW) and some DUT/SUT doesn't
> need router (e.g. NGIPS )
> 
> - Follow on: I'm curious as to the choice of emulated routers here. The
> previous test suggests you avoid routers and switches in the topo, but then
> there are emulated ones here. I'm curious as to what advantages you think
> these bring over the real deal, and why they aren't subject to the same
> limitations previously described?
> 
> [bpm] Comparing real router, the emulated router gives more advantages for
> L7 testing.
> 
> [bpm] - Emulated router doesn't add latency. Even if it adds delay due to
> the routing process, the test equipment can report the added latency, or it
> can consider this for the latency measurement.
> 
> [bpm] - Emulated routers simply do routing function only. But in a "real"
> router, we are not sure what else the router is doing with the packets.
> 
> [bpm] Your question regarding the need for routers:
> 
> [bpm] - We avoid impacting the DUT/SUT performance due to ARP or ND process
> 
> [bpm] - Represent realistic scenario (In the production environment the
> security devices will not be directly connected with the clients.)
> 
> [bpm] - Routing (L3 mode) is commonly used in the NG security devices.
> 
> [bpm] However, in both figures we mentioned that router including emulated
> router is optional. If there is no need have routing functionality on the
> test bed (e.g., if we used very small number of clients and server IPs or
> the DUT operates in Layer 2 mode), it can be ignored.
> 
> [bpm] Also, we described in Option 1, that the external devices are if there
> is need to aggregate the interfaces of the tester or DUT.  For an example,
> DUT has 2 Interfaces, but tester need to use it's 4 interfaces to achieve
> the performance. So here we need switch/router to aggregate tester interface
> from 4 to 2.
> 
> - In section 4.1 the text calls out Option 1 as the preferred test bed,
> which includes L3 routing, but it's not clear why that's needed?
> 
> [bpm] See above.
> 
> - The difference between Option 1 and Option 2 is the inclusion of
> additional physical gear in Option 2 - it's not clear why that's needed, or
> why the tester can't simply directly connect the test equipment to the DUT
> and remove extraneous devices from potential influence on results?
> 
> [bpm] See above.
> 
> - Section 4.2, the table for NGFW features - I'm not sure what the
> difference is between RECOMMENDED and OPTIONAL? (I realize that you might be
> saying that RECOMMENDED is the "must have enabled" features, where as
> optional is at your discretion, but would suggest that you make that clear)
> 
> [bpm] The definition for OPTIONAL and RECOMMENDED is described in,  and
> recommended, RFC2119. We already referenced this under the section 2
> "Requirements".
> 
> - Proscribing a list of features that have to be enabled for the test, or at
> least more than 1, feels like a strange choice here - I'd have expected
> tests cases that either test the specific features one at a time, or suggest
> several combinations, but that ultimately, we'd tell the tester to document
> WHICH features were enabled, to make the test cases repeatable? This allows
> the tester to apply a same set of apples to apples configurations to
> different vendor gear, and omit the 1 feature that doesn't exist on a
> different NGFW (for example), but hold a baseline that could be tested.
> 
> - Table 2: With the assumption that NGIPS/IDS are required to have the
> features under "recommended", I disagree with this list. For example, some
> customers break and inspect at the tap/agg layer of the network - in this
> case, the feed into the NGIDS might be decrypted, and there's no need to
> enable SSL inspection, for example. 
> 
> [bpm] IDS is being removed.
> 
> - Table 3: I disagree that an NGIDS IS REQUIRED to decrypt SSL. This
> behaviour might be suitable for an NGIPS, but the NGIDS is not a bump on the
> wire, and often isn't decrypting and re-encrypting the traffic.
> 
> [bpm] IDS is being removed.
> 
> - Table 3: An NGIDS IMO is still a passive device - it wouldn't be blocking
> anything, but agree that it might tell you that it happened after the fact.
> 
> [bpm] IDS is being removed.
> 
> - Table 3: Anti-evasion definition - define "mitigates". 
> 
> [bpm] Not sure why you are asking this as mitigate is not an uncommon
> term/word. 
> 
> - Table 3: Web-filtering - not a function of an NGIDS.
> 
> [bpm] IDS is being removed.
> 
> - Table 3: DLP: Not applicable for an NGIDS.
> 
> [bpm] IDS is being removed.
> 
> - Can you expand on "disposition of all flows of traffic are logged" -
> what's meant here specifically, and why do they have to be logged? (Logging,
> particularly under high loads, will impact it's own performance marks, and
> colours output)
> 
> [bpm] We intentionally recommended enabling logging which will impact the
> performance. The draft is not aiming to get high performance number with
> minimal DUT/SUT configuration. In contrast, it aims to get reasonable
> performance number with realistic DUT configuration. The realistic
> configuration can vary based on DUT/SUT deployment scenario. 
> 
> [bpm] In most of the DUT/SUT deployment scenarios or customer environments,
> logging is enabled as default configuration.
> 
> [bpm] "Disposition of all flows of traffic are logged": means that the
> DUT/SUT need to log all the traffic at the flow level not each packet.
> 
> [bpm] We will add more clarification for the meaning of "disposition of all
> flows of traffic are logged".
> 
> - ACLs wouldn't apply to an IDS because IDS's aren't blocking traffic :)
> 
> [bpm] IDS is being removed.
> 
> - It might be helpful to testers to say something like "look, here's one
> suggested set of ACLs. If you're using them, great, reference that, but
> otherwise, make note of the ACLs you use, and use the same ones for
> repeatable testing".
> 
> [bpm] The draft gives guidance how to choose the ACL rules. We describe here
> a methodology to create ACL.  
> 
> - 4.3.1.1 The doc proscribes specific MSS values for v4/v6 with no
> discussion around why they're chosen - that color could be useful to the
> reader.
> 
> [bpm] We will add some more clarification that these are the default number
> used in most of the client operating systems currently.
> 
> - 4.3.1.1 - there's a period on the 3rd to last line "(SYN/ACL, ACK). and"
> that should be changed.
> 
> [bpm] Thank you.
> 
> - 4.3.1.1 - As a tester with long time experience with major test equipment
> manufacturers, I can't possibly begin to guess which ones of them would
> conform to this - or even if they'd answer these questions. How helpful is
> this section to the non test houses? I suggest expansion here, ideally with
> either covering the scope of what you expect to cover, or hopefully which
> (open source/generally available) test tools or emulators could be
> considered for use as examples.
> 
> [bpm] We extensively discussed with Ixia and Spirent about this section.
> This section was developed with significant input from these test tools
> vendors in addition to others.
> 
> - 4.3.1.3 - Do the emulated web browser attributes really apply to testing
> the NGIPS?
> 
> [bpm] Yes, we performed many PoC tests with test tools. Ixia and Spirent
> confirmed this.
> 
> - 4.3.2.3 - Do you expect to also leverage TLS 1.3 as a configuration option
> here?
> 
> [bpm] Yes
> 
> - 4.3.4 - I'm surprised to see the requirement that all sessions establish a
> distinct phase before moving on to the next. You might clarify why this is a
> requirement, and why staggering them is specifically rejected?
> 
> [bpm] This draft doesn't describe that all sessions establish a distinct
> phase before moving on to the next. We will remove the word "distinct" from
> the 1st paragraph in section 4.3.4.
> 
> [bpm] Unlike Layer 2/3 testing, Layer 7 testing requires several phases in
> the traffic load profile. The traffic load profile described in the draft is
> the common profile mostly used for Layer 7 testing.
> 
> - 5.1 - I like the sentence, but it leaves a world of possibilities open as
> to how one confirmed that the ancillary switching, or routing functions
> didn't limit the performance, particularly the virtualized components?
> 
> [bpm] The sentence says, "Ensure that any ancillary switching or routing
> functions between the system under test and the test equipment do not limit
> the performance of the traffic generator."
> 
> [bpm] Here we discuss the traffic generator performance, and this can be
> confirmed by doing reference test.
> 
> [bpm] The section 5 recommends reference test to ensure that the maximum
> desired traffic generator's performance. Based on the reference test results
> it can be identified, if the external device added any impact on traffic
> generator's performance.
> 
> [bpm] We will add more content in section 5 to provide more details about
> reference test.
> 
> - 5.3 - this is a nice assertion but again, how do I reasonably make the
> assertion?
> 
> [bpm] We will change the word from "Assertion" to "Ensure". Also, we will
> add more clarity about reference testing.
> 
> - 6.1 - I would suggest that the test report include the configuration of
> ancillary devices on both client/server side as well
> 
> [bpm] We believe that adding configuration of the ancillary devices doesn't
> add more value in the report. Instead of this, we will recommend documenting
> the configuration of the ancillary devices by doing reference test. We will
> add this under the section 5 "Test bed consideration".
> 
> - 6.3 - Nothing on drops anywhere?
> 
> [bpm] Are you referring to packet drops? If you are, there is no packet loss
> in stateful traffic. Instead of packet loss, the stateful traffic has
> retransmissions.
> 
> - 7.1.3.2 - Where are these numbers coming from? How are you determining the
> "initial inspected throughput"? Maybe I missed that in the document overall,
> but it's not clear to me where these KPIs are collected? I suggest this be
> called out.
> 
> [bpm] We will add more clarification in the next version. Thank you.
> 
> - 7.1.3.3 - what is a "relevant application traffic mix" profile?
> 
> [bpm] This is described in section7.1.1  (2nd paragraph). We will add the
> word "relevant" in the 1st sentence of the 2nd pragraph.so the sentence will
> be "Based on customer use case, users can choose the relevant application
> traffic mix for this test.  The details about the traffic mix MUST be
> documented in the report.  At least the following traffic mix details MUST
> be documented and reported together with the test results:
> 
> - 7.1.3.4 - where does this monitoring occur?
> 
> [bpm] The monitoring or measurement must occur in the test equipment.
> Section 4.3.4 describes this.
> 
> - 7.1.3.4 - This looks a bit like conformance testing -  Why does item (b)
> require a specific number/threshold?
> 
> [bpm] These numbers are synonymous with the zero-packet loss criteria for
> [RFC2544] Throughput and recognize the additional complexity of application
> layer performance. This was agreed by the IETF BMWG.
> 
> - 9: Why is the cipher squite recommendation for a real deployment outside
> the scope of this document?
> 
> [bpm] Because new cipher suites are frequently developed. Given that the
> draft will not be easily updated once it is accepted as an RFC we wanted to
> ensure there was flexibility to use future developed cipher suites.  
> 
> Brian Monkman on behalf of....
> 
> Alex Samonte (Fortinet), Amritam Putatunda (Ixia/Keysight), Bala Balarajah
> (NetSecOPEN), Carsten Rossenhoevel (EANTC), Chris Brown (UNH-IOL), Mike Jack
> (Spirent), Ryan Liles (Cisco), Tim Carlin (UNH-IOL), Tim Otto (Juniper)
> 
> 
> -----Original Message-----
> From: [e-mail] <[e-mail]>
> Sent: Wednesday, May 26, 2021 3:34 PM
> To: 'Sarah Banks' <[e-mail]>; 'ALFRED MORTON' <[e-mail]>
> Cc: 'Gabor LENCSE' <[e-mail]>; 'Bala Balarajah'
> <[e-mail]>; 'Bala Balarajah' <[e-mail]>;
> [e-mail]; 'MORTON, ALFRED C (AL)' <[e-mail]>;
> [e-mail]; [e-mail]; 'Carsten
> Rossenhoevel' <[e-mail]>; 'Jack, Mike' <[e-mail]>;
> 'Christopher Brown' <[e-mail]>; 'Ryan Liles (ryliles)'
> <[e-mail]>
> Subject: RE: [bmwg] WGLC on New version of draft-ietf-bmwg-ngfw-performance
> 
> Hi Sarah,
> 
> Just wanted to let you know that members of the team working on the draft
> just met. We will be sending a more detailed response soon. But in the
> meantime, We thought we would share a couple of decisions/comments.
> 
> First, your suggestion that IDS be removed from the document is reasonable.
> Thank you for your input. IDS will be removed from the draft. As such, we
> will ignore any item you raised that was related directly to IDS.
> 
> Your comment, "As a tester with long time experience with major test
> equipment manufacturers, I can't possibly begin to guess which ones of them
> would conform to this - or even if they'd answer these questions" warrants
> an immediate comment.  Both Ixia and Spirent have been closely involved with
> the drafting of this document, have implemented the tests as documented and
> have been extraordinarily willing to answer any questions related to the
> requirements outlined within. The creation of this draft has been actively
> supported by every member of our org. That is one of the reasons it has
> taken us this long to reach where we are today. 
> 
> Our goal from day one was to produce guidance on how to test a network
> security product in a manner that would provide realistic results based on
> real world deployment needs in a manner that can be reproduced and would
> provide results that can be comparable regardless of the test tool or test
> house used. Additionally, we feel strongly that the resulting guidelines
> should become part of the public domain. We believed, and still believe, the
> IETF is the right place. We have been very pleased with the input received
> and the spirit it was provided. It appears that everyone who has commented
> wants this work to be a useful addition to the knowledgebase of BMWG test
> specs that have come before.
> 
> We believe the differences that are evidenced by your comments/questions
> result from philosophical differences between the approach to testing
> documented within the draft and your approach. It is indeed possible we can
> both be right. I hope we can find compromise. 
> 
> We will get the responses to the rest of your items posted as soon as
> possible. Thank you for your patience.
> 
> Brian Monkman on behalf of....
> 
> Alex Samonte (Fortinet), Amritam Putatunda (Ixia/Keysight), Bala Balarajah
> (NetSecOPEN), Carsten Rossenhoevel (EANTC), Chris Brown (UNH-IOL), Mike Jack
> (Spirent), Ryan Liles (Cisco), Tim Carlin (UNH-IOL), Tim Otto (Juniper)
> 
> -----Original Message-----
> From: Sarah Banks <[e-mail]>
> Sent: Thursday, May 20, 2021 4:35 PM
> To: ALFRED MORTON <[e-mail]>
> Cc: [e-mail]; Gabor LENCSE <[e-mail]>; Bala
> Balarajah <[e-mail]>; Bala Balarajah <[e-mail]>;
> [e-mail]; MORTON, ALFRED C (AL) <[e-mail]>
> Subject: Re: [bmwg] WGLC on New version of draft-ietf-bmwg-ngfw-performance
> 
> Hi,
>    Sharing feedback with my participant hat on. I do apologize for sending
> it so late. A few high level comments:
> 
> - I still land in the "I don't agree that this doc clearly covers "all next
> gen security devices" - and they're not clearly defined, either. For the
> three that are covered, I think there's a lumping of the IPS and IDS that
> would be better served separated. From my perspective, an IPS *is* a bump in
> the wire, and does make "blocking" decisions, as called out in this
> document. An IDS however, does not - it tells you it happened after the
> fact, and it's a passive device. This draft is really keyed around "active
> inline" functions - ie/ a bump in the wire - and I think keeping the focus
> there makes sense, and lumping the IDS in does more harm than good. 
> 
> In short, I don't support this draft moving forward with the inclusion of
> IDS - it doesn't make sense to me. One way forward might be to remove the
> IDS as a device from scope (as a suggestion).
> 
> 
> - I've been testing for a long time, but I find this draft ... uncomfortable
> to approach. I expected, and would have preferred, something that walks me
> through a specific set of recommended scenarios with specifics on what to
> configure for repeatable tests that I could compare results with, but there
> are a lot of moving parts here and a lot of assertions that have to be made
> for the test beds as a whole where I think the test results could vary
> wildly when the same topology is handed to Test Person A and Test Person B. 
> 
> - Most of the feedback below omits the IDS pieces, because so much of it
> didn't apply. 
> 
> Thanks,
> Sarah
> 
> 
> 
> - The draft aims to replace RFC3511, but expands scope past Firewalls, to
> "next generation security devices". I'm not finding a definition of what a
> "next generation security device is", nor an exhaustive list of the devices
> covered in this draft. A list that includes is nice, but IMO not enough to
> cover what would be benchmarked here - I'd prefer to see a definition and an
> exhaustive list.
> - What is a NGIPS or NGIDS? If there are standardized definitions pointing
> to them is fine, otherwise, there's a lot of wiggle room here.
> - I still have the concern I shared at the last IETF meeting, where here,
> we're putting active inline security devices in the same category as passive
> devices. On one hand, I'm not sure I'd lump these three together in the
> first place; on the other, active inline devices typically include
> additional functions to allow administrators to control what happens to
> packets in the case of failure, and I don't see those test cases included
> here.
> - Section 4.1 - it reads as if ANY device in the test setup cannot
> contribute to network latency or throughput issues, including the DUTs - is
> that what you intended?
> - Option 1: It'd be nice to see a specific, clean, recommended test bed.
> There are options for multiple emulated routers. As a tester, I expect to
> see a specific, proscribed test bed that I should configure and test
> against. 
> - Follow on: I'm curious as to the choice of emulated routers here. The
> previous test suggests you avoid routers and switches in the topo, but then
> there are emulated ones here. I'm curious as to what advantages you think
> these bring over the real deal, and, why they aren't subject to the same
> limitations previously described?
> - In section 4.1 the text calls out Option 1 as the preferred test bed,
> which includes L3 routing, but it's not clear why that's needed?
> - The difference between Option 1 and Option 2 is the inclusion of
> additional physical gear in Option 2 - it's not clear why that's needed, or
> why the tester can't simply directly connect the test equipment to the DUT
> and remove extraneous devices from potential influence on results?
> - Section 4.2, the table for NGFW features - I'm not sure what the
> difference is between RECOMMENDED and OPTIONAL? (I realize that you might be
> saying that RECOMMENDED is the "must have enabled" features, where as
> optional is at your discretion, but would suggest that you make that clear)
> - Proscribing a list of features that have to be enabled for the test, or at
> least more than 1, feels like a strange choice here - I'd have expected
> tests cases that either test the specific features one at a time, or suggest
> several combinations, but that ultimately, we'd tell the tester to document
> WHICH features were enabled, to make the test cases repeatable? This allows
> the tester to apply a same set of apples to apples configurations to
> different vendor gear, and omit the 1 feature that doesn't exist on a
> different NGFW (for example), but hold a baseline that could be tested.
> - Table 2: With the assumption that NGIPS/IDS are required to have the
> features under "recommended", I disagree with this list. For example, some
> customers break and inspect at the tap/agg layer of the network - in this
> case, the feed into the NGIDS might be decrypted, and there's no need to
> enable SSL inspection, for example. 
> - Table 3: I disagree that an NGIDS IS REQUIRED to decrypt SSL. This
> behaviour might be suitable for an NGIPS, but the NGIDS is not a bump on the
> wire, and often isn't decrypting and re-encrypting the traffic.
> - Table 3: An NGIDS IMO is still a passive device - it wouldn't be blocking
> anything, but agree that it might tell you that it happened after the fact.
> - Table 3: Anti-evasion definition - define "mitigates". 
> - Table 3: Web-filtering - not a function of an NGIDS.
> - Table 3: DLP: Not applicable for an NGIDS.
> - Can you expand on "disposition of all flows of traffic are logged" -
> what's meant here specifically, and why do they have to be logged? (Logging,
> particularly under high loads, will impact it's own performance marks, and
> colours output)
> - ACLs wouldn't apply to an IDS because IDS's aren't blocking traffic :)
> - It might be helpful to testers to say something like "look, here's one
> suggested set of ACLs. If you're using them, great, reference that, but
> otherwise, make note of the ACLs you use, and use the same ones for
> repeatable testing".
> - 4.3.1.1 The doc proscribes specific MSS values for v4/v6  with no
> discussion around why they're chosen - that color could be useful to the
> reader.
> - 4.3.1.1 - there's a period on the 3rd to last line "(SYN/ACL, ACK). and"
> that should be changed.
> - 4.3.1.1 - As a tester with long time experience with major test equipment
> manufacturers, I can't possibly begin to guess which ones of them would
> conform to this - or even if they'd answer these questions. How helpful is
> this section to the non test houses? I suggest expansion here, ideally with
> either covering the scope of what you expect to cover, or hopefully which
> (open source/generally available) test tools or emulators could be
> considered for use as examples.
> - 4.3.1.3 - Do the emulated web browser attributes really apply to testing
> the NGIPS?
> - 4.3.2.3 - Do you expect to also leverage TLS 1.3 as a configuration option
> here?
> - 4.3.4 - I'm surprised to see the requirement that all sessions establish a
> distinct phase before moving on to the next. You might clarify why this is a
> requirement, and why staggering them is specifically rejected?
> - 5.1 - I like the sentence, but it leaves a world of possibilities open as
> to how one confirmed that the ancillary switching or routing functions
> didn't limit the performance, particularly the virtualized components?
> - 5.3 - this is a nice assertion but again, how do I reasonably make the
> assertion?
> - 6.1 - I would suggest that the test report include the configuration of
> ancillary devices on both client/server side as well
> - 6.3 - Nothing on drops anywhere?
> - 7.1.3.2 - Where are these numbers coming from? How are you determining the
> "initial inspected throughput"? Maybe I missed that in the document overall,
> but it's not clear to me where these KPIs are collected? I suggest this be
> called out.
> - 7.1.3.3 - what is a "relevant application traffic mix" profile?
> - 7.1.3.4 - where does this monitoring occur?
> - 7.1.3.4 - This looks a bit like conformance testing -  Why does item (b)
> require a specific number/threshold?
> - 9: Why is the cipher squite recommendation for a real deployment outside
> the scope of this document?
> 
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> bmwg mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/bmwg