Re: [bmwg] Ignas Bagdonas' No Objection on draft-ietf-bmwg-sdn-controller-benchmark-meth-08: (with COMMENT)

["MORTON, ALFRED C (AL)" <[e-mail]>]

Hi Ignas and Suresh (who supported this comment),

> -----Original Message-----
> From: Ignas Bagdonas [mailto:[e-mail]]
> Sent: Thursday, April 19, 2018 9:51 AM
...
> ----------------------------------------------------------------------
> COMMENT:
> ----------------------------------------------------------------------
> 
> The document seems to assume the OpenFlow dataplane abstraction model – which
> is one of the possible models; the practical applicability of such model to
> anything beyond experimental deployments is a completely separate question
> outside of the scope of this document. The methodology tends to apply to a
> broader set of central control based systems, and not only to the data plane
> operations – therefore the document seems to be setting at least something
> practically usable for benchmarking of such central control systems. Possibly
> the document could mention such assumptions made about the overall model where
> the methodology defined applies to.
[acm] 
That's a good wording suggestion, it certainly captures the working group
consensus to prepare the methods independent from OpenFlow, for 
wider applicability.

However, we also received feedback from Stuart Bryant seeking *more* 
message specificity (appended below, see * ), which is not really possible 
for the general methods. Please try to strike a balance between 
these comments in discussion today, if possible!

Al
doc shepherd

-=-=-=-=- Stuart's comment-=-=-=-=-=-
2.3.1.3. Asynchronous Message Processing Rate

Definition:
   The number responses to asynchronous messages (such as new flow
SB> That should be the number of responses per second.

Discussion:
   As SDN assures flexible network and agile provisioning, it is
   important to measure how many network events the controller can
   handle at a time. This benchmark is obtained by sending asynchronous
   messages from every connected Network Device at the rate that the
   controller processes (without dropping them).

SB> So what you are testing here is the control network and the
SB> controller. This is perhaps the only practical way to run the
SB> test, but it seems a pity that you do not deconvolve these
SB> two aspects of the test.
SB>
SB> I suppose this is really network Async Msg Proc rate rather than
SB> controller Async proc rate.
SB>
*SB> We may get to this in the companion document, but doesn't there
*SB> need to be some standardization of the event message to compare
*SB> apple with apples over time?



> 
> A nit: s/Khasanov Boris/Boris Khasanov, unless Boris himself would insist
> otherwise.
>