Re: [Nmlrg] Machine Learning in network - solicitation for use cases

[Jérôme François <[e-mail]>]

Hi ,

>
> The two cases are rather different.
>
> 1. The system signals "attack in progress" to the NOC. The operators have a look
> and decide that there is no attack, it is just some unusual traffic.
> (Example: you are live-streaming the Olympic Games. Two seconds after the
> end of the 100 metres final, there is an enormous burst of traffic.
> The machine learning system signals an attack, because it was not trained
> on the data set from the previous Olympic Games.)
>
> In this case the NOC operators urgently tell the algorithm it is wrong.
> It needs to learn that the signature of a sudden burst just after the
> end of an event is less likely to be an attack than a sudden burst
> at another time.
>
> 2. Someone invents a new kind of DDoS attack, which is therefore not
> in the historical training data. The system doesn't identify it.
> In this case, the NOC operators tell the algorithm "Attack started
> at <time>." This automatically becomes high quality training data
> for the algorithm: the signature of the new traffic at that time
> is 100% certain to be an attack.
>
> I think the hard part is extracting useful signatures from the
> traffic stream in real time; the learning/training part is fairly
> standard.
>
>     Brian
>
>
Extracting the signatures should be the output of the learning part but
ML algorithms will not necessarily produce standards signatures like
patterns, for example this can consist in a set of classifier functions.

Also, I think most of efforts have been done on the automated detection
in the past making the human relatively not active during the detection,
usually limited to tell yes or no (case 1), more interactive real-time
ML would be helpful for complex attacks like APT to support the human
decisions.

jerome