Re: [Nmlrg] Machine Learning in network - solicitation for use cases

[Brian E Carpenter <[e-mail]>]

On 08/09/2015 16:00, Liubing (Leo) wrote:

...
> But I'm curious about what is the item that could be labeled as "This is not an attack " or " You missed an attack ". E.g., the item is an packet, a stream, or any other kind of N-tuple things.

The two cases are rather different.

1. The system signals "attack in progress" to the NOC. The operators have a look
and decide that there is no attack, it is just some unusual traffic.
(Example: you are live-streaming the Olympic Games. Two seconds after the
end of the 100 metres final, there is an enormous burst of traffic.
The machine learning system signals an attack, because it was not trained
on the data set from the previous Olympic Games.)

In this case the NOC operators urgently tell the algorithm it is wrong.
It needs to learn that the signature of a sudden burst just after the
end of an event is less likely to be an attack than a sudden burst
at another time.

2. Someone invents a new kind of DDoS attack, which is therefore not
in the historical training data. The system doesn't identify it.
In this case, the NOC operators tell the algorithm "Attack started
at <time>." This automatically becomes high quality training data
for the algorithm: the signature of the new traffic at that time
is 100% certain to be an attack.

I think the hard part is extracting useful signatures from the
traffic stream in real time; the learning/training part is fairly
standard.

    Brian