Re: [Nmlrg] Machine Learning in network - solicitation for use cases

[Sebastian Abt <[e-mail]>]

Hi all.

I’m a bit late in the threat, but I’d like to share some insights/experience I gained when researching ML-based systems and also when deploying them in operational networks...

> Am 07.09.2015 um 03:52 schrieb Dacheng Zhang <[e-mail]>om>:
> 
> 
> 
> 在 15-9-6 下午5:01， "Sheng Jiang" <[e-mail]> 写入:
> 
>>> Some more detailed introduction.
>>> 
>>> DDoS and APT are very active research topics. Application layer DDoS
>>> attacks are more difficult to detect than layer 4 DDoS attacks. In many
>>> cases, the application layer DDoS does not introduce large amount
>>> traffics. However, by using big data and data mining tech, it is possible
>>> to find out the clues of such attacks.
>> 
>> Hi, Dacheng,
>> 
>> Applying machine learning in DDoS protection is an interest use case. For
>> my understanding, the machine would learn the potential attack behaviors,
>> am I right?
> 
> Yes， you are right.
>> 
>> If yes, I have two questions: a) does the machine learning has the
>> possibility to learn/identify new attack behaviors, which was not
>> recognized before? If yes, what is the working principles?
> 
> Normally we need to generate a normal behavior model and some “abnormal
> behavior models”, the machine will detect whether certain behavior of a
> client will be located in an ‘abnormal’ area.
> I need to check with my colleagues to see whether we could disclose more
> detailed information for the moment.

This depends on your anomaly detection system.  In general, you can apply two-class or one-class systems (multi-class systems are typically deployed as many two-class comparisons).  So, for the DDoS case you could either:

1. Build a model that learns a normal class (benign traffic) and an attack class (DDoS traffic).  For this, you typically need labelled traffic of both classes which are equally balanced in size.  If one class is more dominant than the other, models tend to generalise towards the more dominant class, just because guessing-probability in that case is higher and, hence, classification error is lower.  Using such a model, an ML-based ADS would tell you to which class a specific input sample belongs.

2. Build a model that learns just the normal class.  In that case your system typically needs to be able to map input samples to a point in an euclidean space.  For anomaly detection, you then typically compare distance between new data points and the center/representative of the previously learned normal class.  If distance is too large, than anomaly is assumed.  For learning the normal class you „simply“ need a clean sample of normal traffic.

From my experience and in my opinion, I would only call the second case a true anomaly detection system as the system would not make an assumption of what might be an attack.  Instead, it simply makes an assumption (i.e., the learnt model) of what normality looks like.  The drawback of such systems is that normality might change, e.g. you deploy a new application or - even more difficult - one of your customers deploys a new application or uses a new protocol. In that case you raise false alarms.

>> b) is it possible for autonomic reaction from the network operational
>> perspective after detect such DDoS attack? Give the machine learning may
>> not be accurate, my guess is human intervention is needed.
> 
> In the current practice, machine learning procedure is normally offline.
> 1) machine learning may not very that accurate. 2) big data processing
> needs time and computing resources.  Human involvement is required.

There are also online learning/continuous learning systems which update their models according to learning decisions.  This may also be enhanced by operator feedback (i.e., manually accepting or declining a predicted class after inspection).  This is a very interesting approach, but unfortunately has some fallbacks: attackers can trick systems to displace the decision boundary in a way that attacks get unnoticed.

Regards,
sebastian