Re: [Nmlrg] Machine Learning in network - solicitation for use cases

[Sebastian Abt <[e-mail]>]

> Am 08.09.2015 um 05:39 schrieb Sheng Jiang <[e-mail]>om>:
> 
>>> b) is it possible for autonomic reaction from the network operational
>>> perspective after detect such DDoS attack? Give the machine learning may
>>> not be accurate, my guess is human intervention is needed.
>> 
>> In the current practice, machine learning procedure is normally offline.
>> 1) machine learning may not very that accurate. 2) big data processing
>> needs time and computing resources.  Human involvement is required.
> 
> What may influence the accuracy of the mechanism learning result? In another word, how to improve the accuracy in machine learning mechanism? This question may not be DDoS protection specific.

I think there are many different factors that affect accuracy of a ML system.  Most crucial in my opinion are the following two:

1. You need to find an appropriate description of the class(es) you try to learn.  In ML, this processing of finding/generating an appropriate description is commonly called feature extraction.  For network security this means that you need to find a way to transform your given representation of traffic (e.g., packets, flow records, SNMP counters, …) such that only the bits relevant to describe normality/to distinguish between two classes A and B are reflected and everything else is dismissed, effectively reducing entropy. The resulting feature vectors should have high intra-class and low inter-class similarity - for whatever notion of similarity you choose.

2. Especially for one-class systems that only learn models of normality, it is important to be able to track a change of normality. Otherwise, these systems render themselves useless over time / generate too much false alarms.  As operator, you can only rely on the results if there are no (significant) baseline changes.  However, detecting this is probably not trivial and as far as I know this is not heavily researched by the network security community.  Some years ago, I read a paper that claimed that such baseline confidence checks are successfully employed in voice recognition systems and crucial for those system’s reliability.  Unfortunately, I don’t have this paper at hand.

sebastian