Re: [Nmlrg] Machine Learning in network - solicitation for use cases

[Brian E Carpenter <[e-mail]>]

Sebastian,

On 18/09/2015 07:17, Sebastian Abt wrote:
> 
>> Am 08.09.2015 um 05:39 schrieb Sheng Jiang <[e-mail]>om>:
>>
>>>> b) is it possible for autonomic reaction from the network operational
>>>> perspective after detect such DDoS attack? Give the machine learning may
>>>> not be accurate, my guess is human intervention is needed.
>>>
>>> In the current practice, machine learning procedure is normally offline.
>>> 1) machine learning may not very that accurate. 2) big data processing
>>> needs time and computing resources.  Human involvement is required.
>>
>> What may influence the accuracy of the mechanism learning result? In another word, how to improve the accuracy in machine learning mechanism? This question may not be DDoS protection specific.
> 
> I think there are many different factors that affect accuracy of a ML system.  Most crucial in my opinion are the following two:
> 
> 1. You need to find an appropriate description of the class(es) you try to learn.  In ML, this processing of finding/generating an appropriate description is commonly called feature extraction.  For network security this means that you need to find a way to transform your given representation of traffic (e.g., packets, flow records, SNMP counters, …) such that only the bits relevant to describe normality/to distinguish between two classes A and B are reflected and everything else is dismissed, effectively reducing entropy. 

For a real-time system we need a transformation of the packet stream that can run
at line speed and produce a fairly small number of parameters per second, right? Would you
see that as meaning a statistical algorithm, e.g. one producing entropy measures?

> The resulting feature vectors should have high intra-class and low inter-class similarity - for whatever notion of similarity
you choose.
> 
> 2. Especially for one-class systems that only learn models of normality, it is important to be able to track a change of normality. Otherwise, these systems render themselves useless over time / generate too much false alarms.  As operator, you can only rely on the results if there are no (significant) baseline changes.  However, detecting this is probably not trivial and as far as I know this is not heavily researched by the network security community.  Some years ago, I read a paper that claimed that such baseline confidence checks are successfully employed in voice recognition systems and crucial for those system’s reliability.  Unfortunately, I don’t have this paper at hand.

Strangely enough my first published paper a million years ago was on human
factors in real-time speech recognition. It turned out that emotional changes
such as user frustration caused large changes in gross parameters of the speech
signal, as large as the differences between speakers. So I think your point
is very important: how to distinguish between normal anomalies (like a surge
in valid traffic) from abnormal anomalies (like a surge in DOS packets).
Feature extraction that makes this possible sounds like a hard problem.

   Brian