Re: [Nmlrg] Machine Learning in network - solicitation for use cases

[Sheng Jiang <[e-mail]>]

>>>>> DDoS and APT are very active research topics. Application layer DDoS
>>>>> attacks are more difficult to detect than layer 4 DDoS attacks. In many
>>>>> cases, the application layer DDoS does not introduce large amount
>>>>> traffics. However, by using big data and data mining tech, it is possible
>>>>> to find out the clues of such attacks.
>>>>
>>>> Hi, Dacheng,
>>>>
>>>> Applying machine learning in DDoS protection is an interest use case. For
>my
>>> understanding, the machine would learn the potential attack behaviors,
>am I
>>> right?
>>>>
>>>> If yes, I have two questions: a) does the machine learning has the
>possibility
>>> to learn/identify new attack behaviors, which was not recognized before?
>If
>>> yes, what is the working principles? b) is it possible for autonomic reaction
>>>from the network operational perspective after detect such DDoS attack?
>>> Give the machine learning may not be accurate, my guess is human
>>> intervention is needed.
>>>
>>> I suppose this is in some ways similar to spam processing in email. For
>>> example,
>>> a Bayesian spam filter is surprisingly good, but sometimes makes mistakes,
>so
>>> human training to correct false positives and false negatives is essential.
>>
>> Hi, Brian,
>>
>> I believe there is feedback-style training in spam processing. But what do
>you mean by "human" training? Do you mean the feedback is decided and
>feed by human administrators? I believe this could be done by machine
>learning mechanisms.
>
>In the spam case it is definitely a real live human who must detect an error
>by the classifier. In my case I estimate that maybe 0.5% of the messages
>in my Gmail spam folder are not spam, and maybe 1% in my Gmail Inbox are
>actually
>spam.
>
>I think that for real-time DDOS protection, the solution has to include
>real-time
>input from an operator for both cases: "This is not an attack" and "You missed
>an attack". I think that means that the machine-learning system will always
>run in training mode, even if training is only needed in 1% of cases.

The automatic in DDoS scenarios may be more critic than the SPAM scenarios. In the worst case, the time of waiting human input/feedback may already be enough for attackers to make significant damage. However, this may not be directly relative to machine learning. The programs that react according to machine learning results may be if-else style. It may also potentially be another machine learning mechanism.

Sheng

>> Actually, spam filtering was one of the earliest network-relevant area that
>starts to use machine learning. It would worth to study the machine learning
>applications in spam filtering. Or we could invite some expert in this area to
>join nmlrg discussion.
>
>Agreed
>  Brian
>
>>
>> Best regards,
>>
>> Sheng
>>
>>> How can
>>> human training be achieved for a real-time case like DDOS?
>>>
>>>   Brian
>>>
>>>>
>>>> Best regards,
>>>>
>>>> Sheng
>>>>
>>>>> There were some related discussions in Dots. If you are interested, I
>>>>> could find them out later.
>>>>>
>>>>> _______________________________________________
>>>>> nmlrg mailing list
>>>>> [e-mail]
>>>>> https://www.irtf.org/mailman/listinfo/nmlrg
>>>> _______________________________________________
>>>> nmlrg mailing list
>>>> [e-mail]
>>>> https://www.irtf.org/mailman/listinfo/nmlrg
>>>>
>
>_______________________________________________
>nmlrg mailing list
>[e-mail]
>https://www.irtf.org/mailman/listinfo/nmlrg