Re: [Idnet] Intelligence-Defined Network Architecture and Call for Interests

[김민석 <[e-mail]>]

Thank you so much =:)

-Minsuk Kim

Sent from my iPhone

On 29 Mar 2017, at 12:12 PM, David Meyer <[e-mail]<mailto:[e-mail]>> wrote:


Hey Min-Suk,


On Wed, Mar 29, 2017 at 8:29 AM, 김민석 <[e-mail]<mailto:[e-mail]>> wrote:

Hi Dave,


Thank you for giving me the great information.

I absolutely agree your opinion that we need real ML data applied by data pre-processing so that we have been already trying to make available ML data on many ways such as clustering and classification. (using datasets of contents and URL)

It's so challenge-able steps before using adaptive ML algorithm to network field.

As you mentioned, RL is classical ML algorithm, but it is rapidly going develpment and make great results with tensorflow in many fields, unfortunately not network.

For our tutorial, I attach some of practical examples with tensorflow as below,

https://github.com/tensorflow/models


Additionally, I submitted a personal draft to NMLRG even if it was closed from last meeting. It's about collaborative distributed multi-agent using re-inforcement learning and we trying to apply it to network real architecture.

The attachment is on the email. I really appreciate giving me a small piece of your feedback and comment if you have a chance.

Thanks. I will try to read/comment later today.

Thanks again,

Dave



Sincerely,



Min-Suk Kim

Senior Researcher / Ph.D.
Intelligent IoE Network Research Section,
ETRI







________________________________
보낸 사람 : "David Meyer" <[e-mail]<mailto:[e-mail]>>
보낸 날짜 : 2017-03-29 23:25:48 ( +09:00 )
받는 사람 : 김민석 <[e-mail]<mailto:[e-mail]>>
참조 : Brian Njenga <[e-mail]<mailto:[e-mail]>>, Jérôme François <[e-mail]<mailto:[e-mail]>>, Oscar Mauricio Caicedo Rendon <[e-mail]<mailto:[e-mail]>>, Sheng Jiang <[e-mail]<mailto:[e-mail]>>, [e-mail]<mailto:[e-mail]> <[e-mail]<mailto:[e-mail]>>
제목 : Re: [Idnet] Intelligence-Defined Network Architecture and Call for Interests



Apparently you can't attach a .pptx. The attachment is here (pptx and pdf):



http://www.1-4-5.net/~dmm/ml/misc/musings.pptx
http://www.1-4-5.net/~dmm/ml/misc/musings.pdf





Thx,



Dave





On Wed, Mar 29, 2017 at 7:17 AM, David Meyer <[e-mail]<mailto:[e-mail]>> wrote:





Hey Min-Suk,



Totally agree we need to learn from our environment, and RL is a natural approach. After all, the network is always changing, has adversaries, etc. All of this means. among other things,  that we can't make simplifying assumptions like stationary distributions,  iid data, .... So RL is one way to attack these problems, and the classic algorithms you mention below are certainly a reasonable approach (I've been working with policy gradients [0], trying to model/adapt the two-player game approach of AlphaGo to networking; the problem there is that we don't have a source of labeled expert data like the KGS Go server (https://www.gokgs.com/) to build the supervised policy network....).



You might also want to check out the recent "boot" of evolution strategies as a black-box approach to RL (in particular no gradients). See [1],  [2],  [3]. There is also a ton of code around if you want to try some of this out (see e.g.,https://github.com/dennybritz/reinforcement-learning; this one is in tensorflow). Finally, I've attached a few summary slides with some of my musings on this topic from past talks.



Thanks,



Dave



[BTW, two player minimax games seem to be popping up everywhere: AlphaGo, variational autoencoders [4], GANs [5], and many others; something to thing about for our domain]



[0] https://papers.nips.cc/paper/1713-policy-gradient-methods-for-reinforcement-learning-with-function-approximation.pdf
[1] https://blog.openai.com/evolution-strategies/
[2] https://arxiv.org/pdf/1703.03864.pdf
[3] http://jmlr.csail.mit.edu/papers/volume15/wierstra14a/wierstra14a.pdf
[4] http://www.1-4-5.net/~dmm/ml/vae.pdf
[5] https://arxiv.org/pdf/1406.2661.pdf





On Tue, Mar 28, 2017 at 4:04 PM, 김민석 <[e-mail]<mailto:[e-mail]>> wrote:



Hi Brian,



As you mentioned by the prior email, anticipating network DDos attacks is really trendy issue to solve by ML techniques.

We also make some efforts how to avoid fagile nodes by a trustworthy communication, that means quantifying trustworthiness of node with normalization of various requirements such as security function, bandwidth and etc.

We are freshly approaching in routing layer with confidence using our own requirements, TPD(Trust Policy Distribution) and TD(Trust Degree). These requirements are considered to be solved by Reinforcement Learning (RL) that is one of the ML algorithms. RL is useful to control some of network policy about specific actions and states with reinforced and purnished rewards (+/-), but the problem is too slow to acquire satisified performance. Other ways to say it, anormaly dectection and regression analysis might be both efficient approaching methods to solve the issues Dave mentioned.



Best Regards,



Min-Suk Kim

Senior Researcher / Ph.D.
Intelligent IoE Network Research Section,
Electronics and Telecommunications Research Institute (ETRI)
e-mail          :  [e-mail]<mailto:[e-mail]>
http://www.etri.re.kr/