Re: [Idnet] Intelligence-Defined Network Architecture and Call for Interests

[David Meyer <[e-mail]>]

Hey Min-Suk,

Totally agree we need to learn from our environment, and RL is a natural
approach. After all, the network is always changing, has adversaries, etc.
All of this means. among other things,  that we can't make simplifying
assumptions like stationary distributions,  iid data, .... So RL is one way
to attack these problems, and the classic algorithms you mention below are
certainly a reasonable approach (I've been working with policy gradients
[0], trying to model/adapt the two-player game approach of AlphaGo to
networking; the problem there is that we don't have a source of labeled
expert data like the KGS Go server (https://www.gokgs.com/) to build the
supervised policy network....).

You might also want to check out the recent "boot" of evolution strategies
as a black-box approach to RL (in particular no gradients). See [1],  [2],
 [3]. There is also a ton of code around if you want to try some of this
out (see e.g.,https://github.com/dennybritz/reinforcement-learning; this
one is in tensorflow). Finally, I've attached a few summary slides with
some of my musings on this topic from past talks.

Thanks,

Dave

[BTW, two player minimax games seem to be popping up everywhere: AlphaGo,
variational autoencoders [4], GANs [5], and many others; something to thing
about for our domain]

[0]
https://papers.nips.cc/paper/1713-policy-gradient-methods-for-reinforcement-learning-with-function-approximation.pdf
[1] https://blog.openai.com/evolution-strategies/
[2] https://arxiv.org/pdf/1703.03864.pdf
[3] http://jmlr.csail.mit.edu/papers/volume15/wierstra14a/wierstra14a.pdf
[4] http://www.1-4-5.net/~dmm/ml/vae.pdf
[5] https://arxiv.org/pdf/1406.2661.pdf

On Tue, Mar 28, 2017 at 4:04 PM, 김민석 <[e-mail]> wrote:

> Hi Brian,
>
>
> As you mentioned by the prior email, anticipating network DDos
> attacks is really trendy issue to solve by ML techniques.
>
> We also make some efforts how to avoid fagile nodes by a trustworthy
> communication, that means quantifying trustworthiness of node with
> normalization of various requirements such as security function, bandwidth
> and etc.
>
> We are freshly approaching in routing layer with confidence using our own
> requirements, TPD(Trust Policy Distribution) and TD(Trust Degree). These
> requirements are considered to be solved by Reinforcement Learning
> (RL) that is one of the ML algorithms. RL is useful to control some of
> network policy about specific actions and states with reinforced and
> purnished rewards (+/-), but the problem is too slow to acquire satisified
> performance. Other ways to say it, anormaly dectection and regression
> analysis might be both efficient approaching methods to solve the issues
> Dave mentioned.
>
>
> Best Regards,
>
>
> Min-Suk Kim
>
> Senior Researcher / Ph.D.
> Intelligent IoE Network Research Section,
> *E*lectronics and *T*elecommunications *R*esearch *I*nstitute (*ETRI)*
> e-mail          :  [e-mail] <[e-mail]>
> http://www.etri.re.kr/
>
>
>
>
>
>
>