Re: [Idnet] Intelligence-Defined Network Architecture and Call for Interests

[David Meyer <[e-mail]>]

Hey Min-Suk,


On Wed, Mar 29, 2017 at 8:29 AM, 김민석 <[e-mail]> wrote:

> Hi Dave,
>
>
> Thank you for giving me the great information.
>
> I absolutely agree your opinion that we need real ML data applied by data
> pre-processing so that we have been already trying to make available ML
> data on many ways such as clustering and classification. (using datasets of
> contents and URL)
>
> It's so challenge-able steps before using adaptive ML algorithm to network
> field.
>
> As you mentioned, RL is classical ML algorithm, but it is rapidly going
> develpment and make great results with tensorflow in many fields,
> unfortunately not network.
>
> For our tutorial, I attach some of practical examples with tensorflow as
> below,
>
> https://github.com/tensorflow/models
>
>
> Additionally, I submitted a personal draft to NMLRG even if it was closed
> from last meeting. It's about collaborative distributed multi-agent using
> re-inforcement learning and we trying to apply it to network real
> architecture.
>
> The attachment is on the email. I really appreciate giving me a small
> piece of your feedback and comment if you have a chance.
>

Thanks. I will try to read/comment later today.

Thanks again,

Dave


>
> Sincerely,
>
>
> Min-Suk Kim
>
> Senior Researcher / Ph.D.
> Intelligent IoE Network Research Section,
> ETRI
>
>
>
>
>
>
> ------------------------------
> *보낸 사람 : *"David Meyer" <[e-mail]>
> *보낸 날짜 : *2017-03-29 23:25:48 ( +09:00 )
> *받는 사람 : *김민석 <[e-mail]>
> *참조 : *Brian Njenga <[e-mail]>, Jérôme François <
> [e-mail]>, Oscar Mauricio Caicedo Rendon <
> [e-mail]>, Sheng Jiang <[e-mail]>,
> [e-mail] <[e-mail]>
> *제목 : *Re: [Idnet] Intelligence-Defined Network Architecture and Call for
> Interests
>
>
> Apparently you can't attach a .pptx. The attachment is here (pptx and
> pdf):
>
>
> http://www.1-4-5.net/~dmm/ml/misc/musings.pptx
> http://www.1-4-5.net/~dmm/ml/misc/musings.pdf
>
>
>
>
> Thx,
>
>
> Dave
>
>
>
>
> On Wed, Mar 29, 2017 at 7:17 AM, David Meyer <[e-mail]> wrote:
>
>
>
>>
>> Hey Min-Suk,
>>
>>
>> Totally agree we need to learn from our environment, and RL is a natural
>> approach. After all, the network is always changing, has adversaries, etc.
>> All of this means. among other things,  that we can't make simplifying
>> assumptions like stationary distributions,  iid data, .... So RL is one way
>> to attack these problems, and the classic algorithms you mention below are
>> certainly a reasonable approach (I've been working with policy gradients
>> [0], trying to model/adapt the two-player game approach of AlphaGo to
>> networking; the problem there is that we don't have a source of labeled
>> expert data like the KGS Go server (https://www.gokgs.com/) to build the
>> supervised policy network....).
>>
>>
>> You might also want to check out the recent "boot" of evolution
>> strategies as a black-box approach to RL (in particular no gradients). See
>> [1],  [2],  [3]. There is also a ton of code around if you want to try some
>> of this out (see e.g.,https://github.com/dennyb
>> ritz/reinforcement-learning; this one is in tensorflow). Finally, I've
>> attached a few summary slides with some of my musings on this topic from
>> past talks.
>>
>>
>> Thanks,
>>
>>
>> Dave
>>
>>
>> [BTW, two player minimax games seem to be popping up everywhere: AlphaGo,
>> variational autoencoders [4], GANs [5], and many others; something to thing
>> about for our domain]
>>
>>
>> [0] https://papers.nips.cc/paper/1713-policy-gradient-method
>> s-for-reinforcement-learning-with-function-approximation.pdf
>> [1] https://blog.openai.com/evolution-strategies/
>> [2] https://arxiv.org/pdf/1703.03864.pdf
>> [3] http://jmlr.csail.mit.edu/papers/volume15/wierstra14a/wierstra14a.pdf
>> [4] http://www.1-4-5.net/~dmm/ml/vae.pdf
>> [5] https://arxiv.org/pdf/1406.2661.pdf
>>
>>
>>
>>
>> On Tue, Mar 28, 2017 at 4:04 PM, 김민석 <[e-mail]> wrote:
>>
>>
>>
>>> Hi Brian,
>>>
>>>
>>>
>>> As you mentioned by the prior email, anticipating network DDos
>>> attacks is really trendy issue to solve by ML techniques.
>>>
>>> We also make some efforts how to avoid fagile nodes by a trustworthy
>>> communication, that means quantifying trustworthiness of node with
>>> normalization of various requirements such as security function, bandwidth
>>> and etc.
>>>
>>> We are freshly approaching in routing layer with confidence using our
>>> own requirements, TPD(Trust Policy Distribution) and TD(Trust Degree).
>>> These requirements are considered to be solved by Reinforcement Learning
>>> (RL) that is one of the ML algorithms. RL is useful to control some of
>>> network policy about specific actions and states with reinforced and
>>> purnished rewards (+/-), but the problem is too slow to acquire satisified
>>> performance. Other ways to say it, anormaly dectection and regression
>>> analysis might be both efficient approaching methods to solve the issues
>>> Dave mentioned.
>>>
>>>
>>>
>>> Best Regards,
>>>
>>>
>>> Min-Suk Kim
>>>
>>> Senior Researcher / Ph.D.
>>> Intelligent IoE Network Research Section,
>>> *E*lectronics and *T*elecommunications *R*esearch *I*nstitute (*ET**R*
>>> *I)*
>>> e-mail          :  [e-mail] <[e-mail]>
>>> http://www.etri.re.kr/
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>