Re: [Idnet] Intelligence-Defined Network Architecture and Call for Interests

[David Meyer <[e-mail]>]

Minsuk,

Attached are a few quick comments. I'll read more carefully this afternoon.
I also have to read the references.

Thanks,

Dave


On Wed, Mar 29, 2017 at 10:37 AM, 김민석 <[e-mail]> wrote:

> Thank you so much =:)
>
> -Minsuk Kim
>
> Sent from my iPhone
>
> On 29 Mar 2017, at 12:12 PM, David Meyer <[e-mail]> wrote:
>
>
> Hey Min-Suk,
>
>
> On Wed, Mar 29, 2017 at 8:29 AM, 김민석 <[e-mail]> wrote:
>
>> Hi Dave,
>>
>>
>> Thank you for giving me the great information.
>>
>> I absolutely agree your opinion that we need real ML data applied by data
>> pre-processing so that we have been already trying to make available ML
>> data on many ways such as clustering and classification. (using datasets of
>> contents and URL)
>>
>> It's so challenge-able steps before using adaptive ML algorithm to
>> network field.
>>
>> As you mentioned, RL is classical ML algorithm, but it is rapidly going
>> develpment and make great results with tensorflow in many fields,
>> unfortunately not network.
>>
>> For our tutorial, I attach some of practical examples with tensorflow as
>> below,
>>
>> https://github.com/tensorflow/models
>>
>>
>> Additionally, I submitted a personal draft to NMLRG even if it was closed
>> from last meeting. It's about collaborative distributed multi-agent using
>> re-inforcement learning and we trying to apply it to network real
>> architecture.
>>
>> The attachment is on the email. I really appreciate giving me a small
>> piece of your feedback and comment if you have a chance.
>>
>
> Thanks. I will try to read/comment later today.
>
> Thanks again,
>
> Dave
>
>
>>
>> Sincerely,
>>
>>
>> Min-Suk Kim
>>
>> Senior Researcher / Ph.D.
>> Intelligent IoE Network Research Section,
>> ETRI
>>
>>
>>
>>
>>
>>
>> ------------------------------
>> *보낸 사람 : *"David Meyer" <[e-mail]>
>> *보낸 날짜 : *2017-03-29 23:25:48 ( +09:00 )
>> *받는 사람 : *김민석 <[e-mail]>
>> *참조 : *Brian Njenga <[e-mail]>, Jérôme François <
>> [e-mail]>, Oscar Mauricio Caicedo Rendon <
>> [e-mail]>, Sheng Jiang <[e-mail]>,
>> [e-mail] <[e-mail]>
>> *제목 : *Re: [Idnet] Intelligence-Defined Network Architecture and Call
>> for Interests
>>
>>
>> Apparently you can't attach a .pptx. The attachment is here (pptx and
>> pdf):
>>
>>
>> http://www.1-4-5.net/~dmm/ml/misc/musings.pptx
>> http://www.1-4-5.net/~dmm/ml/misc/musings.pdf
>>
>>
>>
>>
>> Thx,
>>
>>
>> Dave
>>
>>
>>
>>
>> On Wed, Mar 29, 2017 at 7:17 AM, David Meyer <[e-mail]> wrote:
>>
>>
>>
>>>
>>> Hey Min-Suk,
>>>
>>>
>>> Totally agree we need to learn from our environment, and RL is a natural
>>> approach. After all, the network is always changing, has adversaries, etc.
>>> All of this means. among other things,  that we can't make simplifying
>>> assumptions like stationary distributions,  iid data, .... So RL is one way
>>> to attack these problems, and the classic algorithms you mention below are
>>> certainly a reasonable approach (I've been working with policy gradients
>>> [0], trying to model/adapt the two-player game approach of AlphaGo to
>>> networking; the problem there is that we don't have a source of labeled
>>> expert data like the KGS Go server (https://www.gokgs.com/) to build
>>> the supervised policy network....).
>>>
>>>
>>> You might also want to check out the recent "boot" of evolution
>>> strategies as a black-box approach to RL (in particular no gradients). See
>>> [1],  [2],  [3]. There is also a ton of code around if you want to try some
>>> of this out (see e.g.,https://github.com/dennyb
>>> ritz/reinforcement-learning; this one is in tensorflow). Finally, I've
>>> attached a few summary slides with some of my musings on this topic from
>>> past talks.
>>>
>>>
>>> Thanks,
>>>
>>>
>>> Dave
>>>
>>>
>>> [BTW, two player minimax games seem to be popping up everywhere:
>>> AlphaGo, variational autoencoders [4], GANs [5], and many others; something
>>> to thing about for our domain]
>>>
>>>
>>> [0] https://papers.nips.cc/paper/1713-policy-gradient-method
>>> s-for-reinforcement-learning-with-function-approximation.pdf
>>> [1] https://blog.openai.com/evolution-strategies/
>>> [2] https://arxiv.org/pdf/1703.03864.pdf
>>> [3] http://jmlr.csail.mit.edu/papers/volume15/wierstra14a/wi
>>> erstra14a.pdf
>>> [4] http://www.1-4-5.net/~dmm/ml/vae.pdf
>>> [5] https://arxiv.org/pdf/1406.2661.pdf
>>>
>>>
>>>
>>>
>>> On Tue, Mar 28, 2017 at 4:04 PM, 김민석 <[e-mail]> wrote:
>>>
>>>
>>>
>>>> Hi Brian,
>>>>
>>>>
>>>>
>>>> As you mentioned by the prior email, anticipating network DDos
>>>> attacks is really trendy issue to solve by ML techniques.
>>>>
>>>> We also make some efforts how to avoid fagile nodes by a trustworthy
>>>> communication, that means quantifying trustworthiness of node with
>>>> normalization of various requirements such as security function, bandwidth
>>>> and etc.
>>>>
>>>> We are freshly approaching in routing layer with confidence using our
>>>> own requirements, TPD(Trust Policy Distribution) and TD(Trust Degree).
>>>> These requirements are considered to be solved by Reinforcement Learning
>>>> (RL) that is one of the ML algorithms. RL is useful to control some of
>>>> network policy about specific actions and states with reinforced and
>>>> purnished rewards (+/-), but the problem is too slow to acquire satisified
>>>> performance. Other ways to say it, anormaly dectection and regression
>>>> analysis might be both efficient approaching methods to solve the issues
>>>> Dave mentioned.
>>>>
>>>>
>>>>
>>>> Best Regards,
>>>>
>>>>
>>>> Min-Suk Kim
>>>>
>>>> Senior Researcher / Ph.D.
>>>> Intelligent IoE Network Research Section,
>>>> *E*lectronics and *T*elecommunications *R*esearch *I*nstitute (*ET**R*
>>>> *I)*
>>>> e-mail          :  [e-mail] <[e-mail]>
>>>> http://www.etri.re.kr/
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>
>
>