Re: [Idnet] Intelligence-Defined Network Architecture and Call for Interests

David Meyer <dmm@1-4-5.net> Wed, 29 March 2017 14:17 UTC

Return-Path: <dmm@1-4-5.net>
X-Original-To: idnet@ietfa.amsl.com
Delivered-To: idnet@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D721512706D for <idnet@ietfa.amsl.com>; Wed, 29 Mar 2017 07:17:17 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.898
X-Spam-Level:
X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=1-4-5-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Hf4I5uiGWbKv for <idnet@ietfa.amsl.com>; Wed, 29 Mar 2017 07:17:13 -0700 (PDT)
Received: from mail-qt0-x230.google.com (mail-qt0-x230.google.com [IPv6:2607:f8b0:400d:c0d::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 32ACB129513 for <idnet@ietf.org>; Wed, 29 Mar 2017 07:17:12 -0700 (PDT)
Received: by mail-qt0-x230.google.com with SMTP id r45so13590217qte.3 for <idnet@ietf.org>; Wed, 29 Mar 2017 07:17:12 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1-4-5-net.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=2MR0MeLc1N5Iaj/hwFZlwqnDn03KXLzwHauohVdq6PQ=; b=Fj1kK8FfccF0LS9Mq5WLagwDn0+97p/ahCI7fX/7hp4iTdjpdDcJGGecuk1cagNSdS MTTpnVPEYTzUQkOZRHFzTvXRaZXUS+tzJtVpFgoi5bgaL4s8AY8uPYu9CkfsIiokEhC6 0G/38Vn9cRc9YjDZ9kcx5ODNh0V+I36KOrlbv3c5L4TnpByYGxwDbXKtENGKkJ/2dHHq iPDtunAokdWop+HruCB41r7H6TTqLOFaaRZmOZAXdBizH8bq7xJg0mh0XnDwpS4k4LRx 6mL7jkT3YK/tf2LyngBNb2WsaVb3flaOtkZwg+G2lSdIcCFWOL8ylPLf7U7LebXV5nO2 28Ow==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=2MR0MeLc1N5Iaj/hwFZlwqnDn03KXLzwHauohVdq6PQ=; b=PGtNjAAwxfUBjKlDBzV82JKTgtucJv1c6sFueYG2BYn/37sPQ9KfF3baQRff3c3jI5 GBooFShWsjWPyDsm5FR3E7kdweE6HkrZ2ZR8KLeTopbBlgRGUbVn5+1NiT/325HxweXu QdisrFyLuBdD+KW2mTTKoCCReqJBk8vzDzzA3OvRUN5gk+3stqWCLYWMCPm7suuszogL 8bb5m5rl5oaECoCRu75f12Sd3njfHHqykGVOkWtBPcY9tTIBjzgC6sKOvkzl/jYV95rl bd/V79YeLX8X7QExRlGa4jhK3zPGE971NnbhKJ/BfW6b1m+YjitpXae3iLePPzRleJO6 LkWw==
X-Gm-Message-State: AFeK/H2d2n2i5P0TZQBFsdGzTDn7tA4A0wg3ahcNxIeaGDT8ZsNniUcNFh/kgYaa+dUQOUEhPFw/BYGBn0Ve+Q==
X-Received: by 10.200.38.132 with SMTP id 4mr812309qto.91.1490797030729; Wed, 29 Mar 2017 07:17:10 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.12.141.130 with HTTP; Wed, 29 Mar 2017 07:17:08 -0700 (PDT)
X-Originating-IP: [128.223.156.253]
In-Reply-To: <5BC916BD50F92F45870ABA46212CB29CE4C966@SMTP1.etri.info>
References: <5D36713D8A4E7348A7E10DF7437A4B927CD15A18@NKGEML515-MBS.china.huawei.com> <CABo5upUAQaGXTP5Q+pp++ABipMc-Yu2rKp=DGVFky+L3qzdUEg@mail.gmail.com> <CAAAu=jwv=gmtFPJC3RQ9YBjTSukz5p7BoGLmHubJnHCWgkQnCA@mail.gmail.com> <f4a0ef2b-bba1-8b02-ba63-b119438fc13e@inria.fr> <CAAAu=jytqiHmL17z_x6828Jy5YZegV=sJ_RrS0uTsNnSZZr7cg@mail.gmail.com> <5BC916BD50F92F45870ABA46212CB29CE4C966@SMTP1.etri.info>
From: David Meyer <dmm@1-4-5.net>
Date: Wed, 29 Mar 2017 07:17:08 -0700
Message-ID: <CAHiKxWg-D3fCj0at2sxSr76MV8jiHO_TXisyiwSwM7hOXUSOdw@mail.gmail.com>
To: =?UTF-8?B?6rmA66+87ISd?= <mskim16@etri.re.kr>
Cc: Brian Njenga <iambrianmuhia@gmail.com>, =?UTF-8?B?SsOpcsO0bWUgRnJhbsOnb2lz?= <jerome.francois@inria.fr>, Oscar Mauricio Caicedo Rendon <omcaicedo@unicauca.edu.co>, Sheng Jiang <jiangsheng@huawei.com>, "idnet@ietf.org" <idnet@ietf.org>
Content-Type: multipart/mixed; boundary=001a113fed40749c58054bdf3bbd
Archived-At: <https://mailarchive.ietf.org/arch/msg/idnet/6EAGY-twceiUs41956kEaStLU5w>
Subject: Re: [Idnet] Intelligence-Defined Network Architecture and Call for Interests
X-BeenThere: idnet@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: "The IDNet \(Intelligence-Defined Network\) " <idnet.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/idnet>, <mailto:idnet-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/idnet/>
List-Post: <mailto:idnet@ietf.org>
List-Help: <mailto:idnet-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/idnet>, <mailto:idnet-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 29 Mar 2017 14:17:19 -0000

Hey Min-Suk,

Totally agree we need to learn from our environment, and RL is a natural
approach. After all, the network is always changing, has adversaries, etc.
All of this means. among other things,  that we can't make simplifying
assumptions like stationary distributions,  iid data, .... So RL is one way
to attack these problems, and the classic algorithms you mention below are
certainly a reasonable approach (I've been working with policy gradients
[0], trying to model/adapt the two-player game approach of AlphaGo to
networking; the problem there is that we don't have a source of labeled
expert data like the KGS Go server (https://www.gokgs.com/) to build the
supervised policy network....).

You might also want to check out the recent "boot" of evolution strategies
as a black-box approach to RL (in particular no gradients). See [1],  [2],
 [3]. There is also a ton of code around if you want to try some of this
out (see e.g.,https://github.com/dennybritz/reinforcement-learning; this
one is in tensorflow). Finally, I've attached a few summary slides with
some of my musings on this topic from past talks.

Thanks,

Dave

[BTW, two player minimax games seem to be popping up everywhere: AlphaGo,
variational autoencoders [4], GANs [5], and many others; something to thing
about for our domain]

[0]
https://papers.nips.cc/paper/1713-policy-gradient-methods-for-reinforcement-learning-with-function-approximation.pdf
[1] https://blog.openai.com/evolution-strategies/
[2] https://arxiv.org/pdf/1703.03864.pdf
[3] http://jmlr.csail.mit.edu/papers/volume15/wierstra14a/wierstra14a.pdf
[4] http://www.1-4-5.net/~dmm/ml/vae.pdf
[5] https://arxiv.org/pdf/1406.2661.pdf

On Tue, Mar 28, 2017 at 4:04 PM, 김민석 <mskim16@etri.re.kr> wrote:

> Hi Brian,
>
>
> As you mentioned by the prior email, anticipating network DDos
> attacks is really trendy issue to solve by ML techniques.
>
> We also make some efforts how to avoid fagile nodes by a trustworthy
> communication, that means quantifying trustworthiness of node with
> normalization of various requirements such as security function, bandwidth
> and etc.
>
> We are freshly approaching in routing layer with confidence using our own
> requirements, TPD(Trust Policy Distribution) and TD(Trust Degree). These
> requirements are considered to be solved by Reinforcement Learning
> (RL) that is one of the ML algorithms. RL is useful to control some of
> network policy about specific actions and states with reinforced and
> purnished rewards (+/-), but the problem is too slow to acquire satisified
> performance. Other ways to say it, anormaly dectection and regression
> analysis might be both efficient approaching methods to solve the issues
> Dave mentioned.
>
>
> Best Regards,
>
>
> Min-Suk Kim
>
> Senior Researcher / Ph.D.
> Intelligent IoE Network Research Section,
> *E*lectronics and *T*elecommunications *R*esearch *I*nstitute (*ETRI)*
> e-mail          :  mskim16@etri.re.kr <nskim@etri.re.kr>
> http://www.etri.re.kr/
>
>
>
>
>
>
>