Re: [Idnet] Intelligence-Defined Network Architecture and Call for Interests

David Meyer <dmm@1-4-5.net> Fri, 31 March 2017 15:08 UTC

Return-Path: <dmm@1-4-5.net>
X-Original-To: idnet@ietfa.amsl.com
Delivered-To: idnet@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 022A41294ED for <idnet@ietfa.amsl.com>; Fri, 31 Mar 2017 08:08:24 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.598
X-Spam-Level:
X-Spam-Status: No, score=-2.598 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=1-4-5-net.20150623.gappssmtp.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id L3aAXNu4I3zi for <idnet@ietfa.amsl.com>; Fri, 31 Mar 2017 08:08:21 -0700 (PDT)
Received: from mail-qk0-x236.google.com (mail-qk0-x236.google.com [IPv6:2607:f8b0:400d:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C32541294EF for <idnet@ietf.org>; Fri, 31 Mar 2017 08:08:20 -0700 (PDT)
Received: by mail-qk0-x236.google.com with SMTP id d201so39425721qkc.0 for <idnet@ietf.org>; Fri, 31 Mar 2017 08:08:20 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1-4-5-net.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=cPb6Nd8GnkXQVqbnD1Uft790bVbAIg5cQNRB+oc183o=; b=queLnwYicURpmpvtO5m3SHwwBdXOfKvSpp8+uSWCpqJ3HmUBOkrudY2BgJ8wwmnUet 5wiBFrugetSx9d7zoKO0RXKU/acsoTciAaunwT3UvekfupIaiUfYtDnUADkHJUuNXi16 k+xGKWxnMzMb/5b/rJtYqJ6o+01m0SydcIbpprxaRI1CGXz3LBqZT30JDiRg1OO+Q7fJ e97ou+AQjZ0hNh6WuAF3dpcnwyEYeJahcbkNBXKrN1I3zDeqtW+TJEKb7EWf6Az1kphx II1qHfX0D3DGWzewsQiyk02OJTRcsqqRZxOwxXQ6GnT8w+Itg02Mvv4H9gsYDhpLZ5BL g77Q==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=cPb6Nd8GnkXQVqbnD1Uft790bVbAIg5cQNRB+oc183o=; b=LXkTXHk7tKl5hiYrJF71CMF4Bg8lWf8ZHPqgUQMMZOrAouy0N9B2YIdWb0WauJEzJc IAVq4lg8hglFYEMvIhhRi9SP43ShCxary0kqbJ1jNxgVA8qB+Bfhc3+MrIUXrEJwH5Qr ax1HcKhuUQqoVM9NxLEZpaTfAxQUDW3W1UHJ38KZZT/1l0v+jqEW9JvRpjwMN4GJ3I9r WIO46I+og04rHTX6LwwCgPGnAkCSZSZlRyQtr9sgY3uXFtFY0KGWe6Wbs1HwIZRGAg6q lyy/z/VCIsxIfU2H3ozxsNb8vhKUzCuGJhFAdk6o6M85+8MmVVyFLzSwV6vJgwixoNiz 4j3w==
X-Gm-Message-State: AFeK/H0qZh1VZhYClIYK841jQpbcqLOk6vqDBuMcGW6P/3me0QWxiqA84jZSzrvTDjv38HgB0nCai8i7o7JqPQ==
X-Received: by 10.55.65.81 with SMTP id o78mr2976710qka.82.1490972899749; Fri, 31 Mar 2017 08:08:19 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.12.149.34 with HTTP; Fri, 31 Mar 2017 08:08:18 -0700 (PDT)
X-Originating-IP: [128.223.156.253]
In-Reply-To: <3B110B81B721B940871EC78F107D848CF33029@DGGEMM506-MBS.china.huawei.com>
References: <3B110B81B721B940871EC78F107D848CF33029@DGGEMM506-MBS.china.huawei.com>
From: David Meyer <dmm@1-4-5.net>
Date: Fri, 31 Mar 2017 08:08:18 -0700
Message-ID: <CAHiKxWh_zEAQKxQNL2yoDXawfTVo_jCzPztDfAo7R+Gout6g-w@mail.gmail.com>
To: "dingxiaojian (A)" <dingxiaojian1@huawei.com>
Cc: Brian E Carpenter <brian.e.carpenter@gmail.com>, Sheng Jiang <jiangsheng@huawei.com>, "idnet@ietf.org" <idnet@ietf.org>
Content-Type: multipart/alternative; boundary=001a114ac54e0f770b054c082e58
Archived-At: <https://mailarchive.ietf.org/arch/msg/idnet/aR-7ycqxuz2LctvSrgKesIOkoQU>
Subject: Re: [Idnet] Intelligence-Defined Network Architecture and Call for Interests
X-BeenThere: idnet@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: "The IDNet \(Intelligence-Defined Network\) " <idnet.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/idnet>, <mailto:idnet-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/idnet/>
List-Post: <mailto:idnet@ietf.org>
List-Help: <mailto:idnet-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/idnet>, <mailto:idnet-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 31 Mar 2017 15:08:24 -0000

I don't think there is any question that there is a problem with data sets;
as many of you know this has been one of my main points regarding what is
holding back ML for networking over the last 4 or so years. Of course, how
to solve this problem is a different question.

However, as least I don't necessarily agree with this statement: " So my
point is data set, not ML model.  We can select some existing models (SVM,
ELM, bayes, etc) to learn different tasks. We do not research the principle
of ML algorithm, but use them to slove [sic] network problems",  we might
check out what Andrew says while discussing what the scare resources
required to make progress in ML for any domain ([0], near the bottom):

o Data. Among leading AI teams, many can likely replicate others’ software
in, at most, 1–2 years. But it is exceedingly difficult to get access to
someone else’s data. Thus data, rather than software, is the defensible
barrier for many businesses.

o Talent. Simply downloading and “applying” open-source software to your
data won’t work. AI needs to be customized to your business context and
data. This is why there is currently a war for the scarce AI talent that
can do this work.

There is a ton of experience (and literature) reinforcing these points, but
suffice it to say that Andrew knows what he's talking about.

In any event, it seems clear that we are all on the same page with respect
to the problems with data sets (again, exactly how to solve this problem is
open). However,  we seem to have a difference in our understanding of both
the field is today and how we make progress. In particular, Andrew seems to
be saying the opposite of what you say above (quoted text). My experience
tracks more with what Andrew is saying.

Summary:  We need to attack both problems.

Dave

[0]
https://hbr.org/2016/11/what-artificial-intelligence-can-and-cant-do-right-now


On Thu, Mar 30, 2017 at 6:04 PM, dingxiaojian (A) <dingxiaojian1@huawei.com>
wrote:

> Hi Brian,
>     You are right.
>      I have worked in an institute more than five years. The main work is
> use ML to solve the domain problem.  However, for privacy reason, it's very
> hard to get some real domain data sets. So the results learned by any ML
> models is not reliable.
>     So I think the important/first thing we do is to construct real and
> reliable data sets of network domain. Just like UCI repository (
> https://archive.ics.uci.edu/ml/datasets.html) or images datasets (
> https://www.cs.utah.edu/~lifeifei/datasets.html) . Only the common and
> real data sets are agreed with all we, different ML models can be applied
> to validate and predict.
>    So my point is data set, not ML model.  We can select some existing
> models (SVM, ELM, bayes, etc) to learn different tasks. We do not research
> the principle of ML algorithm, but use them to slove network problems.
>
>  Best regards,
>
> Xiaojian
>
>
> -----邮件原件-----
> 发件人: IDNET [mailto:idnet-bounces@ietf.org] 代表 Brian E Carpenter
> 发送时间: 2017年3月30日 23:37
> 收件人: Sheng Jiang <jiangsheng@huawei.com>om>; David Meyer <dmm@1-4-5.net>
> 抄送: idnet@ietf.org
> 主题: Re: [Idnet] Intelligence-Defined Network Architecture and Call for
> Interests
>
> Agreed, and there are (still) two key points:
>
> 1. What is our underlying model (what Dave called a "theory of
> networking")? With no such model, it's very hard to tell the ML system what
> to do.
>
> 2. And as others have said: get hold of large datasets that can processed
> by ML according to that model. For developing open solutions, a corpus of
> open data sets seems essential. As anybody from the network measurement
> community will tell you, getting hold of large data sets from operators is
> extremely difficult for both privacy and commercial reasons.
>
>    Brian
>
>
> On 31/03/2017 03:41, Sheng Jiang wrote:
> > Hi, David,
> >
> > I think I agree with you, but in slight different  expression. Yes, the
> hard parts of getting ML into Network lies on machine learning. But, it is
> not that we need to develop any new ML technical/algorithms for networking
> in particular. It is that we MUST re-set up our network domain knowledge
> from the perspective of applying ML. My slides [0] does not suggest that
> *someone else* will handle the ML part. Actually, oppositely, it suggests
> some experts who have knowledge of both ML and network (probably we) would
> develop tools/algorithms/systems to handle the ML part for other network
> experts (more than 98 percent of current network administrators). So that,
> these network experts would be allowed to manage their network easily with
> intelligence association, but no need to become ML experts themselves.
> Here, we would like to treat the network administrators like the users in
> other successful ML application. We are the domian experts to do the dirty
> AI work for them.
> >
> > I believe we have common understanding in the above description. But
> certainly my slides needs further refine to clarify my viewpoint.
> >
> > Best regards,
> >
> > Sheng
> > ________________________________
> > From: IDNET [idnet-bounces@ietf.org] on behalf of David Meyer
> > [dmm@1-4-5.net]
> > Sent: 29 March 2017 2:01
> > To: Sheng Jiang
> > Cc: idnet@ietf.org
> > Subject: Re: [Idnet] Intelligence-Defined Network Architecture and
> > Call for Interests
> >
> > s/NMRL/NMLRG/   (sorry about that). Dave
> >
> > On Tue, Mar 28, 2017 at 10:59 AM, David Meyer <dmm@1-4-5.net<mailtomailto:
> dmm@1-4-5.net>> wrote:
> > Hey Sheng,
> >
> > I just wanted to revive my key concern on [0] (same one I made at the
> NMRL): The hard parts of getting Machine Learning intelligence into
> Networking is the Machine Learning part. In addition, successful deployment
> of ML requires knowledge of ML combined with domain knowledge. We
> definitely have the domain knowledge; the problem is that we don't have the
> ML knowledge, and this is one of the big factors holding us back; see e.g.
> Andrew's discussion of talent in [1].  Slides such as [0] seem to imply
> that *someone else* (in particular, not us)  will handle the ML part of all
> of this. I'll just note that in general successful deployments of ML don't
> work this way; the domain experts will have to learn ML (and vice versa)
> for us to be successful (again, see [1] and many others).
> >
> > Perhaps a useful exercise would be to write an ID that makes your
> assumptions explicit?
> >
> > Thanks,
> >
> > Dave
> >
> >
> > [0]
> > https://www.ietf.org/proceedings/97/slides/slides-97-nmlrg-intelligenc
> > e-defined-network-01.pdf [1]
> > https://hbr.org/2016/11/what-artificial-intelligence-can-and-cant-do-r
> > ight-now
> >
> >
> > On Tue, Mar 28, 2017 at 9:29 AM, Sheng Jiang <jiangsheng@huawei.com
> <mailto:jiangsheng@huawei.com>> wrote:
> > Hi, all,
> >
> > Although there are many understanding for Intelligence-Defined Network,
> we are actually using this IDN as a term reference to the SDN-beyond
> architecture that we presented in IETF97, see the below link. A reference
> model is presented in page 3, while potential standardization works is
> presented in page 9.
> >
> > https://www.ietf.org/proceedings/97/slides/slides-97-nmlrg-intelligenc
> > e-defined-network-01.pdf
> >
> > Although it might be a little bit too early for AI/ML in network giving
> the recent story of the concluded proposed NMLRG, we still would like to
> call for interests in IDN. Anybody (on site in Chicago this week) are
> interested in this or even wider topics regarding to AI/ML in network,
> please contact me on jiangsheng@huawei.com<mailto:jiangsheng@huawei.com>
> . Then we may have an informal meeting to discuss some common interests and
> potential future activities (not any activities in IETF, but also other STO
> or experimental trails, etc.)  on Thursday morning.
> >
> > FYI, we have already working on a Work Item, called IDN in the ETSI NGP
> (Next Generation Protocol) ISG, links below.
> >
> > https://portal.etsi.org/tb.aspx?tbid=844&SubTB=844
> > https://portal.etsi.org/webapp/WorkProgram/Report_WorkItem.asp?WKI_ID=
> > 51011
> >
> > Meanwhile, please do use this mail list as a forum to discuss any topics
> that may applying AI/ML into network area.
> >
> > Best regards,
> >
> > Sheng
> > _______________________________________________
> > IDNET mailing list
> > IDNET@ietf.org<mailto:IDNET@ietf.org>
> > https://www.ietf.org/mailman/listinfo/idnet
> >
> >
> >
> >
> >
> > _______________________________________________
> > IDNET mailing list
> > IDNET@ietf.org
> > https://www.ietf.org/mailman/listinfo/idnet
> >
>
> _______________________________________________
> IDNET mailing list
> IDNET@ietf.org
> https://www.ietf.org/mailman/listinfo/idnet
> _______________________________________________
> IDNET mailing list
> IDNET@ietf.org
> https://www.ietf.org/mailman/listinfo/idnet
>