Re: [Idnet] Intelligence-Defined Network Architecture and Call for Interests

Aydin Ulas <aydinulas@gmx.net> Thu, 30 March 2017 16:23 UTC

Return-Path: <aydinulas@gmail.com>
X-Original-To: idnet@ietfa.amsl.com
Delivered-To: idnet@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 683231297E1 for <idnet@ietfa.amsl.com>; Thu, 30 Mar 2017 09:23:04 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.701
X-Spam-Level:
X-Spam-Status: No, score=-1.701 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.197, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=no autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rqOrI2AY513R for <idnet@ietfa.amsl.com>; Thu, 30 Mar 2017 09:23:00 -0700 (PDT)
Received: from mail-pg0-x22c.google.com (mail-pg0-x22c.google.com [IPv6:2607:f8b0:400e:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1B0EB129865 for <idnet@ietf.org>; Thu, 30 Mar 2017 09:22:59 -0700 (PDT)
Received: by mail-pg0-x22c.google.com with SMTP id 21so44603442pgg.1 for <idnet@ietf.org>; Thu, 30 Mar 2017 09:22:59 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=E7rq1Gn7whDLgGsdQ0oTq7YmIBWvj80BrQzWVHW1vnY=; b=LDFvh8DNDR1zlyeN9sSYhB7yRfa6/wMCmG6exhhIZEtPsv933G3srFlGB0LhsHM7rh jPrHsrFgHKy1w20FigNMduha3/pfhHPRU3RM8i3BHxAuM4/5iikdaq2CV4hLmpONmAvf 97e6y1eD+3Y25XCP4HjGl4QtjWw6CfbiD5MUEdI9wAVIuahLXRr0wT6WPYOtNW3UR6yC LajxGXuJ1nZF/Ff5c6h4SY3s1+cFCTfd4cMct+QiTWuiwp18dAAqbyZ0vCQdCJBHSZ6I PzK5BhnvmSUHkLdBR14/o8PgcQqcjBE8GX579PbFdIQehTzw9jS5DuiPkLiLKizXvLrt KeBA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=E7rq1Gn7whDLgGsdQ0oTq7YmIBWvj80BrQzWVHW1vnY=; b=fLhomoG+VKMZx/xNDviuZo6wXo0D7wjQNpeqOIhB5uh7Wzztv7SdS8zEXMYWTU+B00 M2OmJrREdoEkGCRg8MB85MOh3VMr5fdrD0glx1FUUhrCWtqprkVSQIw0LHilJ9me9POk b76EL5Y78J0kgtFRflbCNWDM58vamWwoEzJZevnwYQrgDzdCsFOMgJL4xwRb4kqHQKRf C6FZK48xwqPrQIiKAywEAHnPsGF2QydJ/iPC+UykJm3iM4WIlMU6JI9MhVL9Tzt5dcS6 7pbV5n4Z/vD384AU/jdms7tlSaPLoHgleNHS/hrqJDogcfY+IGhaL4pI5YLrJfGo7bwM qTyA==
X-Gm-Message-State: AFeK/H3zNOoO5snyfzp6uwffUa62DyYKcYrT8JPeBpS1aJDyMotLCcjC5CrZ+m5CXGiPrz2DN64BMYoUEFMEEQ==
X-Received: by 10.84.138.193 with SMTP id 59mr578772plp.167.1490890978613; Thu, 30 Mar 2017 09:22:58 -0700 (PDT)
MIME-Version: 1.0
Sender: aydinulas@gmail.com
Received: by 10.100.138.74 with HTTP; Thu, 30 Mar 2017 09:22:18 -0700 (PDT)
In-Reply-To: <e1b1e4cd-470e-5cbd-81b6-2d9cc51b3fc1@gmail.com>
References: <5D36713D8A4E7348A7E10DF7437A4B927CD15A18@NKGEML515-MBS.china.huawei.com> <CAHiKxWgT3hKr2VwhbfpmR_siHgiY4PbiKy3QgesG7uqUTnedmw@mail.gmail.com> <CAHiKxWjbAY5uaBw+M+Wxzy8VZYu=epEtWBakLR8TdkqPRt_BAw@mail.gmail.com> <5D36713D8A4E7348A7E10DF7437A4B927CD16EE8@NKGEML515-MBS.china.huawei.com> <e1b1e4cd-470e-5cbd-81b6-2d9cc51b3fc1@gmail.com>
From: Aydin Ulas <aydinulas@gmx.net>
Date: Thu, 30 Mar 2017 19:22:18 +0300
X-Google-Sender-Auth: j0UNLhNGiMOzzzd6Td8UhrY3EXM
Message-ID: <CA+kz0z1Mv6Xz=uqJEnkKRCzuFCAQzN3pd9EiXKPTmCziNc3R-A@mail.gmail.com>
To: Brian E Carpenter <brian.e.carpenter@gmail.com>
Cc: Sheng Jiang <jiangsheng@huawei.com>, David Meyer <dmm@1-4-5.net>, "idnet@ietf.org" <idnet@ietf.org>
Content-Type: multipart/alternative; boundary=94eb2c1450e22ddf2a054bf51b70
Archived-At: <https://mailarchive.ietf.org/arch/msg/idnet/crFPtcbsw8WmrNm1oqjQUjUOAvk>
Subject: Re: [Idnet] Intelligence-Defined Network Architecture and Call for Interests
X-BeenThere: idnet@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: "The IDNet \(Intelligence-Defined Network\) " <idnet.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/idnet>, <mailto:idnet-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/idnet/>
List-Post: <mailto:idnet@ietf.org>
List-Help: <mailto:idnet-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/idnet>, <mailto:idnet-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 30 Mar 2017 16:30:55 -0000

Dear all,
As a researcher having almost 20 years of experience in machine learning
and working as a solution architect on SDN and NFV for the past four years,
I would like to offer my two cents.
I believe that I have the unique perspective to look at the problem from
the other angle and this mailing list is the perfectly suited ground for
it. The problem (or the solution as David suggested) is the lack of
standardized data sets and also the problems defined over those data sets.
In the machine learning area, we had this luxury almost since the beginning
(see ML repository: https://archive.ics.uci.edu/ml/about.html); it makes
the job relatively easy and also comparable since researchers are working
on the same problem, trying to do better, and also exchanging information.
Of course, with the rise of the Internet, smart phone revolution and the
huge amount of data produced in the last decade, combined with privacy
concerns, things have not been like they were before (Netflix reverse
engineering scandal did not help either) but people are still donating
their data for researchers to use because the community is *more
research,* *less
industry* oriented. Apart from medical data (where doctors are more
conservative to give you and privacy has a stronger impact), the trend
continues to this day and makes it easier for the likes of us. Instead, in
the networking community, as far as I have seen in the past five years,
usually the operators own the data and they are reluctant (also privacy is
an important aspect) to release this data (even an obfuscated version of
it) even for their own needs. I have been trying to detect a protocol using
packets for the past two weeks and the operator which requires this feature
does not provide us the data to work on.

To draw researchers from Machine Learning area what should be done in my
opinion is to provide well-defined problems with data supporting them and
the rest will definitely come in rapid succession. For the data, of course
supervised learning is the most established and easier to apply technique
(so labels are important) but you do not always need labels (at least all
of them) to apply semi-supervised and non-supervised learning algorithms.
Most algorithms are fault tolerant, fault being missing features, missing
labels, etc. There are techniques which support multiple instances,
dissimilarities between objects that I have yet to see in the networking
domain, which could bring unique perspectives to problems. I am trying to
persuade my colleagues who have ML experience to work on networking
problems but it is really hard to provide them the problem and the data
which could get the ball rolling.


Best regards,
Aydin Ulas, PhD,
Argela A.S., Bogazici University, MEF University
https://www.cmpe.boun.edu.tr/~ulas/



On Thu, Mar 30, 2017 at 6:37 PM, Brian E Carpenter <
brian.e.carpenter@gmail.com> wrote:

> Agreed, and there are (still) two key points:
>
> 1. What is our underlying model (what Dave called a "theory of
> networking")? With no such model, it's very hard to tell the ML system what
> to do.
>
> 2. And as others have said: get hold of large datasets that can processed
> by ML according to that model. For developing open solutions, a corpus of
> open data sets seems essential. As anybody from the network measurement
> community will tell you, getting hold of large data sets from operators is
> extremely difficult for both privacy and commercial reasons.
>
>    Brian
>
>
> On 31/03/2017 03:41, Sheng Jiang wrote:
> > Hi, David,
> >
> > I think I agree with you, but in slight different  expression. Yes, the
> hard parts of getting ML into Network lies on machine learning. But, it is
> not that we need to develop any new ML technical/algorithms for networking
> in particular. It is that we MUST re-set up our network domain knowledge
> from the perspective of applying ML. My slides [0] does not suggest that
> *someone else* will handle the ML part. Actually, oppositely, it suggests
> some experts who have knowledge of both ML and network (probably we) would
> develop tools/algorithms/systems to handle the ML part for other network
> experts (more than 98 percent of current network administrators). So that,
> these network experts would be allowed to manage their network easily with
> intelligence association, but no need to become ML experts themselves.
> Here, we would like to treat the network administrators like the users in
> other successful ML application. We are the domian experts to do the dirty
> AI work for them.
> >
> > I believe we have common understanding in the above description. But
> certainly my slides needs further refine to clarify my viewpoint.
> >
> > Best regards,
> >
> > Sheng
> > ________________________________
> > From: IDNET [idnet-bounces@ietf.org] on behalf of David Meyer [
> dmm@1-4-5.net]
> > Sent: 29 March 2017 2:01
> > To: Sheng Jiang
> > Cc: idnet@ietf.org
> > Subject: Re: [Idnet] Intelligence-Defined Network Architecture and Call
> for Interests
> >
> > s/NMRL/NMLRG/   (sorry about that). Dave
> >
> > On Tue, Mar 28, 2017 at 10:59 AM, David Meyer <dmm@1-4-5.net<mailtomailto:
> dmm@1-4-5.net>> wrote:
> > Hey Sheng,
> >
> > I just wanted to revive my key concern on [0] (same one I made at the
> NMRL): The hard parts of getting Machine Learning intelligence into
> Networking is the Machine Learning part. In addition, successful deployment
> of ML requires knowledge of ML combined with domain knowledge. We
> definitely have the domain knowledge; the problem is that we don't have the
> ML knowledge, and this is one of the big factors holding us back; see e.g.
> Andrew's discussion of talent in [1].  Slides such as [0] seem to imply
> that *someone else* (in particular, not us)  will handle the ML part of all
> of this. I'll just note that in general successful deployments of ML don't
> work this way; the domain experts will have to learn ML (and vice versa)
> for us to be successful (again, see [1] and many others).
> >
> > Perhaps a useful exercise would be to write an ID that makes your
> assumptions explicit?
> >
> > Thanks,
> >
> > Dave
> >
> >
> > [0] https://www.ietf.org/proceedings/97/slides/slides-
> 97-nmlrg-intelligence-defined-network-01.pdf
> > [1] https://hbr.org/2016/11/what-artificial-intelligence-can-
> and-cant-do-right-now
> >
> >
> > On Tue, Mar 28, 2017 at 9:29 AM, Sheng Jiang <jiangsheng@huawei.com
> <mailto:jiangsheng@huawei.com>> wrote:
> > Hi, all,
> >
> > Although there are many understanding for Intelligence-Defined Network,
> we are actually using this IDN as a term reference to the SDN-beyond
> architecture that we presented in IETF97, see the below link. A reference
> model is presented in page 3, while potential standardization works is
> presented in page 9.
> >
> > https://www.ietf.org/proceedings/97/slides/slides-
> 97-nmlrg-intelligence-defined-network-01.pdf
> >
> > Although it might be a little bit too early for AI/ML in network giving
> the recent story of the concluded proposed NMLRG, we still would like to
> call for interests in IDN. Anybody (on site in Chicago this week) are
> interested in this or even wider topics regarding to AI/ML in network,
> please contact me on jiangsheng@huawei.com<mailto:jiangsheng@huawei.com>
> . Then we may have an informal meeting to discuss some common interests and
> potential future activities (not any activities in IETF, but also other STO
> or experimental trails, etc.)  on Thursday morning.
> >
> > FYI, we have already working on a Work Item, called IDN in the ETSI NGP
> (Next Generation Protocol) ISG, links below.
> >
> > https://portal.etsi.org/tb.aspx?tbid=844&SubTB=844
> > https://portal.etsi.org/webapp/WorkProgram/Report_
> WorkItem.asp?WKI_ID=51011
> >
> > Meanwhile, please do use this mail list as a forum to discuss any topics
> that may applying AI/ML into network area.
> >
> > Best regards,
> >
> > Sheng
> > _______________________________________________
> > IDNET mailing list
> > IDNET@ietf.org<mailto:IDNET@ietf.org>
> > https://www.ietf.org/mailman/listinfo/idnet
> >
> >
> >
> >
> >
> > _______________________________________________
> > IDNET mailing list
> > IDNET@ietf.org
> > https://www.ietf.org/mailman/listinfo/idnet
> >
>
> _______________________________________________
> IDNET mailing list
> IDNET@ietf.org
> https://www.ietf.org/mailman/listinfo/idnet
>