Re: [Idnet] A few ideas/suggestions to get us going

[David Meyer <[e-mail]>]

Hey João,

Thanks for the references. More to read and understand! I'll just pint out
here that ``Mathematics and the Internet: A Source of Enormous Confusion
and Great Potential'' was written by two of my good friends and colleagues
(John and Walter). See http://www.1-4-5.net/~dmm/ml/talks/2017/nanog61.pptx
for some of the work John and I have done. There's lots more of this stuff
on http://www.1-4-5.net/~dmm/vita.hml under Recent Talks.

Small world!


Dave


On Wed, Mar 29, 2017 at 11:53 AM, João Paulo S. Medeiros <[e-mail]
> wrote:

> Dear David Meyer and all,
>
> I would like to contribute with some ideas and published works from my
> academic education.
>
> I have been working with Machine Learning (ML) and Computer Network since
> 2007. I initially started my research interests with the use of neural
> networks to aid the performance of classification and characterization of
> remote computer fingerprinting (e.g. [0] [1], and most recently [2] [3]).
> Although it is not my current main line of research, my experience will
> agree with the David's comment that ``we need to think about is publicly
> available standardized data''. This probably is one of the main problems
> for researchers trying to advance or reproduce state-of-the-art research on
> Intrusion Detection systems (and others feature extraction + pattern
> recognition tasks) using ML.
>
> My current main line of research is related to two of David's concerns:
> namely, (i) the UTON and the (ii) Controllability of Computer Networks.
> My PhD thesis work was related to the use of model which could be used to
> minimize the overhead of network monitoring. My last published work about
> this is in [4]. I used the theory of Complex Networks Controllability [5]
> to achieve my PhD goal. However, I realized that its too more practical
> to use this theory to build Observable (dual problem) network monitoring
> systems with minimal sensor nodes, since in controllability we need to
> directly change (or induce) the state of network devices. In this sense,
> the theory of Adaptive Filtering (e.g. Kalman Filter) is important too.
> Even so, Controlability of computer networks it's still a very interesting
> and challenging problem involving not only Complex Networks theory, but
> also, probably, Markov Process and ML.
>
> Still about UTON, the network topology almost always plays an important
> role in the ML system design. For many reasons, the topology is not
> available and its estimation is also another important problem we could
> approach using ML [6].
>
> Finally, I would like to share an inspiring paper entitled ``Mathematics
> and the Internet: A Source of Enormous Confusion and Great Potential'' [7].
>
> Best regards!
>
> [0] http://dx.doi.org/10.1109/EFTA.2007.4416854
> [1] http://dx.doi.org/10.1007/978-3-540-89173-4_20
> [2] http://dx.doi.org/10.1007/978-3-319-05885-6_12
> [3] http://dx.doi.org/10.1201/b17333-10
> [4] http://dx.doi.org/10.1109/CIT/IUCC/DASC/PICOM.2015.15
> [5] http://dx.doi.org/10.1038/nature10011
> [6] http://dx.doi.org/10.1109/TNET.2011.2175747
> [7] http://www.ams.org/notices/200905/tx090500586p.pdf
>
> -- Prof. João Paulo Souza Medeiros
>
> On Wed, Mar 22, 2017 at 2:29 PM, David Meyer <[e-mail]> wrote:
>
>> Folks,
>>
>> I thought I'd try to get some discussion going by outlining some of my
>> views as to why networking is lagging other areas in the development and
>> application of Machine Learning (ML). In particular, networking is way
>> behind what we might call the "perceptual tasks" (vision, NLP, robotics,
>> etc) as well as other areas (medicine, finance, ...). The attached slide
>> from one of my decks tries to summarize the situation, but I'll give a bit
>> of an outline below.
>>
>> So why is networking lagging many other fields when it comes to the
>> application of machine learning? There are several reasons which I'll try
>> to outline here (I was fortunate enough to discuss this with the
>> packetpushers crew a few weeks ago, see [0]). These are in no particular
>> order.
>>
>> First, we don't have a "useful" theory of networking (UTON). One way to
>> think about what such a theory would look like is by analogy to what we see
>> with the success of convolutional neural networks (CNNs) not only for
>> vision but now for many other tasks. In that case there is a theory of how
>> vision works, built up from concepts like receptive fields, shared weights,
>> simple and complex cells, etc. For example, the input layer of a CNN isn't
>> fully connected; rather connections reflect the receptive field of the
>> input layer, which is in a way that is "inspired" by biological vision
>> (being very careful with "biological inspiration"). Same with the
>> alternation of convolutional and pooling layers; these loosely model the
>> alternation of simple and complex cells in the primary visual cortex (V1),
>> the secondary visual cortex(V2) and the Brodmann area (V3). BTW, such a
>> theory seems to be required for transfer learning [1], which we'll need if
>> we don't want every network to be analyzed in an ad-hoc, one-off style
>> (like we see today).
>>
>> The second thing that we need to think about is publicly available
>> standardized data sets. Examples here include MNIST, ImageNet, and many
>> others. The result of having these data sets has been the steady ratcheting
>> down of error rates on tasks such as object and scene recognition, NLP, and
>> others to super-human levels. Suffice it to say we have nothing like these
>> data sets for networking. Networking data sets today are largely
>> proprietary, and because there is no UTON, there is no real way to compare
>> results between them.
>>
>> Third, there is a large skill set gap. Network engineers (us!) typically
>> don't have the mathematical background required to build effective machine
>> learning at scale. See [2] for an outline of some of the mathematical
>> skills that are essential for effective ML. There is a lot more to this,
>> involving how progress is made in ML (open data, open source, open models,
>> in general open science and associated communities, see e.g., OpenAi [3],
>> Distill [4], and many others). In any event we need build community and
>> gain new skills if we want to be able to develop and apply state of the art
>> machine learning algorithms to network data, at scale. The bottom line is
>> that it will be difficult if not impossible to be effective in the ML space
>> if we ourselves don't understand how it works and further, if we can build
>> explainable systems (noting that explaining what the individual neurons in
>> a deep neural network are doing is notoriously difficult; that said much
>> progress is being made). So we want to build explainable, end-to-end
>> trained systems, and to accomplish this we ourselves need to understand how
>> these algorithms work, but in training and in inference.
>>
>> This email is already TL;DR but I'll add one more here: We need to learn
>> control, not just prediction. Since we live in an inherently adversarial
>> environment we need to take advantage of Reinforcement Learning as well as
>> the various attacks being formulated against ML; [5] gives one interesting
>> example of attacks against policy networks using adversarial examples. See
>> also slides 31 and 32 of [6] for some more on this topic.
>>
>> I hope some of this gets us thinking about the problems we need to solve
>> in order to be successful in the ML space. There's plenty more of this on
>> http://www.1-4-5.net/~dmm/ml and http://www.1-4-5.net/~dmm/vita.html.
>> I'm looking forward to the discussion.
>>
>> Thanks,
>>
>> --dmm
>>
>>
>>
>>
>> [0]  http://packetpushers.net/podcast/podcasts/pq-show-107-a
>> pplicability-machine-learning-networking/
>>
>> [1]  http://sebastianruder.com/transfer-learning/index.html
>> [2]  http://datascience.ibm.com/blog/the-mathematics-of-machine-learning/
>> [3] https://openai.com/blog/
>> [4] http://distill.pub/
>> [5] http://rll.berkeley.edu/adversarial/arXiv2017_AdversarialAttacks.pdf
>> [6]  http://www.1-4-5.net/~dmm/ml/talks/2016/cor_ml4networking.pptx
>>
>>
>> _______________________________________________
>> IDNET mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/idnet
>>
>>
>