Re: [Idnet] A few ideas/suggestions to get us going

[David Meyer <[e-mail]>]

Hey Henk,


On Thu, Mar 23, 2017 at 2:55 AM, Henk Birkholz <
[e-mail]> wrote:

> Hello,
>
> maybe an excerpt from my personal point of view can be a contribution to
> the discussion starting on this list.
>
>
> In my experience, the gap between... work-flows (in lack of a better term)
> how problem statements are created in the domain of
> network/management/security and - in contrast - the domain of machine
> learning is (aka "appears to me, subjectively") astonishingly vast.
>

Definitely. See the few attached slides for a some ideas I have on the
topic from upcoming talks.


>
> One way to illustrate that gap (and there are multiple ways, I think),
> using a bit of hyperbole:
>
> "network: least viable solution" -> "I only produce the information I
> require"
>
> meets
>
> "machine learning: combination of heterogeneous most viable solutions" ->
> "Please provide me with everything you got, including the best qualified
> semantic annotation of characteristics and context so I can identify and
> select the features that are relevant to provide a contribution".
>
>
> Also - as trivial and repetitive as that might sound - terminology again
> is key. Please note the following quote I actually encountered in my very
> early days collaborating with machine learning architects: "The maximum
> number of ports really is 2^16? Wow, how big can these routers be?".
> While this is of course "one of these entertaining anecdotes" everybody
> already heard at least once already, it also highlights quite prominently
> the existing gap form a different angle.
>
>
> In consequence, guidance that enables an individual with a specialization
> in machine learning skills to just better understand the network domain
> itself - maybe by illustrating very simple, well-known and already solved
> problem statements - might already be a contribution of high value, just
> because it is specifically provided for that group of individuals and using
> terminology that is common and well-understood in that domain.
>

I think what you are saying is that domain knowledge is very important
(key) when building ML solutions. That is for sure true.

Regarding terminology: it is a mess in ML: everyone uses their own
notation. Consider the notation used in [0] vs. for example, the Deep
Learning Book [1]. Compare the notation in [0] to say, Chapter 6 of [1]. So
we don't yet agree even on mathematical notation (though the notation of
[0] isn't widely used) much less the description of networks. It goes on.
I'll just point out here that this again goes to not having a "usable"
theory of network (UTON, I guess :-)).

Thanks,

Dave

[0] https://arxiv.org/pdf/1404.7828.pdf
[1] http://www.deeplearningbook.org/


>
> Viele Grü0e,
>
> Henk
>
>
> On 03/22/2017 06:29 PM, David Meyer wrote:
>
>> Folks,
>>
>> I thought I'd try to get some discussion going by outlining some of my
>> views as to why networking is lagging other areas in the development and
>> application of Machine Learning (ML). In particular, networking is way
>> behind what we might call the "perceptual tasks" (vision, NLP, robotics,
>> etc) as well as other areas (medicine, finance, ...). The attached slide
>> from one of my decks tries to summarize the situation, but I'll give a
>> bit of an outline below.
>>
>> So why is networking lagging many other fields when it comes to the
>> application of machine learning? There are several reasons which I'll
>> try to outline here (I was fortunate enough to discuss this with the
>> packetpushers crew a few weeks ago, see [0]). These are in no particular
>> order.
>>
>> First, we don't have a "useful" theory of networking (UTON). One way to
>> think about what such a theory would look like is by analogy to what we
>> see with the success of convolutional neural networks (CNNs) not only
>> for vision but now for many other tasks. In that case there is a theory
>> of how vision works, built up from concepts like receptive fields,
>> shared weights, simple and complex cells, etc. For example, the input
>> layer of a CNN isn't fully connected; rather connections reflect the
>> receptive field of the input layer, which is in a way that is "inspired"
>> by biological vision (being very careful with "biological inspiration").
>> Same with the alternation of convolutional and pooling layers; these
>> loosely model the alternation of simple and complex cells in the primary
>> visual cortex (V1), the secondary visual cortex(V2) and the Brodmann
>> area (V3). BTW, such a theory seems to be required for transfer learning
>> [1], which we'll need if we don't want every network to be analyzed in
>> an ad-hoc, one-off style (like we see today).
>>
>> The second thing that we need to think about is publicly available
>> standardized data sets. Examples here include MNIST, ImageNet, and many
>> others. The result of having these data sets has been the
>> steady ratcheting down of error rates on tasks such as object and scene
>> recognition, NLP, and others to super-human levels. Suffice it to say we
>> have nothing like these data sets for networking. Networking data sets
>> today are largely proprietary, and because there is no UTON, there is no
>> real way to compare results between them.
>>
>> Third, there is a large skill set gap. Network engineers (us!) typically
>> don't have the mathematical background required to build effective
>> machine learning at scale. See [2] for an outline of some of the
>> mathematical skills that are essential for effective ML. There is a lot
>> more to this, involving how progress is made in ML (open data, open
>> source, open models, in general open science and associated communities,
>> see e.g., OpenAi [3], Distill [4], and many others). In any event we
>> need build community and gain new skills if we want to be able to
>> develop and apply state of the art machine learning algorithms to
>> network data, at scale. The bottom line is that it will be difficult if
>> not impossible to be effective in the ML space if we ourselves don't
>> understand how it works and further, if we can build explainable systems
>> (noting that explaining what the individual neurons in a deep neural
>> network are doing is notoriously difficult; that said much progress is
>> being made). So we want to build explainable, end-to-end trained
>> systems, and to accomplish this we ourselves need to understand how
>> these algorithms work, but in training and in inference.
>>
>> This email is already TL;DR but I'll add one more here: We need to learn
>> control, not just prediction. Since we live in an inherently adversarial
>> environment we need to take advantage of Reinforcement Learning as well
>> as the various attacks being formulated against ML; [5] gives one
>> interesting example of attacks against policy networks using adversarial
>> examples. See also slides 31 and 32 of [6] for some more on this topic.
>>
>> I hope some of this gets us thinking about the problems we need to solve
>> in order to be successful in the ML space. There's plenty more of this
>> on http://www.1-4-5.net/~dmm/ml and http://www.1-4-5.net/~dmm/vita.html.
>> I'm looking forward to the discussion.
>>
>> Thanks,
>>
>> --dmm
>>
>>
>>
>>
>> [0]  http://packetpushers.net/podcast/podcasts/pq-show-107-applic
>> ability-machine-learning-networking/
>>
>> [1]  http://sebastianruder.com/transfer-learning/index.html
>> [2]  http://datascience.ibm.com/blog/the-mathematics-of-machine-learning/
>> [3] https://openai.com/blog/
>> [4] http://distill.pub/
>> [5] http://rll.berkeley.edu/adversarial/arXiv2017_AdversarialAttacks.pdf
>> [6]  http://www.1-4-5.net/~dmm/ml/talks/2016/cor_ml4networking.pptx
>>
>>
>>
>> _______________________________________________
>> IDNET mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/idnet
>>
>>
>
> _______________________________________________
> IDNET mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/idnet
>
>