Re: [Nmlrg] links to each slide of presentations//RE: slides of NMLRG #3, June 27th, Athens

[David Meyer <[e-mail]>]

Sheng,

Thanks for the pointers. One thing I notice is that we don't have much on
what the characteristics of network data might be, and as such, what kinds
of existing learning algorithms might be suitable and where additional
development might be required. For example, is the collected data IID? Is
it time series? Is the underlying distribution stationary? If not, how does
that constrain algorithms we might use or develop? For example, how does a
given algorithm deal with concept drift/internal covariate shift (in the
case of DNNs; see e.g., batch normalization [0]). There are many other such
questions, such as is the data categorical (e.g., ports, IP addresses) or
is it continuous/discrete (e.g., counters). And if the data in question is
categorical, what is the cardinality of the categories (this will inform
how such data can be encoded); in the case of IP addresses we can't really
one-hot encode addresses because their cardinality is too large (2^32 or
2^128); this has implications for how we build classifiers (in particular,
for softmax layers in DNNs of various kinds).

Related to the above is the question of features? What are good features
for networking? Where do they come from? Are they domain specific? Can we
learn features the way a DNNs does in the network space? Can we use
autoencoders to discover such features? Or can we use GANs to train DNNs
for network classification tasks in an unsupervised manner? Are there
other, non-ad-hoc (well founded) methods we can use, or is every use case a
one-off (one would hope not).

We can carry the same kinds of analyzes to the algorithms applied. For
example, while something k-means is an effective way to get a feeling for
how continuous/discrete hangs together, if our data is categorical
statistical clustering approaches such as LDA might provide a more
well-founded approach (of course, as with most Bayesian techniques is the
question of approximate inference arises since in most interesting cases
the integral that we need to solve, namely the marginal probability data
isn't tractable so we need to resort to MCMC or more likely variational
inference). And what about the use of SGD/batch normalization etc with
DNNs, and perhaps more importantly, can we use network data to train DNN
policy networks for reinforcement learning like we saw in deep Q-learning
and AlphaGo?

These comments are all by way of saying that we don't have a solid
theoretical understanding (yet) of how techniques that have been so
successful in other domains (e.g., DNNs for perceptual tasks) generalize to
networking use cases. We will need this understanding if our goal is to
provide accurate, repeatable, and explainable results.

In order to accomplish all of this we need, as I have been saying , not
only a good understanding of how these algorithms work but also
standardized data sets and associated benchmarks so we can tell if we are
making progress (or even if our techniques work). Analogies here include
MNIST and ImageNet and their associated benchmarks, among others. As
mentioned  standardized data sets are key to making progress in the ML for
networking space (otherwise how do you know your technique works and/or
improves on another techniques?). One might assume that these data sets
would need to be labeled (as supervised learning is where most of the
progress is being made these days), but not necessarily; Generative
Adversarial Networks (GANs) have emerged as a new way to train DNNs in an
unsupervised manner (this is moving very rapidly; see e.g.,
https://openai.com/blog/generative-models/).

The summary here is that the "distance" between theory and practice in ML
is effectively zero right now due to the incredible rate of progress in the
field; this means we need  to understand both sides of the theory/practice
coin in order to be effective. None of the slide decks provide much
background on what the proposed algorithms are, how they work, or why they
should be expected to work on network data.

Finally, if you are interested in LDA or other algorithms there are a few
short explanatory pieces I have written for my team on
http://www.1-4-5.net/~dmm/ml (works in progress).

Thanks,

Dave

[0] https://arxiv.org/pdf/1502.03167.pdf

On Tue, Jun 28, 2016 at 7:03 AM, Sheng Jiang <[e-mail]> wrote:

> Oops... The proceeding page for interim meeting seems not be as
> intelligent as the proceeding pages of IETF meetings. Our proceeding page
> does not autonomically show the slides. I have sent an email to IETF
> secretary to ask them to fix it. Meanwhile, in this email, here is the
> links for each presentations:
>
> Slide Filename Edit Replace Delete
> Chair Slides
>
> https://www.ietf.org/proceedings/interim-2016-nmlrg-01/slides/slides-interim-2016-nmlrg-01-0.pdf
>
> Introduction to Network Machine Learning & NMLRG
>
> https://www.ietf.org/proceedings/interim-2016-nmlrg-01/slides/slides-interim-2016-nmlrg-01-1.pdf
>
> Data Collection and Analysis At High Security Lab
>
> https://www.ietf.org/proceedings/interim-2016-nmlrg-01/slides/slides-interim-2016-nmlrg-01-2.pdf
>
> Use Cases of Applying Machine Learning Mechanism with Network Traffic
>
> https://www.ietf.org/proceedings/interim-2016-nmlrg-01/slides/slides-interim-2016-nmlrg-01-3.pdf
>
> Mobile network state characterization and prediction
>
> https://www.ietf.org/proceedings/interim-2016-nmlrg-01/slides/slides-interim-2016-nmlrg-01-4.pdf
>
> Learning how to route
>
> https://www.ietf.org/proceedings/interim-2016-nmlrg-01/slides/slides-interim-2016-nmlrg-01-5.pdf
>
> Regards,
>
> Sheng
> ________________________________________
> From: nmlrg [[e-mail]] on behalf of Sheng Jiang [
> [e-mail]]
> Sent: 28 June 2016 21:03
> To: [e-mail]
> Subject: [Nmlrg] slides of NMLRG #3, June 27th, Athens
>
> Hi, nmlrg,
>
> All slides that have been presented in our NMLRG #3 meeting, June 27th,
> 2016, Athens, Greece, with EUCNC2016, have been uploaded. They can be
> accessed through below link
>
> https://www.ietf.org/proceedings/interim/2016/06/27/nmlrg/proceedings.html
>
> Best regards,
>
> Sheng
> _______________________________________________
> nmlrg mailing list
> [e-mail]
> https://www.irtf.org/mailman/listinfo/nmlrg
> _______________________________________________
> nmlrg mailing list
> [e-mail]
> https://www.irtf.org/mailman/listinfo/nmlrg
>