Re: [Nmlrg] new ways of dealing with sparse categorical data like IP addresses (Google Wide & Deep)

Sheng Jiang <jiangsheng@huawei.com> Fri, 01 July 2016 18:29 UTC

From: Sheng Jiang <jiangsheng@huawei.com>
To: David Meyer <dmm@1-4-5.net>, "nmlrg@irtf.org" <nmlrg@irtf.org>
Thread-Topic: [Nmlrg] new ways of dealing with sparse categorical data like IP addresses (Google Wide & Deep)
Thread-Index: AQHR0jbbC8ZP2gDp8UaVg4AogmAngqACUbXn
Date: Fri, 01 Jul 2016 18:26:13 +0000
Message-ID: <5D36713D8A4E7348A7E10DF7437A4B927CA8A31A@NKGEML515-MBX.china.huawei.com>
References: <CAHiKxWiqk8XAp-XAhmnp3K9vmNOW+YHJ_pjCpFdE_86NdoepYQ@mail.gmail.com>
In-Reply-To: <CAHiKxWiqk8XAp-XAhmnp3K9vmNOW+YHJ_pjCpFdE_86NdoepYQ@mail.gmail.com>
Accept-Language: en-GB, zh-CN, en-US
Content-Language: en-GB
Content-Type: multipart/alternative; boundary="_000_5D36713D8A4E7348A7E10DF7437A4B927CA8A31ANKGEML515MBXchi_"
MIME-Version: 1.0
Archived-At: <https://mailarchive.ietf.org/arch/msg/nmlrg/CK7_NoU9s5AXRB6_xoOckzgldi0>
Subject: Re: [Nmlrg] new ways of dealing with sparse categorical data like IP addresses (Google Wide & Deep)
Precedence: list

Hi, David,



Thanks for this information. This looks interesting. It is something worth of trying. However, I cannot claim I understand it pruperly. Particularly, there are a few explanation actually confused me a lot more, like a wide linear model (for memorization), a deep neural network (for generalization), sparse inputs (categorical feature with a large number of possible feature values), also your mentioned IPv{4,6} addresses as a sparse categorical feature, etc. I guess there are a lot terminology and statement in this field have not been properly specificed or standardized. So there are lots chances to misunderstand each other.



Meanwhile, one of my read from the blog is that it is not new to combine more than one learning/analysing approaches for a same task. Multiple approaches could be run simultaneously. If they could reach the same result, they prove each other. Then, we could have more confidence for the predict result. I am sure there will be some case, the result are different. Then what? I am not sure. But it would be interested to continue working on it.



Regards,



Sheng

________________________________
From: nmlrg [nmlrg-bounces@irtf.org] on behalf of David Meyer [dmm@1-4-5.net]
Sent: 30 June 2016 2:48
To: nmlrg@irtf.org
Subject: [Nmlrg] new ways of dealing with sparse categorical data like IP addresses (Google Wide & Deep)

Check out Wide & Deep: https://research.googleblog.com/2016/06/wide-deep-learning-better-together-with.html

Among other things, Wide & Deep is

"useful for generic large-scale regression and classification problems with sparse inputs (categorical features with a large number of possible feature values), such as recommender systems, search, and ranking problems."

In the network space we have many such sparse categorical features such as IPv{4,6} addresses. Wide & Deep might give us a few  hints about how we might approach this problem.

--dmm

Re: [Nmlrg] new ways of dealing with sparse categ… Sheng Jiang
[Nmlrg] new ways of dealing with sparse categoric… David Meyer