Re: [Idnet] IETF99 for applying AI/ML into network management: Follow-up

[yanshen <[e-mail]>]

Dear Pedro,

> -----Original Message-----
> From: Pedro Martinez-Julia [mailto:[e-mail]]
> Sent: Wednesday, July 26, 2017 8:24 AM
> To: yanshen <[e-mail]>
> Cc: [e-mail]
> Subject: Re: [Idnet] IETF99 for applying AI/ML into network management:
> Follow-up
> 
> Dear Yanshen,
> 
> Thanks for your answer, do not worry about the timing, and find my answer
> in-line :-).
> 
> On Tue, Jul 25, 2017 at 02:10:26PM +0000, yanshen wrote:
> > Dear Pedro,
> >
> > Sorry for replying late.
> >
> > Recently I meet an interesting question is what data should belong to
> > internal and what data should be external?
> 
> For me that depends on the algorithmic representation. We have two types of
> variables:
> 
> - Measurable and controllable variables, such as the load of a resource
>   managed by the solution, which can be known and can be altered by some
>   decision of such solution (increase/decrease the amount of resources).
> 
> - Measurable but non-controllable, which are outside the control of the
>   solution. They can be known and used to take decisions but cannot be
>   altered (directly or indirectly) by the solution. A simple example can
>   be found in the occurrence of some event that can affect to a system,
>   such as the number of attendees to a baseball match. The management
>   solution cannot control such variable but uses it to determine the
>   amount of network resources assigned to the network of the stadium.
> 

I have a similar thought. I try to divide the variables into objective and subjective (may corresponding to your controllable and non-controllable. Perhaps a little different). 
	- The objective data means that we can capture, input or measure it periodically from any source (whatever it is). This class of data needs to be focused and formatted, including the context and format and so on. One of the cases is QoS value which I just mentioned. 
	- The subjective data means that it is imported into the machine (AI/Brain/Knowledge system) temporarily or irregularly. This class of data may be high-level and diversity. The solution for them I think should be pushed to application layer. It is not our dishes. Another thought is that we should try to obtain more subjective data by objective way, for example, we should try to change the way of obtaining the "reward" feedback from "user randomly report" to "periodically capture". Unless such, it should not be included into our consideration. 

I think it is significant to combine our two dimensions into one. Of course other dimensions may also considerable. 

> > A simple case is the ML algorithm, such as reinforcement learning,
> > needs "rewards" to  correct the error and improve its model during
> > iterations. Some reward parameters may be from subscribers. For
> > example, when we use the QoE value to present the reward in each
> > period. These feedback QoE values should be internal or external?
> 
> Those rewards are "controllable variables", and thus must be introduced to the
> whole loop, so the management solution uses them, and maybe other
> information, to determine actions that will change/improve them.
> 

Yeah, it is controllable. But it is troubled by using my benchmark that some of the rewards can be object but others are not.

> > BTW, I think there should be some methods to support the "reward"
> > feedback. For instance, there is some proposal to use RSVP protocol's
> > option to transmit the QoS value. And in the scenarios of video
> > transmission, some proposal uses TCP option to feedback the QoS of
> > current flow.
> 
> That is a good option for a new draft, but it would be more related to other WG
> than to IDNET "per se". I find it very interesting so, if you plan to work on it,
> please count on me. I think we can circulate some initial version to the NMRG and
> from there we can see where it fits.
> 

My pleasure! Keep in touch. 

Yansen

> > Maybe there is room for the future.
> >
> > Thanks for your reply and suggestions.
> >
> > Yansen
> 
> Let's be in contact...
> 
> Regards,
> Pedro
> 
> --
> Pedro Martinez-Julia
> Network Science and Convergence Device Technology Laboratory Network
> System Research Institute National Institute of Information and Communications
> Technology (NICT) 4-2-1, Nukui-Kitamachi, Koganei, Tokyo 184-8795, Japan
> Email: [e-mail]
> ---------------------------------------------------------
> *** Entia non sunt multiplicanda praeter necessitatem ***