Re: [Idnet] IETF99 for applying AI/ML into network management: Follow-up

Pedro Martinez-Julia <> Wed, 26 July 2017 00:24 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 5F0C9132162 for <>; Tue, 25 Jul 2017 17:24:27 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -4.202
X-Spam-Status: No, score=-4.202 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id wL2MOsXolrzs for <>; Tue, 25 Jul 2017 17:24:25 -0700 (PDT)
Received: from ( [IPv6:2001:df0:232:300::1]) by (Postfix) with ESMTP id A0DB013215C for <>; Tue, 25 Jul 2017 17:24:25 -0700 (PDT)
Received: from ( []) by with ESMTP id v6Q0OJJ0075994; Wed, 26 Jul 2017 09:24:19 +0900 (JST)
Received: from ( []) by with ESMTP id v6Q0OJ4N075989; Wed, 26 Jul 2017 09:24:19 +0900 (JST)
Received: from spectre (unknown []) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (NICT Mail Spool Server2) with ESMTPS id D0CE2B15D; Wed, 26 Jul 2017 09:24:18 +0900 (JST)
Date: Wed, 26 Jul 2017 09:24:17 +0900
From: Pedro Martinez-Julia <>
To: yanshen <>
Cc: "" <>
Message-ID: <20170726002417.GZ18300@spectre>
References: <> <20170720171907.GF15832@spectre> <> <20170721091729.GJ15832@spectre> <> <20170724085956.GI18300@spectre> <>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <>
User-Agent: Mutt/1.8.3 (2017-05-23)
X-Virus-Scanned: clamav-milter 0.98.7 at zenith1
X-Virus-Status: Clean
Archived-At: <>
Subject: Re: [Idnet] IETF99 for applying AI/ML into network management: Follow-up
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: "The IDNet \(Intelligence-Defined Network\) " <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Wed, 26 Jul 2017 00:24:27 -0000

Dear Yanshen,

Thanks for your answer, do not worry about the timing, and find my
answer in-line :-).

On Tue, Jul 25, 2017 at 02:10:26PM +0000, yanshen wrote:
> Dear Pedro,
> Sorry for replying late. 
> Recently I meet an interesting question is what data should belong to
> internal and what data should be external?

For me that depends on the algorithmic representation. We have two types
of variables:

- Measurable and controllable variables, such as the load of a resource
  managed by the solution, which can be known and can be altered by some
  decision of such solution (increase/decrease the amount of resources).

- Measurable but non-controllable, which are outside the control of the
  solution. They can be known and used to take decisions but cannot be
  altered (directly or indirectly) by the solution. A simple example can
  be found in the occurrence of some event that can affect to a system,
  such as the number of attendees to a baseball match. The management
  solution cannot control such variable but uses it to determine the
  amount of network resources assigned to the network of the stadium.

> A simple case is the ML algorithm, such as reinforcement learning,
> needs "rewards" to  correct the error and improve its model during
> iterations. Some reward parameters may be from subscribers. For
> example, when we use the QoE value to present the reward in each
> period. These feedback QoE values should be internal or external? 

Those rewards are "controllable variables", and thus must be introduced
to the whole loop, so the management solution uses them, and maybe other
information, to determine actions that will change/improve them.

> BTW, I think there should be some methods to support the "reward"
> feedback. For instance, there is some proposal to use RSVP protocol's
> option to transmit the QoS value. And in the scenarios of video
> transmission, some proposal uses TCP option to feedback the QoS of
> current flow.

That is a good option for a new draft, but it would be more related to
other WG than to IDNET "per se". I find it very interesting so, if you
plan to work on it, please count on me. I think we can circulate some
initial version to the NMRG and from there we can see where it fits.

> Maybe there is room for the future.
> Thanks for your reply and suggestions.
> Yansen

Let's be in contact...


Pedro Martinez-Julia
Network Science and Convergence Device Technology Laboratory
Network System Research Institute
National Institute of Information and Communications Technology (NICT)
4-2-1, Nukui-Kitamachi, Koganei, Tokyo 184-8795, Japan
*** Entia non sunt multiplicanda praeter necessitatem ***