Re: [Idnet] IDN dedicated session call for case

[Pedro Martinez-Julia <[e-mail]>]

Dear colleagues,

I want to declare my interest and add my rough/initial contribution
about the benchmarking framework.

A proper benchmarking framework comprises a set of reference procedures,
methods, and models that can (or better *must*) be followed to assess
the quality of an AI mechanism proposed to be applied to the network
management/control area. It is therefore essential to standardize such
framework but it is even more important for it to have demonstrated
widely accepted effectiveness (not just assumed), so it must be derived
or adapted from a framework with demonstrated usefulness that already
exists in other area with similar objectives and overall methodology.

Moreover, and much more specific to the IDNET topics, is the inclusion,
dependency, or just the general relation of a standard format enforced
to the data that is used (input) and produced (output) by the framework,
so a kind of "data market" can arise without requiring to transform the
data. The initial scope of input/output data would be the datasets, but
also the new knowledge items that are stated as a result of applying the
benchmarking procedures defined by the framework, which can be collected
together to build a database of benchmark results, or just contrasted
with other existing entries in the database to know the position of the
solution just evaluated. This increases the usefulness of IDNET.

That is all for now. Let's be in contact and go further in our effort to
get a proper document (and session) in the IETF 100. If it is achieved,
I want to have some time slot to present this idea (~ 10 min), so please
count on me to organize the schedule of the potential session. Thank you
very much.

Regards,
Pedro

-- 
Pedro Martinez-Julia
Network Science and Convergence Device Technology Laboratory
Network System Research Institute
National Institute of Information and Communications Technology (NICT)
4-2-1, Nukui-Kitamachi, Koganei, Tokyo 184-8795, Japan
Email: [e-mail]
---------------------------------------------------------
*** Entia non sunt multiplicanda praeter necessitatem ***