Re: [Idnet] IDN dedicated session call for case

Albert Cabellos <albert.cabellos@gmail.com> Wed, 09 August 2017 02:44 UTC

Return-Path: <albert.cabellos@gmail.com>
X-Original-To: idnet@ietfa.amsl.com
Delivered-To: idnet@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id DEF0012426E for <idnet@ietfa.amsl.com>; Tue, 8 Aug 2017 19:44:25 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.999
X-Spam-Level:
X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LaR3RKrqecKp for <idnet@ietfa.amsl.com>; Tue, 8 Aug 2017 19:44:23 -0700 (PDT)
Received: from mail-qt0-x22f.google.com (mail-qt0-x22f.google.com [IPv6:2607:f8b0:400d:c0d::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 2395D124B09 for <idnet@ietf.org>; Tue, 8 Aug 2017 19:44:23 -0700 (PDT)
Received: by mail-qt0-x22f.google.com with SMTP id 16so29571934qtz.4 for <idnet@ietf.org>; Tue, 08 Aug 2017 19:44:23 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=0zgO9TddZUT//1fvg/MalxCZIHxKP5rfg/lP16P5UK4=; b=mkMMTOWXf0+IsuEoBwvjfSCM+9ALzLZQqQjkOy8SsA4CTVY4X8gw+EPdhAjoLxSGOO 11B/3eXDW7S2+iIEIfSNiJ2/DDR8dsBJNe4+fYLwBnIESBzuP6zl4EcSj/K54ZKvaBtn 8/T8Vrt/t7MsIt0m1l5ZxeZPPvC4YlSBYCvOCuzhuonLmqrpQ8GI1/IzMX7NFn5ROieY dsu1rs7PM9Kybm7vJFom5kDAL+xEB2kygaPuidIcatnRRR1IqhuHKyq0ZlZLk/GtX3PQ 1FlvVK+EH498cfDPZ6MMy8DJjF6/lQYJjaWgT0I0sLZXTW/e/Lc9q5Cs0n28thqWnwmT OzDQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=0zgO9TddZUT//1fvg/MalxCZIHxKP5rfg/lP16P5UK4=; b=k6HZJkGAHlIVxoN9wVsd9mD0kuRrZZSyy3gXmeLvzON9kpqbq7mMY3okQxkbnxCLws DXhNbovFW9JFGRsJb685xSP0fWmIvTWFpWr4iGxkVpFwfIOQ2V2bfp2YnKhz48s9HiEx CAQN6fMAH+ncihgu9i7GlskTLncEfC458wJGG3oUX5aNEs776+309fhzWn5tQibjQCEk NuSkvo0/24H9I+CsnQ0F0NXdSRclNVh6n6sf+kP9XUwon4eP0TM3j3WbpA5bWiOAjX5T pqofDKK2RwLPxhWKCW8DB0h048ZFS+GeOw/Hih2IyPe4ksip71sGLhoJs1Yjn65/Uvmy 5FJA==
X-Gm-Message-State: AHYfb5je2GlK6m139UT1b6gEhNQxnoRaUvoTa7N2zxr/rewllSCSxySV 3jqZxWfaYtEGMsPvquqxlERJVqcoTOBH
X-Received: by 10.200.33.164 with SMTP id 33mr9384077qty.109.1502246662031; Tue, 08 Aug 2017 19:44:22 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.140.86.52 with HTTP; Tue, 8 Aug 2017 19:44:21 -0700 (PDT)
In-Reply-To: <6AE399511121AB42A34ACEF7BF25B4D297B223@DGGEMM505-MBS.china.huawei.com>
References: <6AE399511121AB42A34ACEF7BF25B4D297A34A@DGGEMM505-MBS.china.huawei.com> <CAGE_QeztLKUF55OjKcsxqW=MUMAX60vR+6935-n+nnKPRVX2zg@mail.gmail.com> <CAKGrHYwKo+Af=tRp7jg7HgHq8=v=7Wyv9Hf-d4C+renSVHG-0g@mail.gmail.com> <6AE399511121AB42A34ACEF7BF25B4D297B223@DGGEMM505-MBS.china.huawei.com>
From: Albert Cabellos <albert.cabellos@gmail.com>
Date: Wed, 9 Aug 2017 11:44:21 +0900
Message-ID: <CAGE_QezsRem02BjC31R92sSkUbpoT1p8wNKVpTQdZkYkV++qWw@mail.gmail.com>
To: yanshen <yanshen@huawei.com>
Cc: "idnet@ietf.org" <idnet@ietf.org>, =?UTF-8?B?w5Z6Z8O8IEFsYXk=?= <ozgu@simula.no>
Content-Type: multipart/alternative; boundary="001a113f382ca7cdc30556490e2a"
Archived-At: <https://mailarchive.ietf.org/arch/msg/idnet/V2n52SSn-WfLO6Cv5bv-kJT1MwQ>
Subject: Re: [Idnet] IDN dedicated session call for case
X-BeenThere: idnet@ietf.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: "The IDNet \(Intelligence-Defined Network\) " <idnet.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/idnet>, <mailto:idnet-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/idnet/>
List-Post: <mailto:idnet@ietf.org>
List-Help: <mailto:idnet-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/idnet>, <mailto:idnet-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 09 Aug 2017 02:44:26 -0000

Hi Yanshen

In my view they are not the same use-case

- QoS use-case: As far as I understand here you aim to configure the
routing/switching infrastructure to achieve QoS for flows.

- QoE use-case: Here the idea is that QoE can not typically be measured
directly over the network, it has to be measured either over the
application (e.g., buffering in on-demand video services) or by polling the
human users. Thus, measuring QoE is expensive.

What can be measured are the low-level network metrics (delay, jitter, SNR,
etc). Then the question is, which is the relation between low-level metrics
and QoE metrics? The idea is to create a data-set containing low-level and
QoE metrics. Then and thanks to ML we model the relation between low-level
and QoE metrics, this means that we understand that when delay<N, jitter<M
and SNR>K QoE levels are satisfactory. With this, operators know what
low-level performance they need to target to offer good QoE.

A nice relation is that once you establish the target performance of
low-level metrics to achieve QoE, you can then use the 'QoS use-case' (or
similar) to operate the network.

Albert

On Tue, Aug 8, 2017 at 7:02 PM, yanshen <yanshen@huawei.com> wrote:

> Dear Albert,
>
>
>
> At least two supporters you have : )
>
>
>
> I think that the QoS and QoE is just similar with my opinion mentioned
> before that is the data can be divided into subjective and objective.  This
> will be related with the data format and the way of obtaining. And your
> case build up a bridge between the subjective and objective.
>
>
>
> Yansen
>
>
>
>
>
> *From:* Özgü Alay [mailto:ozgu@simula.no]
> *Sent:* Tuesday, August 08, 2017 2:02 PM
> *To:* Albert Cabellos <albert.cabellos@gmail.com>
> *Cc:* yanshen <yanshen@huawei.com>om>; idnet@ietf.org
> *Subject:* Re: [Idnet] IDN dedicated session call for case
>
>
>
> Dear Albert,
>
> We are interested in this use case and will support the activities in this
> area.
>
> Best Regards,
>
> Özgü
>
>
>
> On 8 August 2017 at 06:52, Albert Cabellos <albert.cabellos@gmail.com>
> wrote:
>
> Hi all
>
>
>
> Here´s another use-case:
>
>
>
> Use case N+2: QoE
>         Description: Collect low-level metrics (SNR, latency, jitter,
> losses, etc) and measure QoE. Then use ML to understand what is the
> relation between satisfactory QoE and the low-level metrics. As an example
> learn that when delay>N then QoE is degraded, but when M<delay<N then QoE
> is satisfactory for the customers (please note that QoE cannot be measured
> directly over your network). This is useful to understand how the network
> must be operated to provide satisfactory QoE.
>         Process: 1. Low-level data collection and QoE measurement ; 2.
> Training Model (input low-level metrics, output QoE); 3. Real-time data
> capture and input; 4. Predict QoE; 5. Operate network to meet target QoE
> requirement, go to 3.
>         Data Format:    Time : [Start, End, Unit, Number of Value,
> Sampling Period]
>                                 Position: [Device ID, Port ID]
>                                 Direction: IN / OUT
>                                 Low-level metric : SNR, Delay, Jitter,
> queue-size, etc
>
>
>         Message :       Request: ask for the data
>                                 Reply: Data
>                                 Notice: For notification or others
>                                 Policy: Control policy
>
>
>
> Kind regards
>
>
>
> Albert
>
>
>
> On Wed, Aug 2, 2017 at 7:12 PM, yanshen <yanshen@huawei.com> wrote:
>
> Dear all,
>
> Since we plan to organize a dedicated session in NMRG, IETF100, for
> applying AI into network management (NM), I’d try to list some Use Cases
> and propose a roadmap and ToC before Nov.
>
> These might be rough. You are welcome to refine them and propose your
> focused use cases or ideas.
>
> Use case 1: Traffic Prediction
>         Description: Collect the history traffic data and external data
> which may influence the traffic. Predict the traffic in short/long/specific
> term. Avoid the congestion or risk in previously.
>         Process: 1. Data collection (e.g. traffic sample of
> physical/logical port ); 2. Training Model; 3. Real-time data capture and
> input; 4. Predication output; 5. Fix error and go back to 3.
>         Data Format:    Time : [Start, End, Unit, Number of Value,
> Sampling Period]
>                                 Position: [Device ID, Port ID]
>                                 Direction: IN / OUT
>                                 Route : [R1, R2, ..., RN]  (might be
> useful for some scenarios)
>                                 Service : [Service ID, Priority, ...]
> (Not clear how to use it but seems useful)
>                                 Traffic: [T0, T1, T2, ..., TN]
>         Message :       Request: ask for the data
>                                 Reply: Data
>                                 Notice: For notification or others
>                                 Policy: Control policy
>
> Use case 2: QoS Management
>         Description: Use multiple paths to distribute the traffic flows.
> Adjust the percentages. Avoid congestion and ensure QoS.
>         Process: 1. Data capture (e.g. traffic sample of physical/logical
> port ); 2. Training Model; 3. Real-time data capture and input; 4. Output
> percentages; 5. Fix error and go back to 3.
>         Data Format:    Time : [Timestamp, Value type (Delay/Packet
> Loss/...), Unit, Number of Value, Sampling Period]
>                                 Position: [Link ID, Device ID]
>                                 Value: [V0, V1, V2, ..., VN]
>         Message :       Request: ask for the data
>                                 Reply: Data
>                                 Notice: For notification or others
>                                 Policy: Control policy
>
> Use case N: Waiting for your Ideas
>
> Also I suggest a roadmap before Nov if possible.
>
> ### Roadmap ###
> Aug. : Collecting the use cases (related with NM). Rough thoughts and
> requirements
> Sep. : Refining the cases and abstract the common elements
> Oct. : Deeply analysis. Especially on Data Format, control flow, or other
> key points
> Nov.: F2F discussions on IETF100
> ### Roadmap End ###
>
> A rough ToC is listed in following. We may take it as a scope before Nov.
> Hope that the content could become the draft of draft.
>
> ###Table of Content###
> 1. Gap and Requirement Analysis
>         1.1 Network Management requirement
>         1.2 TBD
> 2. Use Cases
>         2.1 Traffic Prediction
>         2.2 QoS Management
>         3.3 TBD
> 3. Data Focus
>         3.1 Data attribute
>         3.2 Data format
>         3.3 TBD
> 4. Aims
>         4.1 Benchmarking Framework
>         4.2 TBD
> ###ToC End###
>
>
> Yansen
>
> _______________________________________________
> IDNET mailing list
> IDNET@ietf.org
> https://www.ietf.org/mailman/listinfo/idnet
>
>
>
>
> _______________________________________________
> IDNET mailing list
> IDNET@ietf.org
> https://www.ietf.org/mailman/listinfo/idnet
>
>
>