Re: [Idnet] IDN dedicated session call for case

Hi Albert,

My fault. I make it confused. I agree with you that they are different cases. And both of them are valuable.

Actually I means that the data involved in QoS use-case mostly can be captured, formatted and processed periodically and regularly. Or say most of them are objective (Pedro would like to call them controllable). As you mentioned that the some of the QoE data may be measured by polling the human users. I divide this into subjective class (or say uncontrollable). These two types of data may need different methods to process.

This consideration may be related to the data structure design and context design in future standardization work.

This is my two cents.

Regards,

Yansen

From: Albert Cabellos [mailto:albert.cabellos@gmail.com]
Sent: Wednesday, August 09, 2017 10:44 AM
To: yanshen <yanshen@huawei.com>
Cc: idnet@ietf.org; Özgü Alay <ozgu@simula.no>
Subject: Re: [Idnet] IDN dedicated session call for case

Hi Yanshen

In my view they are not the same use-case

- QoS use-case: As far as I understand here you aim to configure the routing/switching infrastructure to achieve QoS for flows.

- QoE use-case: Here the idea is that QoE can not typically be measured directly over the network, it has to be measured either over the application (e.g., buffering in on-demand video services) or by polling the human users. Thus, measuring QoE is expensive.

What can be measured are the low-level network metrics (delay, jitter, SNR, etc). Then the question is, which is the relation between low-level metrics and QoE metrics? The idea is to create a data-set containing low-level and QoE metrics. Then and thanks to ML we model the relation between low-level and QoE metrics, this means that we understand that when delay<N, jitter<M and SNR>K QoE levels are satisfactory. With this, operators know what low-level performance they need to target to offer good QoE.

A nice relation is that once you establish the target performance of low-level metrics to achieve QoE, you can then use the 'QoS use-case' (or similar) to operate the network.

Albert

On Tue, Aug 8, 2017 at 7:02 PM, yanshen <yanshen@huawei.com<mailto:yanshen@huawei.com>> wrote:
Dear Albert,

At least two supporters you have : )

I think that the QoS and QoE is just similar with my opinion mentioned before that is the data can be divided into subjective and objective.  This will be related with the data format and the way of obtaining. And your case build up a bridge between the subjective and objective.

Yansen

From: Özgü Alay [mailto:ozgu@simula.no<mailto:ozgu@simula.no>]
Sent: Tuesday, August 08, 2017 2:02 PM
To: Albert Cabellos <albert.cabellos@gmail.com<mailto:albert.cabellos@gmail.com>>
Cc: yanshen <yanshen@huawei.com<mailto:yanshen@huawei.com>>; idnet@ietf.org<mailto:idnet@ietf.org>
Subject: Re: [Idnet] IDN dedicated session call for case

Dear Albert,
We are interested in this use case and will support the activities in this area.
Best Regards,
Özgü

On 8 August 2017 at 06:52, Albert Cabellos <albert.cabellos@gmail.com<mailto:albert.cabellos@gmail.com>> wrote:
Hi all

Here´s another use-case:

Use case N+2: QoE
        Description: Collect low-level metrics (SNR, latency, jitter, losses, etc) and measure QoE. Then use ML to understand what is the relation between satisfactory QoE and the low-level metrics. As an example learn that when delay>N then QoE is degraded, but when M<delay<N then QoE is satisfactory for the customers (please note that QoE cannot be measured directly over your network). This is useful to understand how the network must be operated to provide satisfactory QoE.
        Process: 1. Low-level data collection and QoE measurement ; 2. Training Model (input low-level metrics, output QoE); 3. Real-time data capture and input; 4. Predict QoE; 5. Operate network to meet target QoE requirement, go to 3.
        Data Format:    Time : [Start, End, Unit, Number of Value, Sampling Period]
                                Position: [Device ID, Port ID]
                                Direction: IN / OUT
                                Low-level metric : SNR, Delay, Jitter, queue-size, etc

        Message :       Request: ask for the data
                                Reply: Data
                                Notice: For notification or others
                                Policy: Control policy

Kind regards

Albert

On Wed, Aug 2, 2017 at 7:12 PM, yanshen <yanshen@huawei.com<mailto:yanshen@huawei.com>> wrote:
Dear all,

Since we plan to organize a dedicated session in NMRG, IETF100, for applying AI into network management (NM), I’d try to list some Use Cases and propose a roadmap and ToC before Nov.

These might be rough. You are welcome to refine them and propose your focused use cases or ideas.

Use case 1: Traffic Prediction
        Description: Collect the history traffic data and external data which may influence the traffic. Predict the traffic in short/long/specific term. Avoid the congestion or risk in previously.
        Process: 1. Data collection (e.g. traffic sample of physical/logical port ); 2. Training Model; 3. Real-time data capture and input; 4. Predication output; 5. Fix error and go back to 3.
        Data Format:    Time : [Start, End, Unit, Number of Value, Sampling Period]
                                Position: [Device ID, Port ID]
                                Direction: IN / OUT
                                Route : [R1, R2, ..., RN]  (might be useful for some scenarios)
                                Service : [Service ID, Priority, ...]  (Not clear how to use it but seems useful)
                                Traffic: [T0, T1, T2, ..., TN]
        Message :       Request: ask for the data
                                Reply: Data
                                Notice: For notification or others
                                Policy: Control policy

Use case 2: QoS Management
        Description: Use multiple paths to distribute the traffic flows. Adjust the percentages. Avoid congestion and ensure QoS.
        Process: 1. Data capture (e.g. traffic sample of physical/logical port ); 2. Training Model; 3. Real-time data capture and input; 4. Output percentages; 5. Fix error and go back to 3.
        Data Format:    Time : [Timestamp, Value type (Delay/Packet Loss/...), Unit, Number of Value, Sampling Period]
                                Position: [Link ID, Device ID]
                                Value: [V0, V1, V2, ..., VN]
        Message :       Request: ask for the data
                                Reply: Data
                                Notice: For notification or others
                                Policy: Control policy

Use case N: Waiting for your Ideas

Also I suggest a roadmap before Nov if possible.

### Roadmap ###
Aug. : Collecting the use cases (related with NM). Rough thoughts and requirements
Sep. : Refining the cases and abstract the common elements
Oct. : Deeply analysis. Especially on Data Format, control flow, or other key points
Nov.: F2F discussions on IETF100
### Roadmap End ###

A rough ToC is listed in following. We may take it as a scope before Nov. Hope that the content could become the draft of draft.

###Table of Content###
1. Gap and Requirement Analysis
        1.1 Network Management requirement
        1.2 TBD
2. Use Cases
        2.1 Traffic Prediction
        2.2 QoS Management
        3.3 TBD
3. Data Focus
        3.1 Data attribute
        3.2 Data format
        3.3 TBD
4. Aims
        4.1 Benchmarking Framework
        4.2 TBD
###ToC End###

Yansen

_______________________________________________
IDNET mailing list
IDNET@ietf.org<mailto:IDNET@ietf.org>
https://www.ietf.org/mailman/listinfo/idnet

_______________________________________________
IDNET mailing list
IDNET@ietf.org<mailto:IDNET@ietf.org>
https://www.ietf.org/mailman/listinfo/idnet