Re: [xrblock] Call for adopting xr-quality-monitoring-06 as WG item

[Qin Wu <[e-mail]>]

Hi, Colin:
I understand your concerns. It is painful, but that's the tradeoff we should made.
please see my clarification belows.
----- Original Message ----- 
From: "Colin Perkins" <[e-mail]>
To: "Shida Schubert" <[e-mail]>
Cc: <[e-mail]>
Sent: Tuesday, December 20, 2011 1:13 AM
Subject: Re: [xrblock] Call for adopting xr-quality-monitoring-06 as WG item


> Shida,
> 
> I don't think this draft is ready to adopt:
> 
> - The single-stream metric mixes audio and video MOS into a single metric block, with a grab-bag of MOS type and calculation algorithm parameters. It seems that little thought has gone into designing the >XR block format to prohibit invalid combinations (e.g., MT = MOS-V with CALg = G.107), and there is no guidance on which algorithm is appropriate for which MOS type. 

[Qin]:  We do have guidance to prevent such combination  in three places
a. See MoS Type (MT) field definition, it said:
"
      If MoS type is MoS-LQ and MoS-CQ, the MoS value can be calculated
      based on ITU-T G.107[G.107], ITU-T P.564 [P.564]or ETSI TS 101
      329-5 [ETSI], if the Mos type is MoS-V or MoS-AV, the Mos value
      can be calculated based on ITU-T P.NAMS [P.NAMS]or ITU-T P.NBAMS
      [P.NBAMS]. 
"
b. Also see Mos Type (MT) field defintion, it said:
"
     If MoS Type does not match the MoS algorithm in the report (e.g.,
      specify a voice MOS algorithm for a video quality MOS), the
      receiver should also discard this report.
"
c.  See Calculation Algorithm (CALg) field definition
It distinguish Audio MoS algorithm from video MoS algorithm by adding "(voice)" into Audio MoS algorithm.

>The format seems to follow the RFC 3611 model, throwing together a set of unrelated parameters, rather than the approach documents in the monarch draft.

[Qin]: In this draft, we are not allowed to put "single stream segment","multi-layer segment" and "multi-channel segment" into one XR Block Report. please see the note in the segement i field definition
it said:
"
      Note that in these
      three segment type,any two segment types can not be present in the
      same metric block.
"
That is to say if we choose to use "single stream segment" to report MoS, we will not use "multi-channel segment" or "multi-layer segment" . If we choose to use multi-channel segment, we will not use 
"single stream segment" or "multi-layer segment".

Also we may need to measure MoS-CQ using three different audio MoS algorithms, e.g., 
Using ITU-T P.564 to calculate MoS-CQ1
Using G.107 to calculate MoS-CQ2
Using ETSI TS 101 329-5 to calculate MoS-CQ3

In this case,  MoS-CQ1 and MoS-CQ2 and MoS-CQ3 are all MoS value, but using different audio MoS algorithm, so the question is how to report them, we have two options:
1. Report each MoS-CQ value carrried in the "single stream segment" using three XR Blocks 
2. Report three MoS-CQ vlaues carried in three "single stream segment" using only one XR Block.
I think the option 2 is a better approach, this approach only report MoS value related to one single MoS Type and doesn't put a set of unrelated parameters together and therefore doesn't conflict the rule we set in MONARCH.
However if we also measure MoS-LQ using three different audio MoS algorithms and report MoS-CQ value and MoS-LQ value carried in two "single stream segment" using one single XR Block, that does mean "put a set of unrelated parameters together" and therefore conflict with Monarch.

So my suggestion is to add a sentence to say only one MoS type is allowed to report in one single XR Block.

If you agree with this, I can revise the draft to make this clear. 

> - The multi-layer segment is ill-specified: does it support audio only, video only, or both depending on the Media Type bit? 

[Qin]: multi-layer segment does support both video and audio, 

>Which calculation algorithm makes sense here, since there is no guidance given, and it's possible to specify any algorithm? 

[Qin]: currently we only allow 5 MoS algorithms.since three Audio MoS algorithms have already been standardized and the other two are in standardization progress.

>What is the SSID field? How it is used? 

[Qin]:SSID is sub stream identifier and can be used to identifier each layer if all the layers are carried in the same RTP stream.

>It's not sufficient to simply say that a "NAL unit type is one example", the draft needs to give normative rules for the use of this field.

[Qin]:Agree.

> - The media type, MOS type, and calculation algorithm for the multi-channel segment are ill-specified in similar ways to the multi-layer segment.

[Qin]: Yes, you are right.

> Also, it's not clear that the channel mapping in RFC 3550 Section 4.1 is the only one in use.

[Qin]: Good point, but that's what we have in mind currently.

> - In general, it's not clear to me that including multiple different types of MOS report (single-stream, multi-layer, and multi-channel) in a single draft fits with the monitoring architecture, which encourages small, single-use, XR block definitions. I would suggest that the three different MOS report types would be better defined as three drafts: there is little in common between them.

[Qin]: As I clarified above, these three MOS report types can not be put into one single XR Block.They are excluded between one another. 
But If WG think it is a good idea to merge them together, I am okay with that.

> - The draft provides no meaningful guidelines for registration of new MOS algorithms, and has an inadequate IANA registry.

[Qin]: We do have in the (-06), section 5.4 of  xr-quality-monitoring-06

"
5.4.  New registry of calculation algorithms

   This document creates a new registry to be called "RTCP XR QoE metric
   block - Calculation Algorithm" as a sub-registry of the "RTP Control
   Protocol Extended Reports (RTCP XR) Block Type Registry".  Policies
   for this new registry are as follows:

   o  Entries in the registry are integers.  The valid range is 0 to 7
      corresponding to the 3-bit field "CAlg" in the block.  Values are
      to be recorded in decimal.

   o  Initial assignments are as follows:

      1.  ITU-T P.564 Compliant Algorithm [P.564] (Voice)
      2.  G.107 [G.107] (Voice)
      3.  ETSI TS 101 329-5 Annex E ETSI] (Voice)
      4.  ITU-T P.NAMS [P.NAMS]
      5.  ITU-T P.NBAMS [P.NBAMS]
"
Hope this makes sense.

> Colin
> 
> 
> On 19 Dec 2011, at 11:55, Shida Schubert wrote:
>> This is a call to see whether people are happy 
>> with the latest xr-quality-monitoring-06 draft for addressing 
>> the milestone "QoE Metrics Reporting".
>> 
>> http://tools.ietf.org/html/draft-wu-xrblock-rtcp-xr-quality-monitoring-06
>> 
>> xr-que-00 which used to be an AVT draft also addresses 
>> the same milestone. 
>> 
>> http://tools.ietf.org/id/draft-clark-xrblock-rtcp-xr-qoe-00.txt
>> 
>> The xr-quality-monitoring-06 claims to merge the two 
>> into one. 
>> 
>> Please response to express your option for the followings. 
>> 
>> 1. Adopt the draft-wu-xrblock-rtcp-xr-quality-monitoring-06 as a WG item. 
>> 
>> 2. Not ready to adopt.
>>   * Would be very helpful if you provide your reasonings.
>> 
>> Regards
>>  Shida (as chair)
>> _______________________________________________
>> xrblock mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/xrblock
> 
> 
> -- 
> Colin Perkins
> http://csperkins.org/
> 
> 
> 
> _______________________________________________
> xrblock mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/xrblock