Re: [xrblock] comment on draft-ietf-xrblock-rtcp-xr-qoe-02.txt

[Qin Wu <[e-mail]>]

Hi,
Does "the reference" you refer to for the MoS definitions apply to video?
Are there any standards we can refer to for the concept of "reference" 
and "scaling"?

Regards!
-Qin
----- Original Message ----- 
From: "Alan Clark" <[e-mail]>
To: "Dan (Dan)" <[e-mail]>; "Qin Wu" <[e-mail]>; <[e-mail]>; "Al Morton" <[e-mail]>
Sent: Tuesday, August 28, 2012 1:03 AM
Subject: Re: [xrblock] comment on draft-ietf-xrblock-rtcp-xr-qoe-02.txt


> Hi Dan
> 
> A registry would be needed for the MOS definition - not for the codec type.
> The MOS definitions would need to include both the reference and the scale,
> for example:
> 
> 1. Reference:
> - Narrowband (8kHz) speech
> - Wideband (16kHz) speech
> -....
> 
> Note that you can represent a Narrowband codec on both Narrowband and
> Wideband scales but a Wideband codec has to be represented on a Wideband
> scale.
> 
> Current thought within the industry is to map Superwideband and Fullband
> codecs onto the Wideband scale however if this changes then there may be
> additional references needed
> 
> For video - the issue of MOS reference is still a discussion item.
> 
> 2. Scaling:
> This is a slightly more complicated issue. G.107 was developed in the mid
> 1990's based on subjective test data from the 1980's and early 90's; a
> "good" G.711 call would have a MOS of 4.45 as calculated by G.107.  If you
> look at subjective test data from the later 1990's and 2000's you would find
> that G.711 MOS scores tend to be around 4.1-4.2.  This may be due to
> subjective test reference condition changes or possibly to listeners being
> more used to digital audio and hence more critical.  Also the Japanese
> national standard uses a lower MOS scaling that would give a "good" G.711
> score of 3.8.
> Telchemy uses the terms "ITU Scaled" to mean the scaling used for G.107,
> "ACR Scaled" to mean - consistent with typical subjective test data, and
> "Japanese TTC Scaled" to mean - consistent with the Japanese national
> standard.  All of our customers seem to use the ACR Scaled range as this
> aligns with typical quoted MOS scores for codecs. I'm not aware of ITU-T
> SG12 having worked on this specific topic of scaling, although they are of
> course very aware of the subject matter.
> 
> It would of course be possible to include a reference MOS in the QoE report
> as an alternative to a more complex registry.
> 
> Best Regards
> 
> Alan
> 
> 
> 
> 
> 
> 
> 
> 
> 
> On 8/27/12 10:21 AM, "Dan (Dan)" <[e-mail]> wrote:
> 
>> Hi Alan,
>> 
>> In the meeting in Vancouver a proposal was made to create a IANA
>> registry for the calculation algorithms. Assuming that we go this way
>> (starting from Al's initial list or something derived from it) do you
>> believe that we need a second registry for the codec type?
>> 
>> Dan
>> 
>> 
>>> -----Original Message-----
>>> From: [e-mail] [mailto:[e-mail]] On
>>> Behalf Of Alan Clark
>>> Sent: Thursday, August 16, 2012 5:59 PM
>>> To: Qin Wu; [e-mail]; Al Morton
>>> Subject: Re: [xrblock] comment on
>> draft-ietf-xrblock-rtcp-xr-qoe-02.txt
>>> 
>>> 
>>> Some further work is definitely needed on this draft with regard to
>> the
>>> MOS terminology. The various calculation methods for MOS either
>> already
>>> do, or soon will, provide MOS values for narrowband, wideband, super
>>> wideband and fullband codecs; these MOS values occupy the same range
>> and
>>> hence the QoE block needs to indicate what the MOS reference is.
>>> 
>>> In addition there are national standards (e.g. Japan) that use a
>>> different MOS scaling.
>>> 
>>> Alan
>>> 
>>> 
>>> 
>>> On 8/15/12 10:33 PM, "Qin Wu" <[e-mail]> wrote:
>>> 
>>>> Hi, Al:
>>>> Thank for your general comments, I take some time to look at the
>>>> standards for calculation again.
>>>> please see my replies inline.
>>>> 
>>>> Regards!
>>>> -Qin
>>>> ----- Original Message -----
>>>> From: "Al Morton" <[e-mail]>
>>>> To: <[e-mail]>
>>>> Sent: Thursday, August 02, 2012 5:01 AM
>>>> Subject: [xrblock] comment on draft-ietf-xrblock-rtcp-xr-qoe-02.txt
>>>> 
>>>> 
>>>>> Regarding:
>>>>> http://tools.ietf.org/id/draft-ietf-xrblock-rtcp-xr-qoe-02.txt
>>>>> 
>>>>>>    Calculation Algorithm (CALg):3 bits
>>>>>> 
>>>>>>          000 - ITU-T P.564 Compliant Algorithm [P.564] (Voice)
>>>>>>          001 - G.107 [G.107] (Voice)
>>>>>>          010 - ETSI TS 101 329-5 Annex E [ ETSI] (Voice)
>>>>>>          011 - ITU-T P.NAMS [P.NAMS] (Multimedia)
>>>>>>          100 - ITU-T P.NBAMS [P.NBAMS] (Multimedia)
>>>>>>          101~111 - Reserved for future extension.
>>>>>> 
>>>>>>       G.107 and P.564 and ETSI TS101 329-5 specify three
>> Calculation
>>>>>>       algorithms or MoS algorithms that are used to estimate
>> speech
>>>>>>       quality or conversation quality.  P.NAMS and P.NBAMS specify
>>> two
>>>>>>       MoS algorithms that are used to estimate multimedia quality
>>>>>>       including video quality, audio quality and audio-video
>>> quality.
>>>>> 
>>>>> Specifying the standard for the calculation is a good start, but
>>>>> these standards all have options and input parameters that must
>> also
>>>>> be specified in order to know what the MOS means.
>>>> 
>>>>> For example, we need to know what codec was used, because G.726
>>>>> carries an equipment Impairment factor that limits the upper bound
>> on
>>>>> MOS, while G.711 does not.
>>>> 
>>>> [Qin]: I agree MoS value depends on the codec that is in use. That's
>>>> why, in the current draft, We use 7-bit payload Type in the metric
>>>> block to signal what codec or payload format is  in use for
>> reporting
>>>> interval.
>>>> Please see the definition of Payload Type (PT) in the section 3.2.1.
>>>> 
>>>>> 
>>>>> Some of the input parameters will be assumed (e.g., at default
>>>>> values) while others will be measured, and the distinction between
>>>>> measured and assumed parameters should also be made.
>>>> 
>>>> [Qin]:
>>>> The input parameters may come from Delay metric block, Discard
>> metric
>>>> block, Burst Gap metrics block, JB metrics block, loss concealment
>>>> metric block, concelament seconds metrics block, these metrics
>> blocks
>>>> are defined in other XR Block drafts by XRBLOCK WG .
>>>> 
>>>> In QoE metric block, 4 or 5 pameters are defined, i.e., a.Segment
>> Type
>>>> b.MoS Type c.MoS algorithm d. MoS value In these parameters, a~c can
>>>> be counted as default values while d is counted as measured value.
>>>> The other default values rely on specific computation model defined
>> in
>>>> each standard, e.g., P.564, G.104, P.NAMS.
>>>> Assume each standard only provide one algorithm or one model, we can
>>>> use  the standards name and payload type(i.e.,codec) to identify
>> each
>>>> MoS algorithm used or each MoS model used.
>>>> 
>>>>> As I mentioned, metrics with many optional parameters are somewhat
>>>>> difficult to identify in a simple way, as we found with the IPPM
>>>>> Metrics Registry (which we withdrew when we found it was
>> insufficient
>>>>> to describe the measurement results in an exact way).
>>>>> 
>>>>> Colin Perkins suggested the possibility a registry for the
>>> calculation alg.
>>>>> A registry could provide a single value index to a complicated set
>> of
>>>>> input parameters and other assumptions, and this might work for
>> you.
>>>> 
>>>> [Qin]: Yes,depends on the input parameters and option we got, we can
>>>> calculate the different MoS values however the computation model is
>>>> same and will not be affected by the input parameters and options.
>>>> So maybe rather than registering MoS algorithm, we should register
>>>> assessment model or computation model.
>>>> We can use computation model defined in the standard and payload
>> type
>>>> as single value index.
>>>> 
>>>>> hope this helps,
>>>>> Al
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> xrblock mailing list
>>>>> [e-mail]
>>>>> https://www.ietf.org/mailman/listinfo/xrblock
>>>>> 
>>>> _______________________________________________
>>>> xrblock mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/xrblock
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> xrblock mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/xrblock
> 
> 
> 
> _______________________________________________
> xrblock mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/xrblock
>