Re: [pm-dir] Request for an RFC 6369 review of draft-ietf-xrblock-rtcp-xr-summary-stat-06

[Benoit Claise <[e-mail]>]

Dear pm-dir,

[reducing the list to pm-dir and Gonzallo, XRBLOCK AD]
I filed a DISCUSS on this document.
See 
https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-summary-stat/ballot/#benoit-claise 
for the details.
We should give the right guidelines to performance metrics related draft 
authors.
Maybe we should even mention 
http://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-pdv/ballot/#benoit-claise 
in our WIKI?

Regards, Benoit
> Dear authors,
>
> I'm reviewing version 8 for the IESG call next week.
>> Benoit,
>>
>> Since there is a short term deadline
>> (the Last Call has ended on Feb 1)
>> I'll provide the PM Dir review.
>>
>> Al
>>
>> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>> Since the draft describes reporting formats and (apparently) does not
>> intend to define new metrics, much of 6390 guidance is out of scope.
> This is the key question.
> Do you define new metrics?
> When I see the following (to take just one example):
>
>       Burst Loss Rate: 16 bits
>
>            The fraction of packets lost during bursts since the beginning of
>            reception, expressed as a fixed point number with the binary point
>            at the left edge of the field.  This value is calculated by
>            dividing Packets Loss in Bursts by Total Packets expected in
>            Bursts, multiplying the result of the division by 7FFF, with the
>            maximum value 7FFF, and taking the integer part as follows:
>
>            Packets Loss in Bursts / Total Packets expected in Bursts
>
>            If the measurement is unavailable, the value 0x8000 MUST be
>            reported.
>
> ... it seems to me that you define new metrics, and that therefore the 
> RFC 6390 template should be applied.
> What do I miss?
>
> Regards, Benoit
>> However, see the comments on section 4.1.2 below regarding metric definitions.
>>
>> Comments:
>>
>> Section 1.1 of the Intro says:
>>
>>     This draft defines three new block types to augment those defined in
>>     [RFC3611] for use in a range of RTP applications.
>>
>> ok so far, but the next paragraph is hard to decipher, perhaps a simple
>> numbered list of the new blocks would help:
>>
>>      1. Burst/Gap Loss Summary Statistics Metrics Block
>>      2. Burst/Gap Discard Summary Statistics Metrics Block
>>      3. Frame Impairment Statistics Summary Metrics Block
>>
>> back to the existing text:
>>     The first two block types support the reporting of burst gap loss/
>>     discard summary statistics including packet loss/discard proportion,
>>     mean and variance and belong to the class of transport-related end
>>     system metrics defined in [RFC6792].  These two blocks are intended
>>     to be used in conjunction with information from the Burst Gap Loss
>>     Metrics Block or Burst Gap Discard Metrics Block, and on which these
>>     two block therefore depend.
>> add a reference to [RFC3611] for these two blocks.
>>
>>     The metrics in the Burst Gap Loss block
>>     or Burst Gap Discard Metrics Block can be used independently of the
>>     metrics defined in the first two blocks.
>>
>> That's obvious, because anyone implementing RFC3611 is already doing it.
>>
>> The authors appear to have avoided using "video" as an adjective when
>> referring to video frames throughout the memo. For example:
>>
>>     The third block supports the reporting of detailed statistics for
>>     each frame type, including the number of frames received, lost and
>>     discarded of each frame type in the Group of Pictures (GOP) and
>>     additional data allowing the calculation of statistical parameters
>>     (e.g.,the proportion of each frame type impaired by packet loss and
>>     discard).  The metrics defined in this block belong to the class of
>>     application layer metrics defined in [RFC6792].
>>
>> These are all video frames above, no?  At least say so once.
>>
>> Section 2.1, on Terminology, says
>>
>>      Picture Type
>>
>>        Picture Types used in the different video algorithms are composed
>>        of the Key frame and Derived frames.  The Key frame is also called
>>        a reference frame and used as a reference for predicting other
>>        pictures.  It is coded without prediction from other pictures.
>>        Derived frames are derived from a Key frame using a prediction
>>        algorithm.
>>
>> At first this seems quite generic, but the definition may fail when one slice
>> of the video frame is independently coded, and other slices are predictively
>> coded (this has been done to smooth the video rate). Is any video frame that
>> includes independent or reference coding a Key frame?
>>
>> Section 3.1.2 says:
>>     Burst Loss Rate: 16 bits
>>
>>        The fraction of packets lost during bursts since the beginning of
>>        reception, expressed as a fixed point number with the binary point
>>        at the left edge of the field.  This value is calculated by
>>        dividing Packets Loss in Bursts by Total Packets expected in
>>        Bursts as follows:
>>            Packets Loss in Bursts / Total Packets expected in Bursts
>>
>> How is a burst loss ratio = 1.0 reported?  All the digits are to the
>> right of the decimal place.
>>
>>     Burst Duration Mean:16bits
>>
>>        The mean burst duration is obtained as the quotient:
>>
>>        mean = Sum of Burst Durations / Number of Bursts
>>
>> How is Divide by Zero handled?
>>
>>     Burst Duration Variance:16bits
>>
>>        The variance of the burst duration is obtained using the standard
>>        result:
>>
>>        var = ( Sum of Squares of Burst Durations - Number of Bursts *
>>        mean^2 ) / (Number of Bursts - 1)
>>
>> How is Divide by Zero handled? (e.g., Number of Bursts = 1)
>>
>> Section 4.1.2 says
>>
>>     Number of full frames lost (lost_full_frames): 32 bits
>>
>>        If one frame is completely lost, this frame is regarded as one
>>        lost full frame.  The lost_full_frames is equivalent to the number
>>        of full frames lost in the above sequence number interval.
>>
>> Is this a metric definition, fully lost video frames?
>> How is this event detected so it can be counted?
>> Is there a reference for this metric available?
>>
>>     Number of partial frames lost (lost_partial_frames): 32 bits
>>
>>        If one frame is partially lost, this frame is regarded as one lost
>>        fractional frame.  The value of the lost_partial_frames field is
>>        equivalent to the number of partial frames lost in the above
>>        sequence number interval.
>>
>> Is this a metric definition, partially lost video frames?
>> How is this event detected so it can be counted?
>> Is there a reference for this metric available?
>>
>> -=-=-=-=-=-=-
>>
>>> -----Original Message-----
>>> From:[e-mail]  [mailto:[e-mail]] On Behalf
>>> Of Benoit Claise
>>> Sent: Sunday, January 27, 2013 2:09 PM
>>> To:[e-mail]
>>> Cc:[e-mail]; draft-ietf-xrblock-rtcp-xr-summary-
>>> [e-mail]; Gonzalo Camarillo
>>> Subject: Re: [pm-dir] Request for an RFC 6369 review of draft-ietf-
>>> xrblock-rtcp-xr-summary-stat-06
>>>
>>> Dear all,
>>>
>>> Here is an official request for the RFC 6390 review.
>>> Who would like to volunteer?
>>>
>>> Regards, Benoit
>>>> Hi Benoit,
>>>>
>>>> as suggested by Dan in his PROTO writeup, I would like to request the
>>>> performance metrics directorate to perform an RFC 6390 review of the
>>>> following draft as part of its IETF LC:
>>>>
>>>> https://datatracker.ietf.org/doc/draft-ietf-xrblock-rtcp-xr-summary-
>>> stat/
>>>> Could you please arrange such a review?
>>>>
>>>> Thanks,
>>>>
>>>> Gonzalo
>>>>
>>>>
>>> _______________________________________________
>>> pm-dir mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/pm-dir
>
>
>
> _______________________________________________
> pm-dir mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/pm-dir